evaluation
mae(y_true, y_pred)
Return mean absolute error (MAE).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
mape(y_true, y_pred)
Return mean absolute percentage error (MAPE).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
mase(y_true, y_pred, y_train, sp=1)
Return mean absolute scaled error (MASE).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
y_train |
DataFrame
|
Observed training values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
mse(y_true, y_pred)
Return mean squared error (MSE).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
overforecast(y_true, y_pred)
Return total overforecast.
Overforecast (positive forecast bias) is the difference between actual and predicted for predicted values greater than actual.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
rank_fva(y_true, y_pred, y_pred_bench=None, scoring=None, descending=False)
Sorts point forecasts in y_pred
across entities / time-series by score.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Panel DataFrame of observed values. |
required |
y_pred |
DataFrame
|
Panel DataFrame of point forecasts. |
required |
y_pred_bench |
DataFrame
|
Panel DataFrame of benchmark forecast values. |
None
|
scoring |
Optional[metric]
|
If None, defaults to SMAPE. |
None
|
descending |
bool
|
Sort in descending order. Defaults to False. |
False
|
Returns:
Name | Type | Description |
---|---|---|
ranks |
DataFrame
|
Cross-sectional DataFrame with two columns: entity name and score. |
rank_point_forecasts(y_true, y_pred, sort_by='smape', descending=False)
Sorts point forecasts in y_pred
across entities / time-series by score.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Panel DataFrame of observed values. |
required |
y_pred |
DataFrame
|
Panel DataFrame of point forecasts. |
required |
sort_by |
str
|
Method to sort forecasts by:
|
'smape'
|
descending |
bool
|
Sort in descending order. Defaults to False. |
False
|
Returns:
Name | Type | Description |
---|---|---|
ranks |
DataFrame
|
Cross-sectional DataFrame with two columns: entity name and score. |
rank_residuals(y_resids, sort_by='abs_bias', descending=False)
Sorts point forecasts in y_pred
across entities / time-series by score.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_resids |
DataFrame
|
Panel DataFrame of residuals by splits. |
required |
sort_by |
str
|
Method to sort residuals by: |
'abs_bias'
|
descending |
bool
|
Sort in descending order. Defaults to False. |
False
|
Returns:
Name | Type | Description |
---|---|---|
ranks |
DataFrame
|
Cross-sectional DataFrame with two columns: entity name and score. |
rmse(y_true, y_pred)
Return root mean squared error (RMSE).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
rmsse(y_true, y_pred, y_train, sp=1)
Return root mean squared scaled error (RMSSE).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
y_train |
DataFrame
|
Observed training values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
smape(y_true, y_pred)
Return symmetric mean absolute percentage error (sMAPE).
Use third version of SMAPE formula from https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error to deal with zero division error
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |
underforecast(y_true, y_pred)
Return total underforecast.
Underforecast (negative forecast bias) is the difference between actual and predicted for predicted values less than actual.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y_true |
DataFrame
|
Ground truth (correct) target values. |
required |
y_pred |
DataFrame
|
Predicted values. |
required |
Returns:
Name | Type | Description |
---|---|---|
scores |
DataFrame
|
Score per series. |