multi_objective

Module with functions to compute, compare, and optimize multi-objective forecasts.

mae(y_true, y_pred)

Return mean absolute error (MAE).

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.

mase(y_true, y_pred, y_train, sp=1)

Return mean absolute scaled error (MASE).

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required
y_train	DataFrame	Observed training values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.

mse(y_true, y_pred)

Return mean squared error (MSE).

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.

overforecast(y_true, y_pred)

Return total overforecast.

Overforecast (positive forecast bias) is the difference between actual and predicted for predicted values greater than actual.

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.

rmse(y_true, y_pred)

Return root mean squared error (RMSE).

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.

rmsse(y_true, y_pred, y_train, sp=1)

Return root mean squared scaled error (RMSSE).

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required
y_train	DataFrame	Observed training values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.

score_backtest(y_true, y_preds, agg_method=None)

Return DataFrame of forecast metrics across entities.

Metrics returned:

• MAE
• MASE
• MSE
• Overforecast
• RMSE
• RMSSE
• SMAPE
• Underforecast

Note: MAPE is excluded to avoid potential divide by zero errors. We recommend looking at SMAPE instead.

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_preds	DataFrame	Stacked predicted values across CV splits. DataFrame contains four columns: entity, time, target, "split".	required
agg_method	Optional[str] = None	Method ("mean", "median") to aggregate scores across entities by. If None, forecasts in overlapping splits are weighted equally, i.e. no aggregation is applied.	None

Returns:

Name	Type	Description
scores	DataFrame	DataFrame with computed metrics column by column across entities row by row.

score_forecast(y_true, y_pred, y_train)

Return DataFrame of forecast metrics across entities.

Metrics returned:

• MAE
• MASE
• MSE
• Overforecast
• RMSE
• RMSSE
• SMAPE
• Underforecast

Note: SMAPE is used instead of MAPE to avoid potential divide by zero errors.

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required
y_train	DataFrame	Observed training values.	required

Returns:

Name	Type	Description
scores	DataFrame	DataFrame with computed metrics column by column across entities row by row.

smape(y_true, y_pred)

Return symmetric mean absolute percentage error (sMAPE).

Use third version of SMAPE formula from https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error to deal with zero division error

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.

summarize_scores(scores, agg_method='mean')

Given a DataFrame of forecast metrics, return a dataclass of metrics aggregated by agg_method.

Parameters:

Name	Type	Description	Default
scores	DataFrame	DataFrame of scores. N rows of entities by M columns of metrics.	required
agg_method	str	Method ("mean", "median") to aggregate scores across entities by.	'mean'

Returns:

Name	Type	Description
metrics	Metrics	Dataclass of scores aggregated across entities.

underforecast(y_true, y_pred)

Return total underforecast.

Underforecast (negative forecast bias) is the difference between actual and predicted for predicted values less than actual.

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required

Returns:

Name	Type	Description
scores	DataFrame	Score per series.