Hyperparameter-Tuning Details in the Planning Advisor

This topic explains the text and underlying calculations that are provided through the hyperparameter-tuning notifications in the Planning Advisor for your demand or demand and supply plan.

The type of the notification in the Planning Advisor for hyperparameter tuning is Summary of hyperparameter tuning. The results are cumulative and cover hyperparameter tuning in previous plan runs and the present plan run.

When you select the notification in the Planning Advisor, these details are displayed:

• The first bulleted point indicates the percentage of forecasting nodes (combinations) for which hyperparameter tuning was attempted and the name of the forecasting profile through which the hyperparameter tuning was done.

  The percentage of forecasting nodes for hyperparameter tuning is determined by the value of the HypertuneSamplePercent forecasting parameter.

• The second bulleted point provides the date of the first plan run with hyperparameter tuning and date of the latest plan run with hyperparameter tuning.
• The third bulleted point provides the number of evaluated forecasting nodes.

  Hyperparameter tuning was either successful or unsuccessful for these forecasting nodes.

• The fourth bulleted point provides the number of forecasting nodes for which the hyperparameter tuning was successful and the percentage of successfully tuned forecasting nodes.
• The fifth bulleted point provides the cumulative improvement of the mean absolute percentage error (MAPE) for the tuned forecasting nodes as a percentage value at the forecast level and in units.

  The percentage for improvement of the MAPE is reflected by a weighted average that's calculated as follows:

  SUM((<measure>Tuned Base MAPE – <measure>Tuned Best MAPE) * InputMeasureHistoryAverage * ForecastBuckets), where the tuned base MAPE and tuned best MAPE are provided by measures, InputMeasureHistoryAverage is the average demand over the relevant proportion of the average history, and ForecastBuckets is the number of forecast buckets for the plan.

  The improvement of the MAPE in units is calculated as follows:

  SUM((<measure>Tuned Base MAPE – <measure>Tuned Best MAPE) * InputMeasureHistoryAverage / SUM(InputMeasureHistoryAverage)), where the tuned base MAPE and tuned best MAPE are provided by measures and InputMeasureHistoryAverage is the average demand over the relevant proportion of the average history.

Note these points:

• The information pertains to disaggregated forecasting nodes.

  For example, consider a scenario in which hyperparameter tuning was done for two aggregated forecasting nodes, one that can be disaggregated to eight forecasting nodes and the other to two forecasting nodes. Hyperparameter tuning was successful for only the first aggregated forecasting node. Therefore, the percentage of forecasting nodes for which hyperparameter tuning was successful is 80.

• The calculations don't include forecasting nodes for which the base MAPE couldn't be calculated, and only the best MAPE could be calculated.