Overview of Hyperparameter Tuning for Forecasting Parameters
You can use hyperparameter tuning to select and use the best values for the forecasting parameters of the forecasting profiles for your demand or demand and supply plans.
Note these points about hyperparameter tuning:
- This feature reduces your need to engage with forecasting-model experts to fine-tune your forecasting profiles and improve their accuracy.
- You need to run a plan with a forecasting profile that's configured for
hyperparameter tuning only once to arrive at the best values for forecasting
parameters. Using historical demand data, Oracle Demand Management will perform
multiple iterations of forecasting with the provided hyperparameter-tuning values to
select the best ones.
Without this feature, you'd have had to run your plan multiple times to test the values for forecasting parameters.
- You can provide your own values for hyperparameter tuning.
If you're unsure about how these values should be provided, you can use the default set of values for hyperparameter tuning.
- Demand Management identifies and saves the set of best forecasting-parameter values for each forecasting node (combination) for your plan. Each set is unique to a forecasting node.
- If the difference between the mean absolute percentage error (MAPE) for the tuned
forecast and MAPE for the base forecast is equal to or over the threshold that you
set, you can save the tuned forecast for a forecasting node in a separate measure
for comparison with the base forecast after the plan run, or you can save the tuned
forecast for the forecasting node in the output measure for the forecasting
profile.
The tuned settings for the forecasting parameters for every forecasting node are saved in a measure for this purpose. This measure also provides the summary of hyperparameter tuning for the forecasting node and the level of the forecasting node in the forecast tree.
- The tuned forecast and base forecast are calculated on the percentage of your historical demand data that you specify for holdback validation (out-of-sample testing).
- If the forecasting parameters for a forecasting node are successfully tuned, you can use nodal tuning to automatically apply these settings to the forecasting node in later plan runs.
- You can select the percentage of forecasting nodes in your plan for which
hyperparameter tuning should be done in each plan run.
During each plan run, a different set of forecasting nodes is selected for hyperparameter tuning so that all the forecasting nodes in the plan are covered in time.
- A set of forecasting parameters is available in the Hyperparameter Tuning category
for use with this feature.
For information, see the topic titled Forecasting Parameters for Hyperparameter Tuning.
- A set of predefined measures is available for use with this feature. If your
forecasting profile doesn't use a predefined output measure, a set of measures will
be created for hyperparameter tuning on the basis of your predefined output
measure.
For information, see the topic titled Measures for Hyperparameter Tuning.
- You can change some settings for hyperparameter tuning at the plan level and
override the corresponding settings at the forecasting-profile level.
Thus, multiple plans that use a common forecasting profile for hyperparameter tuning can each have a set of unique settings for hyperparameter tuning. For information, see the topic titled Specify Hyperparameter-Tuning Values at the Plan Level.
- You can configure the Planning Advisor for display of hyperparameter-tuning
notifications.
You can also configure a page with tables and graphs for analyzing the results of hyperparameter tuning, and you can open this page from the hyperparameter-tuning notifications in the Planning Advisor.
For information, see the topics titled How You Set Up Hyperparameter-Tuning Notifications in the Planning Advisor and View Hyperparameter-Tuning Notifications in the Planning Advisor.
- To reduce the plan runtime, Demand Management uses a random-walk algorithm to reduce
the number of tested combinations for forecasting-parameter values. You can also
disable some forecasting methods during hyperparameter tuning.
For information, see the topic titled How You Improve the Performance of Hyperparameter Tuning.
Limitations of Hyperparameter Tuning
- You can perform hyperparameter tuning for only forecasting profiles that use Bayesian machine learning.
- You can configure hyperparameter tuning for only user-defined forecasting profiles.
- You can use a forecasting profile that's configured for hyperparameter tuning in only a demand or demand and supply plan.
- You can't use hyperparameter tuning along with the feature for optimizing
the parameters for forecasting methods.
For more information, see the topic titled How You Optimize Parameters for Forecasting Methods.