PlanIQ - Algorithm selection by item: mix and match your forecast

AnaplanOEG
AnaplanOEG
edited December 2022 in Best Practices

What is algorithm selection by item?

Algorithm selection by item is the use of more than one algorithm to produce optimal forecasts across multiple items. Instead of using forecasts generated by a single algorithm for all items in the historical data module, where the algorithm is selected based on best overall performance – multiple algorithms are selected based on forecast accuracy by item (or groups of items).

Each forecast model trained in PlanIQ allows for a single algorithm to be used in order to generate forecasts for the items in the historical dataset. In order to use more than one algorithm, multiple forecast models and actions should be configured and executed. Then, the accuracy of the forecasts should be calculated per algorithm and per item; The algorithm which produces the most accurate forecast is then selected for each item, resulting in the final forecast.

Evaluation

First, define a backtesting window, which is a past period of time that is included in the forecast horizon. Doing so would allow for forecasts to be evaluated against past observations (actuals). Then, calculate an accuracy metric of choice (e.g. RMSE, MAPE) at the individual item level over the backtest period. Finally, compare the metric across all algorithms to identify the best performing algorithm for each item in the back testing window.

Requirements & Considerations

All algorithms can theoretically be used for algorithm selection by item, though the number of records and/or the desire to use related data or attributes may determine which algorithms can be leveraged for this purpose. For instance, data collections must contain at least 300 historical values to meet the neural network machine learning algorithms (DeepAR+ and CNN-QR) data requirements. In another example, if related data should be taken into account for forecasting, then only Prophet and the neural network machine learning algorithms would apply.

The level at which the algorithm selection should be made can be typically determined by the data; Selection at the individual item level will provide more granularity but could lead to overfitting compared to group/category level algorithm selection.

How To

Below is an example of an algorithm selection by item process:

  1. Create a data collection
  2. Create forecast models based on the new data collection for all algorithms that the dataset captured in the data collection is suited for and you wish to evaluate
  3. Create a forecast action for each forecast model created
  4. Calculate the forecast accuracy by item for each algorithm
    1. For each algorithm, calculate the forecast metric for each item
    2. Compare the forecast accuracy across the different algorithms for each item
    3. Select the best performing algorithm for each item based on forecast accuracy
  5. Create a line item populated with the final selected forecasts for each item

Got feedback on this content? Let us know in the comments below.

Contributing authors: Nitzan Paz, Christophe Keomanivong, Frankie Wolf, Timothy Brennan, Andrew Martin, and Evgenya Kontorovich.

Tagged:

Categories