Historical: trend (ETS, ENSEMBLE)
Historical: linear_trend (ENSEMBLE, PROPHET, MVLR)
Historical: exponential_downwards_trend (MVLR, ENSEMBLE)
Historical: exponential_upwards_trend (MVLR, ENSEMBLE)
Historical: Auto Regressive 1(SARIMAX, ENSEMBLE)
Historical: Auto Regressive 2 (SARIMAX, ENSEMBLE)
Historical: Auto Regressive 3 (SARIMAX, ENSEMBLE)
Historical: Seasonal Auto Regressive 24 (SARIMAX, ENSEMBLE)
Historical: Seasonality_Yearly (MVLR, ENSEMBLE, PROPHET, ETS)
Related: Z__related_data_column lagged 1 month (MVLR, SARIMAX, ENSEMBLE, PROPHET)
Related: related_data_column_4 (MVLR, SARIMAX, ENSEMBLE, PROPHET, TIMESFM)
dayofweek (LIGHTGBM)
dayofmonth (LIGHTGBM)
dayofyear (LIGHTGBM)
distance_from_ts_start (DEEP AR)
metadata_2_g1_list__code__0 (LIGHTGBM)

Here are my questions:
1. Why are multiple trend types generated (linear, exponential up/down, etc.)?
If exponential_downwards_trend = -0.22 and exponential_upwards_trend = 0.22 for the same item, how should I interpret that? Do they cancel each other?

2. What is the difference between Historical: Auto Regressive 1, Historical: Auto Regressive 2, Historical: Auto Regressive 3, and Historical: Seasonal Auto Regressive 24?
Is Auto Regressive capturing short-term lag memory while Seasonal AR captures longer cycles?

3. I see metadata fields being used in LIGHTGBM.
Anapedia mentions metadata usage for DeepAR. Is metadata also used as categorical features in LightGBM?

4. How does a list item code have an effect in the result? Are they encoded categorically, or could they be interpreted numerically?

Any clarification or official documentation would be helpful.

Seyma Tas

If you missed our January 29, 2026 platform release event, the recording is now available!

In this webinar, we highlighted the most recent feature releases and how to take advantage of these in your environment. Our expert-led session demonstrated new features and innovations, provided a technical deep-dive and “how-to”, followed by a question and answer session with the audience.

Topics covered include:

• Modeling in Polaris: Explore how on-demand calculation can positively impact model performance.
• Dimensionality in Polaris: Learn how builders and users can pivot up to eight nested dimensions in a grid view.
• Planning experience: Check out new features that allow page builders to elevate the design of UX pages and present information side-by-side for end users.
• Forecaster: See the latest enhancements to Anaplan Forecaster, a next generation forecasting capability embedded in the Anaplan platform.

Recording

Q&A

• Where is On-demand Calculation (ODC) defined? Anapedia
• Which summary methods are supported with ODC? All summary methods are supported except Formula, Ratio, and None.
• Is data input still supported for combined grids? Yes. Data input is still available for combined grids if you are surfacing leaf levels in the grids.
• Does filtering work on the rows and columns in a combined grid? Yes.
• If 3 grids were added to a combined grid, in what order are they presented? The grids are presented in the order in which they were added to the combined grid.
• Does Anaplan Forecaster only use historical information? Or can you combine inputs in addition to history, i.e., revenue expectations to increase by x%? Forecaster does support external drivers to the forecast. Data collections have to include historical data; related data and attributes are optional. Please see Anapedia for more information.

Note: There may be additional Q&A included in the recording that was answered live.

Catch up on recent releases:

• October
  • Release notes
  • Supplemental blog
• November
  • Release notes
  • Supplemental blog
• December
  • Release notes

Follow along for future events!

Stay up to date with all platform release announcements by subscribing to the platform release page on the Anaplan Community, and all platform events by subscribing to the events page. Look for the “bell” icon on the page to subscribe.

Customers are frequently wondering what can be used as related data. When thinking about related data, we can think about different types of “internal” time dependent inputs: 

• Historical and future promotions 
• Stockouts 
• Price fluctuations 
• Holidays and special events 

Actual data may be different dependent on a specific use-case, for example when forecasting cloud or software license costs, related data can be in form of number of engineers or number of employees. 

External “external” factors are also use-case dependent and include, among other things: 

• Weather 
• Traffic and mobility data 
• Currency exchange rates 
• Interest rates 
• Commodity prices 
• Consumer price indexes 

Additionally, it may be helpful to provide additional related information in order to help algorithms learn from the past better. 

Lastly, it may be useful to provide related data as an additional input even if historic seasonality may be sufficient to predict similar future behavior. We will further explore this scenario in one of the examples in this article. 

Before we dive into the examples, a reminder that most Forecast Algorithms are looking at several types of related data: 

• Related data provided by the user – when provided. 
• Holidays calendar provided by the user in form of related data or in form of built-in Forecaster holiday calendars. 
• Historical behavior of the particular item (time series) – historical trend and seasonality components. 

On top of this information, Forecaster performs automated analysis of the data in order to identify leading and lagging indicators and all that information is the used as part of feature selection where the algorithms pick and choose only relevant information in order to use it as part of the forecast.  

From the perspective of this article, we need to remember that algorithms are looking at both your historical data, as well as related data and historical behaviors. 

Another thing to remember is that it is important to provide related data not only for the known past but also into the future. Most of Forecaster algorithms ignore related data if it’s not provided in the future for the entire length of forecast horizon. 

It is not difficult to provide future information for events that are “internal” to your business, for example promotions or changes in price. It is more difficult to do it when external factors are being used. In this case, our recommendation is to first forecast the external factor and then use the forecast as input into the main forecast. Even if your forecast is only directionally accurate it will allow you to generate a forecast based on an assumption that you control and it also allows you to perform what if analysis where you can adjust the external data forecast and observe the impact of the changes on the main forecast. 

Dataset 

For the discussion in this article I used the following champagne sales data (as a single time series) provided by this Kaggle dataset*. It covers monthly sales of champagne between 1964 and 1972. I have modified the timestamps in the data in order to “move it” into present and used most recent 5 years of this data (from 2018 to 2023). 

The experiments 

When looking at historical data it’s evident that the data has a clear yearly seasonality pattern with sales peaks around December and drops of sales around August. 

It can be observed that overall trend is positive with a peak in end of 2021 and then there’s a certain drop or downward trend. In general, 2021 had higher actuals than years before and after. This is also evident in the Seasonality and Trend analysis data provided by Forecaster. 

We will explore and observe impact of several experiments: 

• Forecasting based on historical data only. 
• Forecasting based on both historical and related data. 

During our experiments I started the forecasts from January 2021 and used actuals until December 2020. The forecast horizon for this experiment was 12 months. During several iterations I added 12 months of actuals and projected additional 12 months of forecast. 

Experiments with MVLR 

We will observe last period of forecast, between January 2023 and December 2023. When based on historical data only MAPE of P2 forecast (0.5 quantile) was 10.72% which is pretty accurate. Looking at explainability, we can see that the algorithm rightfully decided that yearly seasonality identified by Forecaster is the most impacting factor in our forecast. Second impactful factor is “exponential upwards trend” that is identified by Forecaster. 

Another experiment that I performed included related data that I generated. I used two types of related inputs: 

• Holidays – indicating the holiday peaks in December and drops in August. In this case I used numeric values where regular months had a value of 1, December had a value of 100 and August a value of ‘-20’. The numbers are arbitrary. 
• Covid19 – indicating months of COVID pandemic. Months of 2020 and 2021 had value of 100 and other months had value of 0. 

When using this related data to perform same forecasts I was able to reduce MAPE rate of P2 (0.5 quantile) to 5.21% which is a significant improvement. 

Even visually, we can see that P2 curve is much closer to actuals during 2023 period. In terms of explainability, seasonality is still the most impacting factor but we can see much bigger impact of other inputs including the impact of COVID. It is possible that introduction of COVID related time series is making the algorithm more sensitive to trend changepoint that happened after 2021. 

Experiments using Prophet 

When forecasting based on the same data using Prophet we see a similar behavior. When looking at MAPE P2 forecast (0.5 quantile) based on historical data only between January 2023 and December 2023 the value of MAPE is quite high, it’s ±44%. 

The most important factor here is yearly seasonality but we can see that the forecast in 2023 is pretty much on the same level as in 2021 and this is causing the high MAPE value. In this case, the customer could decide to use values between P1 (0.1 quantile) and P2 (0.5 quantile) as forecast baseline. 

When introducing related data as specified above, MAPE of P2 is reduced to 27%. In this case, impact of August “lows” and December “highs” is higher, and it seems that the algorithm is giving related holidays data more weight than to seasonality alone. Additionally, it seems that COVID related data also helps the algorithm to be more sensitive to trend changepoint and by that reduces the over-forecasting in the majority of 2023 months. The customer could decide to use values between P1 (0.1 quantile) and P2 (0.5 quantile) as forecast baseline based on the specific use-case. 

Comparing MVLR and Prophet 

In this experiment, we saw that for 2023 MVLR has outperformed Prophet both with and without related data. However, if we look at 2022 data, Prophet was more accurate than MVLR. In both cases the algorithms over-forecasted. This is probably because 2021 actuals were much higher than previous years and both algorithms clearly tried to continue the trend. However, while MVLR probably gave more “importance” to trend vs Prophet that was more “reserved” in its predictions and eventually provided more accurate forecast. 

Summary 

In this article we have explored various impacts that related data can have on your forecast. Few points can be highlighted: 

• If your historical data has strong seasonal patterns, it may still be useful to amplify the impact of those patterns with an additional related data (just like I did with holidays in this example) – in case of several patterns in data, where both seasonality and trend exist. 
• Even though we’re not in COVID era anymore, some of your historical data may be impacted by COVID and it may make sense to flag months / weeks impacted by COVID and lockdowns in order to indicate to the engine that certain parts of history were not the norm. 
• Different algorithms may perform differently at different time periods. Some of our customers choose a winning algorithm based on past performance across several time periods. This helps them insure that over time selection of winning algorithm is more stable and predictable. 

The experimentation was based on MVLR and Prophet specifically because we wanted to explore impact of related data. In larger data collections you are invited to try out DeepAR, LightGBM, TimesFM, or Ensemble. 

*Note: you must join the Forecaster group to view the starred articles. Feel free to join! 

Anaplan's next quarterly platform release webinar will be held January 29, 2026 at 8 a.m. Pacific Time. During this session, we will explore the latest features so you can optimize your Anaplan experience.

Our expert-led session will equip you with a practical guide and live demonstrations to help you get the most out of new features and innovations. Get your questions answered live during Q&A.

Features to be demo'd:

• Modeling in Polaris: Explore how on-demand calculation can positively impact model performance.
• Dimensionality in Polaris: Learn how builders and users can pivot up to eight nested dimensions in a grid view.
• Planning experience: Check out new features that allow page builders to elevate the design of UX pages and present information side-by-side for end users.
• Forecaster: See the latest enhancements to Anaplan Forecaster, a next generation forecasting capability embedded in the Anaplan platform.

Can't make it? Register for the event and a recording will be sent to all who register. A recap and recording will also be posted on Community.

Register here.

Note: You have the option to 1) use the automatically-generated backtest results exported from Forecaster, or 2) build a custom process to evaluate forecast performance. The following sections describe how and when to use either approach.  

Out of the box: Forecaster automated backtest period and accuracy metrics  

Forecaster automatically generates some forecast accuracy metrics as soon as you build a forecast model. The automated metrics are there to help quickly check overall accuracy, especially when building multiple forecast models in succession with the same data set. These metrics include RMSE, MAPE, MAE, MAAPE, sMAPe, and MASE, and an overall model quality metric, which are described in more detail in Anapedia Understand Advanced Metrics.

The backtest window for these automatically generated metrics is the length of the Forecaster forecast horizon. The automated metrics are calculated and aggregated across all items and all periods in the backtest window.  

However, you likely want to know the accuracy period and item to compare relative performance across items and over time, which is not captured in the automatically generated metrics.  

Therefore, it is highly recommended that you calculate these metrics within Anaplan to ensure the forecast performance evaluation is aligned to the business use case, especially if you are dealing with multiple items that may perform differently depending on the underlying data. This way you can ensure the model forecast accuracy is consistent across time periods and items.  

To be able to drill down to the forecast performance at the item time period level, you can import the same backtest results that are used to generate the automated accuracy metrics, and then calculate variance between forecast versus actuals (backtest results). 

Custom built: User defines back-test window and timeframe   

Instead of relying on the automatically-generated backtest results, you may also create your own backtest window and calculate forecast accuracy metrics yourself in Anaplan. There are several reasons to do this. First, you may want to customize the length of the backtest window to be shorter or longer than the automatically-generated results. However, keep in mind we recommend using a backtest window that is the same length as the forecast horizon as a baseline. That way you can evaluate performance across the entire forecast window. Especially if there are major seasonal or cyclical trends that impact business, you should try to capture the full cycle in the backtest window.  

Another reason to construct your own backtest window is because the automated back test results are created at the time the model is built and not updated thereafter. Therefore, we recommend that you independently calculate accuracy metrics on an ongoing basis for reporting purposes until the point that you can use the real Forecaster forecast results instead of the backtest simulation results.  

After obtaining the backtest results, the next step is to calculate accuracy metrics based on a comparison of forecast versus actuals (backtest results). For more guidance on how to select which accuracy metrics are best suited to the business use case, see Anapedia Understanding Advanced Metrics.

How to build a backtest window for forecast accuracy testing:  

1. Identify the period that covers important seasonality while still conserving sufficient historical data to forecast; for example, the last year.  
2. Build a Forecaster module filtered view as if it were a given past date; for example, as if it were a year prior. Save a new export action based on a saved filtered view and create a new Forecaster data collection. 
3. Move forward in single period increments to simulate a real-world forecast. For example, monthly, quarterly forecasts for the past year, or weekly forecasts over a period.  

• You can do this manually by adjusting the saved view referenced by the data collection forecast action. 
• Or, you can partially automate this by creating a module that manages a time filter that is referenced in a saved view.  

1. You can use a similar approach to monitor performance on an ongoing basis. 
2. Now that you have forecast versus actuals (backtest results), you can compare multiple accuracy metrics.  
3. Review the different considerations for selecting which accuracy metrics are best suited to your use case. It is ideal to test multiple metrics.
4. If possible, translate into real-world units such as quantity or dollar value. This will provide the best approximation of the most likely cost of accuracy for different scenarios. 

Check out the latest Anaplan feature updates and enhancements in our November 2025 official release notes. The information below offers supplemental information to that post.

Intelligence

• Anaplan Forecaster: General enhancements
  Anaplan Forecaster now supports list subsets for the item ID list and custom time, list format line items in attributes data, and data residency in a new AWS region (AE1). Additionally, weekly data no longer has to start on Mondays.

Planning Experience

• Polaris: On-demand Calculation
  On-Demand Calculation is now available in Polaris, which allows the engine to defer significant amounts of aggregation until the data is 'viewed', such as in an export or by a user in a UX page. This enhancement may positive impact model open times, recalculation times on data input, and model size at open.
• Anaplan XL (version 2511): General enhancements
  Multiple enhancements for Anaplan XL have been released, including: The Excel workbook status bar for grids, freeforms, and table query objects now displays information about each query when a query cell is selected; the Activate Workbook button now only appears when an Anaplan XL workbook is disabled; the Extras menu now includes links to Anaplan Community and video tutorial pages; and the context menu now includes an option to add a workbook slicer.
• Anaplan XL (version 2511): Freeform reports
  There is now an option to repeat member labels when drilling on a nested dimension, improving readability and context.

What's next?

Applications

• More flexibility with Saved Views as data sources
  You can select a Saved View as the data source when pulling information from an Anaplan model — no longer limited to using entire modules. This gives you more control and precision when integrating data.
• Application Framework available to all customers
  The ability to implement Anaplan Applications using the Application Framework will soon be available to all customers. Once an Anaplan Application is purchased and users are assigned the Application Owner role, the Application Framework will automatically become visible. The Application Framework provides a structured, guided way to set up and manage Applications. With access to the Application Framework, customers can get their planning processes up and running faster and unlock more value from Anaplan Applications.

Anaplan Data Orchestrator (ADO)

• Enhancements to Data Preview
  An enhancement has been made to the preview functionality within a model. Both Source & Derived Datasets can be previewed using a new Preview capability. For Source Datasets this opens automatically when the dataset is selected from the inventory page, whereas for Derived Datasets this is accessed through the preview option on the ellipsis menu on the inventory page. This allows:
  • The data that is displayed to be filtered using a Boolean expression based on values in the dataset
  • The data that is displayed to be sorted based on the values in a number of columns from the dataset
  • Displays up to 10,000 rows of data
  • Displays the total number of rows in the dataset and the number that meet the filter conditions
  • Allows a transformation view to be created based on the filter definition
  • Allows the data to be exported to a CSV (this should be the selected data but it is currently exporting all the data and will be corrected soon.)
    Note that it is not currently possible to save the preview definition and return to it later.
• Filter for process history
  Filters can be defined for the process history in a similar manner to other inventory pages.
  This is applied to the entries that meet the selection and is also applied to the rows that are included in the export.

Enterprise Experience

• Hierarchy Chart pagination
  We have added the ability to enable pagination on hierarchy charts so it Is easier to see the hierarchy structure on a page. See example below of a paginated Org Chart and the setting to enable it.
• Nest more than 3 dimensions in an axis
  Page builders and end users will now be able to use pivot to nest more than 3 dimensions in an axis. Unlocking deeper data exploration and more flexible analysis for highly dimensioned views. Important note: This feature is for Polaris models only.
• Grid rich text formatting on mobile
  End users on mobile can now apply rich text formatting to editable text in grids and field cards to add some commentary and highlight key areas. This includes the following formatting: Bold, Italics, Underline, Strikethrough, Text alignment, Bullet and Numbered list.

Forecaster

• Run forecasts where work happens
  Forecast Actions can soon be triggered directly from UX pages using a configurable button. This means business users won’t need to navigate away from their dashboards or rely on back-end processes to run forecasts - helping teams embed forecasting faster and more directly into day-to-day decision-making.
• Test and deploy forecasts with confidence
  We’re introducing more flexible repointing for forecast results and explainability, allowing outputs to be written to a different model or workspace than the one used for configuration. This lets teams safely test and validate forecasts in non-production environments before deploying to production - without duplicating setup work - streamlining promotion cycles and reducing risk.

Previous release notes

• October 2025 platform releases and what’s next
• Combined Grids is now live!
• Recording now available! October 29 platform release event
• September 2025 platform releases and what’s next

Thank you!

This article serves as a guide to help Forecaster users deal with situations in which their datasets include missing values or 'empty' cells.

It's important to differentiate between a true zero and a missing value. A dataset with many missing values (a sparse dataset) is different than a cold start scenario where little or no data exists because a certain product is new to the market.

Many missing values in a dataset may impair ‌forecast accuracy. This is especially true for more recent (later) data in the time series. Our recommendation is to have less than 30% of missing values per time series (per item). Forecaster limits the missing values per item to 50% in the historical data. If a dataset contains more than 50% missing values, Forecaster displays a message indicating that too many values are missing.

Forecaster assumes that datasets that originate in Anaplan modules with records set to zero are true zeros and will be treated as such. In addition, in cases where a custom time dimension is used (i.e., where the time dimension is based on a list of timestamps), records with missing timestamps will be treated as zeros as well (rather than missing).

There are multiple ways to deal with missing values. Several options are:

• Use the ‘___exclude_value’ column so that missing values are not interpreted as zero (see Anapedia for more details). If the value is 1 in the ‘___exclude_value’ column, Forecaster will automatically fill it in with the mean of the values around it.
• Manually review and fill in missing values.
• Aggregate the data by using reducing its frequency (e.g., instead of a daily frequency, aggregate the data to a weekly level).
• Aggregate multiple distinct items to a category of items based on item hierarchy or other dimensions such as location (for example, combining multiple cities to a state level).

One more way to deal with sparse datasets is to use robust forecasting algorithms such as LightGBM, TimesFM, and DeepAR. These algorithms may take longer to train and typically require more historical data but can better handle sparsity in time series data. In cases where it's not clear which algorithm to choose, it's best to choose Ensemble. Forecaster will then look at different algorithms and choose the one that gives the best forecast for each item.

While time series data often have many missing values, Forecaster can help you get the most out of historical data to make accurate forecasts.

• What is Forecaster?
• Who is it for?
• Use-case examples
• To get started
• Once you get more comfortable
• Deep dive on algorithms
• Support

Are you interested in starting time series forecasting with Forecaster on Anaplan, but don’t know where to start? You have come to the right place!

This article is the pathway to all things Forecaster.

First things first

What is Forecaster?

Forecaster is an easy-to-use time-series forecasting tool directly integrated with the Anaplan platform. It allows users to accurately predict future values based on historical data, attributes, and other related data. Forecaster enables planners and business stakeholders to make data-driven decisions, backed by advanced machine learning methods.

Who is it for?

Use-case examples

Demand planning
Improve the accuracy of your demand forecasts and quickly get to a consensus demand plan with collaborative planning.

Marketing and promotional spend optimization
Improve the ROI and develop more effective marketing programs programs by understanding the drivers that impact demand uplift.

Financial planning
Quickly generate OpEx forecasts and analyze new scenarios, variances, and opportunities so board members can speed decisions.

Assortment planning
Reduce SKU proliferation by designing the optimal assortment for each segment, channel, and category based on internal and external signals.

Workforce planning
Reduce operating costs and plan for the right levels of staffing and skills by accurately predicting volume and variety of tasks.

Inventory optimization
Reduce inventory carrying costs by optimizing inventory levels across the supply chain without affecting service levels.

Now, let's go under the hood

To get started

We recommend taking the Anaplan Academy course and reading these articles:

1. Anapedia step-by-step technical documentation
2. Design and build your item list for forecasting
3. Probabilistic forecasting using forecast quantiles
4. How to use item attributes to refine your forecast

Once you get more comfortable

Let’s dig a little deeper with the following:

1. Considerations Before Starting to Forecast
2. Introduction to forecast evaluation
   

Deep dive on algorithms

If you are interested in learning more about the different algorithms available in Forecaster, please look at this article:

• Deep dive on the algorithms under the hood

Support

Finally, to support you best in your experience, please refer to the Support self-service article

• Anapedia Forecaster

Got feedback on this content? Let us know in the comments below.

Below are the main topics to consider before starting to forecast with Forecaster. Please consider the business problem you are trying to solve as you work your way through the list. 

Forecast Horizon 

When deciding on the length of the forecast horizon (the length of time into the future for which a forecast is generated), avoid simply basing it on the allowed horizon for the chosen algorithm. While it is a good starting point, be sure to consider both the business use case and the available data. Start with the forecast horizon needed to achieve your goal and work from there. If projections for one year out are needed, verify that there is enough historical data to support it. If using related data, it's recommended to include forward-looking data that covers the forecast horizon. Reference the forecast horizon guide on Anapedia to learn more about the supported horizons for each algorithm. 

Related Data and Attributes 

Related data (variables that correlate with or affect the forecasted time series) and attributes (categorical information that groups historical data by their shared characteristics) are valuable information that could improve the quality and accuracy of forecasts. Before deciding whether to use such data as part of your forecast, consider the following. 

Data Value 

The most important thing is the data itself. Consider if the data adds information that is relevant for the forecast. For instance, related data which displays patterns that correspond to historical data could lead to more accurate forecasts. However, having constant or near-constant values in related or attribute data would have limited value and might limit the algorithms you can use. Forecaster's deep learning algorithms, TimesFM, LightGBM, and DeepAR, could also be negatively impacted by too few or too many unique attribute values. The number of related data or attributes also matters, where more isn't always better.

Mean and Variance of Related Data 

The nature of mean and variance of the related data is also worth considering. Is the entire dataset available representative of your business now and in the future? Old patterns do not necessarily represent more recent behavior of the forecast items, and therefore including all available related data might result in a less accurate forecast. If there have been significant changes in your operation that affect related values, it might make sense to only use more recent data. 

Zeros in Related Data 

Also worth considering is the number of zeros in the related data. Too many zeros will prevent a forecast from being generated or will result in a poor forecast. It's important to understand the cause for the high portion of zeros to counter with the appropriate solution. If the data is too sparse (time series where many of the values are zero), consider the frequency and/or granularity of the data and the desired forecast. Please refer to the sections on data frequency, data granularity, and incomplete data for more information. 

Data Frequency 

The forecast frequency should also be decided before starting with the forecasting process in Forecaster. The business goal and whether the historical and related data sets support the frequency needed should both be considered. If the forecast frequency needed is different from what the data set can support, consider the minimum (lowest resolution) frequency the forecast is needed at to still be able to achieve your business goal. Other considerations for data frequency include: 

Sparsity of Data Set 

If a data set is too sparse at the current frequency, either overall or for certain forecast items, this can lead to lower quality forecast and difficulty in using certain evaluation metrics. If aggregating the data to a lower frequency is a possible alternative given your business needs, forecasting with more dense data will generally reduce variance and improve the forecast quality. If only certain forecast items are affected by a sparsity issue, consider removing the items or create a separate forecast model for them. 

Frequency Mismatch of Historical and Related Data 

In the presence of related data, Forecaster will forecast at the frequency of the related data. If the historical and related data are on different time scales, Forecaster is only able to aggregate the historical data to match the related if the latter is on a less frequent time scale. Forecaster will not change historical data to match the related data otherwise. Consider what is the business goal for the forecast. Will forecasting at the related data frequency make sense or are you able to break out the historical data. If neither are options, does forgoing related data produce acceptable forecasts. Note that related data can be aggregated in Anaplan before being brought into Forecaster. 

Data Granularity 

The granularity of the data sets presents similar challenges as data frequency. The main difference being focused at the level at which items are forecasted at instead of the time scale or frequency. Instead of considering forecasting at a week versus month level, granularity looks at the SKU or product category level. The main issue is the same, whether the data supports forecasting at the level the business use case calls for, and if not, can the granularity of the data sets be changed while still achieving the forecasting goal. It is important to keep in mind that unless you can use the forecast, lower granular data or transform it back into a useable higher granularity, it would be best to keep the data as is. 

Incomplete Data 

Incomplete data, like sparsity, deals with missing data. The difference is the issue centers around missing data due to unavailable data or changes in business. Common causes for incomplete data are new product introduction/cold start scenarios, obsolete items, and lack of related data in the forecast horizon. 

New Product Introduction/Cold Start 

Forecasting for new products or other items with no historical data will limit the algorithms that can be used to DeepAR. It would also call for the use of attributes and other requirements.

Obsolete Items 

Whether to include obsolete products or items no longer necessary in historical data should be based on several factors: 

• Potential value of the historical data for the item 
• Reason the item is no longer needed 
• If there are similar or replacement items to be forecasted 

Also, obsolete items will count toward quota consumption. These factors should be considered as including these items could produce a weaker forecast by adding non-relevant information. Generally, unless there are replacement items for the obsolete products, it's better to remove them than keep them in the data. 

No Related Data for Forecast Horizon 

Related data can be a valuable source of information that adds to the accuracy and quality of the forecast. Not all algorithms can take advantage of it, and those that do support it require forward-looking (future) related data values. 

Outliers 

Historical and related data sets with many outliers will be difficult to predict. A quick check for outliers would be to plot the data sets and search for outliers visually. How to address outliers once identified will depend on the cause of the outlier. A data entry error versus something that occurs due to a known factor, such as a holiday, will require different ways to handle the value or values.

Forecast Evaluation 

Evaluating the forecast accuracy will help determine the best algorithm to use and if data adjustments are needed to support the business use case. It can also be used to compare the performance of Forecaster against historical baselines — if those are available. When evaluating a forecast, please note the following: 

• Backtesting: Test data taken from historical data to evaluate how well the forecast performed. 
• Accuracy metric: Deciding on an appropriate metric based on business use case and data considerations. Learn more about forecast evaluation on Anaplan Community. 
• Weighting of SKUs, product lines or categories: Consider the relative importance of the forecast items in the data set. For instance, if 20% of the items drive 80% of the revenue, you may want to focus on those items when performing the forecast evaluation.

Forecaster produces probabilistic forecasts. This means that Forecaster algorithms output a distribution of possible values, rather than a single point forecast typically outputted by other solutions. This distribution can be divided into quantiles. 

Applied to forecasting, quantiles help to address uncertainty in forecasted values. Quantiles define a prediction interval within which the actual value is likely to fall with a given probability. For example, a P10 quantile (P denotes Probability) indicates that the true observed value is expected to be lower than the forecasted value 10% of the time (i.e. probability of 10%), while a P90 quantile indicates that the true value is expected to be lower than the forecasted value 90% of the time. The difference between the P10 and P90 in this example defines an interval of 80%, which means that the probability of the true value falling between the forecasted values associated with the P10 and P90 quantiles is 80%. Increasing the difference between the quantiles would increase the interval and the probability. 

Note that in the case of the median, or P50, 50% of the distribution falls on either side of the cut point. For quartiles, 25% of the distribution is in each interval at P25, P50, P75. 

Selection of quantiles should be informed by business considerations about the relative costs associated with over and under forecasting. 

Selecting quantiles  

The lower quantile (P10 in the example) can be used in instances where the cost of over-forecasting outweighs the cost of under-forecasting, such as when there are high costs associated with overproduction or overstocking. Use cases could include a manufacturing setting where there is a high cost of capital, as well as a contact center with labor-cost saving objective. 

The upper quantile (P90 in the example) can be used in cases where the cost of under-forecasting outweighs the cost of over-forecasting. For example, in a retail setting with sufficient inventory space, the lost sales due to being understocked outweigh the cost of being overstocked, so forecasting at a higher quantile may be useful. 

A P50 quantile, indicating that the true value is expected to be lower than the forecasted value 50% of the time, provides a balance between concerns of over-forecasting and under-forecasting. 

In cases where the historical data is highly volatile or doesn’t seem to follow any pattern, it could be challenging to fit a forecast model that produces accurate predictions. Users may find the P50 quantile forecast to consistently under- or over-predict throughout the forecast horizon. In such instances, using the lower or upper quantile forecast as the point forecast of choice could produce more accurate predictions. However, if you choose to use a quantile forecast other than P50, it is recommended to reassess this selection in subsequent forecast refresh cycles. 

Using Quantiles with Forecaster 

Forecaster supports 3 forecast quantiles generated with a single forecast action. The middle quantile stays fixed at P50. The lower and upper quantiles can be configured by the user. By default, Forecaster sets the lower and upper quantiles to P10 and P90. 

There are two steps to include quantiles in Forecaster output: 

1. The forecast results module and associated import action must include line items “P1”, “P2”, and “P3”. 

1. The quantiles must be specified in the Forecast Action step. 

Example forecast accuracy dashboard 

When should you assess forecast accuracy? 

Throughout the process of building an operational Forecaster forecast there are several occasions when you will likely need to assess forecast accuracy: 

• Comparing the alternative models and selecting the best one 
• Estimating the expected performance of the future forecast 
• It is important to assess the risk of over- or underperformance, and to be able to plan for variance from the point forecast. 
• Establishing accuracy metrics can also help identify if the forecast deviates from expected performance. 
• Monitoring forecast performance over time and determining when to re-train with new data as old model becomes deprecated. 
• Identifying potential areas with poor performance (e.g., times of year, regions, product groups, etc.) before deployment and take corrective measures 
• Performance may improve with addition of related time series or metadata. 
• You may detect extreme and unpredictable values that bias the model, and you can remove them by using the “exclude_value” option (for more on the exclude value function, see Anapedia). 

Which factors impact forecast accuracy? 

For business users, one of the main challenges to evaluating the performance and accuracy of time series forecasts is that it is extremely dependent on unique business context and use case. 

There are many factors to consider as you establish a forecast evaluation process including: 

1. Seasonality and other cyclical influences 
2. Length of training period 
3. Forecast horizon duration 
4. Business cost of over- vs under- forecasting 
5. Sensitivity to user inputs and assumptions 
6. Periodicity of the dataset (monthly, weekly, daily, hourly, etc.) 
7. Industry and particular line of business 
8. Single-item forecast variable (SKU, Group, etc.) vs. multiple items 
9. Nature of item hierarchy, at which level of hierarchy are business decisions made 
10. Other factors specific to your industry or use case, such as business evaluation criteria that are impacted by forecast results. Consider that different stakeholders or business groups may be relying on the same forecast results for different purposes and those different groups may be interested in different metrics or over different time frames (short- vs long-term performance). 

It is recommended that you note the above factors for your use case and align with business stakeholders on which metrics are most important for decision making. 

Evaluating multiple approaches 

Different approaches to forecast accuracy will be better suited to your specific use case. Many use cases require a combination of methods and metrics to provide accurate performance evaluation. 

It is important to consider multiple approaches to calculating forecast accuracy when implementing a new forecasting project with Forecaster. When appropriate metrics are identified, performance should be monitored on an ongoing basis. As shown below, we recommend benchmarking the Forecaster forecast against any existing forecasts. 

Example comparison of actuals vs Forecaster and other forecasts  

In this article, we review the algorithms that drive time series forecasting with Forecaster. These algorithms range from traditional statistical algorithms such as Exponential Smoothing (ETS), to those based on complex neural network algorithms such as DeepAR. 

Before talking about the algorithms in more detail, it's important to know what types of data these algorithms can support. In addition to historical values, datasets can also include related time series data and item attributes. Related time series are data that change over time. They may correlate with the target values and may help improve the accuracy of the forecast. Examples include features such as price, promotions, and weather. Item attributes are categorical features that provide valuable context for the items in historical data. Unlike related time series datasets, item attributes datasets provide static​ data. That is, ‌data values remain constant over time, such as a product category or type.

Algorithm Methods

Two of the time series forecasting algorithms are SARIMAX (ARIMA extended to incorporate seasonality and related data) and ETS (Exponential Smoothing). These are common statistical algorithms for time-series forecasting. They're especially useful for simple data sets with under 100 different periods. These algorithms work by attempting to "explain" a given time series based on its own past values (SARIMAX also uses related data), so that the resulting equation can be used to forecast future values. Both algorithms perform well when trend and seasonality are likely to explain most of the variance in the time series data. A disadvantage is that they aren't applicable in cold-start scenarios (forecasting with no historical data). ETS also gives more weight to the most recent history. So, if ‌data patterns have recently changed, ETS may be more sensitive to those changes. 

MVLR (Multi-variate linear regression) is a foundational statistical forecasting method. It trains a model using a historical dataset and establishes a linear relationship between the input features. The basic assumption in ‌multivariate analysis is that ‌time-dependent features not only depend on their historical values but also show a relationship between them. MVLR models can create fast and accurate forecast models based on features. They can also give insights on how and which drivers are most important. Under the hood, MVLR uses historical data, related data (optional), calendar data (optional), and synthetic data (automatically created by Forecaster, based on either historical or related data). Examples of synthetic data include trends such as exponential and linear, seasonality effects, as well as lagged values.  

The Prophet is an algorithm based on an additive modeling procedure where non-linear trends are fit with yearly, weekly, and daily seasonality. It works best with time series with strong seasonal effects, and is compatible with holidays or other previously known important, but irregular events. An advantage is that it's suitable for "what-if" analysis. While Prophet supports a few missing observations or outliers, it isn't suitable for sparse datasets. Prophet can also provide insights on how and which drivers most impact forecast results. Under the hood, Prophet employs historical data, related data (optional), calendar (optional) and synthetic data (automatically created by Forecaster, based on either historical or related data). Examples of synthetic data include linear trends, seasonality effects, as well as lagged values. 

DeepAR is a deep learning neural network algorithm. It works best with larger historical datasets containing hundreds of time series. DeepAR can incorporate related data and attributes across time series to identify underlying structures and similarities. Furthermore, DeepAR is suitable for advanced forecasting scenarios such as sparse datasets, "what-if" analyses, and cold-start scenarios. 

LightGBM is an advanced machine learning model that uses gradient-boosting decision tree methods to generate forecast results. LightGBM performs well in most situations, including with large data sets and even when there's missing data. LightGBM can support related data line items and holiday calendar options. This algorithm also supports "what-if" analysis, since you can use related data line items to simulate future scenarios. 

TimesFM is an advanced foundation model. It's pre-trained on large-scale real-world data, and can generate forecasts across varied data sets. It's an adaptable and efficient algorithm that can produce accurate forecasts with limited historical data. 

You can choose a specific algorithm from the above, or use Ensemble. Ensemble automatically compares the performance of multiple algorithms (ETS, SARIMAX, MVLR, and Prophet) and applies the optimal algorithm for a given item based on your selected metric (MASE, MAPE, or RMSE). This algorithm maximizes forecast accuracy without the need to run multiple algorithms over the entire data set.

Incorporating related time series and holiday information 

To make the most out of algorithms that support related data, it is recommended that you incorporate as many related time series as possible (up to the maximum limit). Consider if the related data adds information that is relevant for the forecast. For example, related data that correlate to historical data could lead to more accurate forecasts. However, keep in mind that related data time series that lack variability have little value and should not be included. When possible, select the built-in holiday calendar during Forecast Model creation to help improve accuracy.

Conclusion 

Forecaster algorithms enable you to generate accurate forecasts. Algorithm performance depends on the specific use case, data set, and historical patterns. No single algorithm is better than another. A best practice is to compare the results of different algorithms and select the algorithm or combination that produces the most accurate results for your data.

In instances where those items share attributes (characteristics) with pre-existing items, the DeepAR algorithm can use the attributes to utilize similar items’ historical data to produce forecasts for the new items. Attributes add the most value when the categories are varied, so that there are neither too many unique values nor a single category that is the same for every item. 

To use attributes, users must select DeepAR as the forecast model algorithm, and train the model based on a data collection that contains an attributes module. Please note the following: 

1. Every item in the historical data module should be present in the attributes module. 
2. The attributes module can contain up to 10 attribute fields. Forecaster assumes those fields to be text (string). 
3. There is no naming convention for these fields. 

For more information about the attributes module, please refer to Anapedia.

Example of an Attribute Module 

• a list of items to forecast against (e.g., SKU by Store),
• a list of time periods containing the history 
• the data or measure to be forecasted (e.g. sales volumes) 

This article focuses on the creation of the Item ID list. When combined in a module with a time dimension and a line item for the data, it provides all the data points needed for the algorithms to work on. It can be made up of elements from a single business dimension, such as SKU. More likely, however, is that it is a combination of two or more independent business dimensions, such as SKU by Store, or Product by Channel by Customer. Forecaster needs the Item ID list to be a single list with a unique code property. If a unique code does not already exist, you can make this list by combining valid combinations. 

What is the best practice for concatenation in Anaplan? 

There are 2 possibilities at this stage: 

1. Generate the list in a source system or database, and import it directly into Anaplan. This is the simplest solution and applies the least burden on the Anaplan calculation engine. 
2. Leverage historical data to identify valid combinations. This article will walk you through the steps. 

Historical transactions may held be in a sparse module which is dimensioned by key attributes such as Product and Customer. Alternatively, they may be held in a dense module dimensioned by a generic list (transaction ids) with key attributes such as Product and Customer as line items.  We will look at both examples. 

Concatenation is a function that manipulates text. Text is “expensive” in Anaplan, in terms of memory. Formulas calculating text can be a drain on performance over large lists and data sets. We are recommending the following best practice to minimize the time taken to calculate the text you need to create your concatenated lists. 

Step-by-step overview 

Let’s consider the first example of a data module with historical data dimensioned by Customer and Product by Month. 

For simplicity we’ll just look at codes. The customer code is 10 digits and numeric, the product code is numeric or alphanumeric and of variable length. We have 3,532 items in the Product list, and 2,254 in the Customers list. Combined with 3 years of months and we have a line item with over 318m cells. 

We will demonstrate 4 levels of efficiency in the concatenation formula. 

Level 1 – code inefficient 

Now, we could calculate a concatenated code by adding a text formatted line item in this module as follows: 

There are two concatenations here because we are adding an underscore as a delimiter, but you can add any delimiter you want. This is very inefficient in such a large module. 

Ok, subtotals are off, so we are down to 286m cells: 

But the engine is calculating each concatenation 286m times, with the same results being repeated many times. 

There is an important adage when it comes to modeling in Anaplan: 

"Calculate once, reference many times" - (see The Zen of Anaplan for more details) 

In other words, it is much quicker for the engine to pick up a result that is already calculated and reference it, than to calculate it again. 

Level 2 – code better 

To avoid the repetitions, let’s split the Customer and Product calculations into System Modules (one for each). System modules are an important construct in out recommended DISCO methodology for building models. For more details about DISCO, check out this link. 

The cell counts for each line item are 2,254 and 3,532 respectively, a total of 5,786 calculations, far lower than the previous option. 

Back in our transaction module, we can use a more optimal formula like this: 

Most of the formula is referencing, not calculating, the customer and product codes. However, the underscore is still being “calculated” 286m times. 

Level 3 – Code to use 

We can improve this by creating a tiny module (3 cells here, one for each of 3 different kinds of delimiter) containing the underscore 

….and then referencing this in the Customer Sys module to concatenate it to Customers (smaller list than products): 

The underscore is calculated once only, then referenced first in the Customer Sys module and from there back in the transactions module as follows: 

Level 4 – Optimized code to use 

But there is one more glaring inefficiency here. The codes do not vary with time. Let’s create a module without time. 

The formula for the Item_ID is the same. Pivoted like this, the view can be used as the source for creating your Target Time Series list of Item IDs. 

However, there is one more thing we should do here. You can see that the transaction module has sparsity, meaning that there are combinations of Customers and Products with no data. You can eliminate these by introducing a condition to the formula to blank out codes with no data associated with them. 

Blanks now appear where the data totals zero across all periods. Note that SELECT:Time.All Periods is one of the few accepted uses of the Select function per the Planual. 

Add a filter to eliminate these from the view. Save the view to use as a source for importing into the Target Time Series list. 

Below is the list as created from this saved view. Notice the absence of 1755004000_6990 in the final Item_ID list here, since there is no data for this combination… 

Now let’s look at the second type of transaction module, based on a list of transactions. 

By definition this is a much more dense module than the previous one, since there would not be a transaction for an invalid combination of product and customer. It is also smaller for the same reason. To derive our Item ID list, we could simply add a single line item to do the concatenation like this: 

But of course, as you can see, the result is repeated for every time period. This would still work as a source for the Item ID list but the import would perform more slowly while it wrote error logs for all the duplications. Better is to add another line item to use as a filter to eliminate the duplications at source. This is a Boolean item which uses the ISFIRSTOCCURENCE function to highlight just one occurrence of each combination. 

Apply the filter to just include the distinct combinations: 

Then save a view to use as the source for building the Item_ID list: 

So, we have looked at a couple of examples for creating our Item ID list by concatenating independent dimensions. Ultimately, Forecaster will return the results from the forecast it generates into a results module dimensioned by the same Item ID list. 

We almost certainly need to “de-concatenate” these results into a dimensionalized module for downstream use and reporting in Anaplan. For that we need a SYS module based on the Item ID list, which maps the Items to Product and Customer separately. 

You can see from the blueprint of the SYS module how this is achieved with simple FINDITEM formulae. 

We also need a mapping of the Timestamp list used by Forecaster to the Anaplan native timescale. This is done in a SYS module related to Time: 

The blueprint for this SYS module is given here: 

Now we can construct a module with Anaplan time, Product and Customer as follows: 

Job done! 

 By leveraging these best practices, you will make the most of Forecaster Forecastering capabilities and your performance will be optimized! 

How do we handle items with limited to no history? 

Adding “Cold Start” Items 

We can use Forecaster to produce forecasts for which no history exists. A deeper dive article on cold starts is available here. For the purpose of this article, we just focus on how to add such an item to the concatenated list. 

We will assume that both the customer and product already exist in the data hub. If either or both are completely new, they will have been created in the source system and loaded into Anaplan. Alternatively, it could just be that we have never sold a particular product to an existing customer before. 

Build a simple module without time or versions, dimensioned by a generic list, with line items as per this blueprint: 

The module view looks like this: 

The Item Code is constructed, and a filter column records TRUE when the item does not yet exist in the ITEM ID list. 

If we look at the data module, we can see that there is no data for those two items, which makes sense as the concatenation would not have been created by the process detailed above. 

Apply the filter and hide all columns except for the Item Code. Save as a view to use to import to the Item ID list. 

Now create an import action to the Item ID list, using this view as the source. Once you run this, notice that the filter has now been cleared, because the items now exist: 

You should then build an action to clear this module out to be used again in the next forecast cycle. 

Question? Leave a comment!

• First things first
• What is PlanIQ?
• Who is it for?
• Use case examples
• Now, let's go under the hood
• To get started
• Once you get more comfortable
• Deep dive on algorithms
• Support

You are interested in starting time series forecasting with PlanIQ on Anaplan, but you don’t know where to start? You have come to the right place!

This article is the pathway to all things PlanIQ.

First things first

What is PlanIQ?

PlanIQ is an easy to use time-series forecasting tool directly integrated with the Anaplan platform. It allows users to accurately predict future values based on historical data, attributes, and other related data. PlanIQ enables intelligent planning backed by advanced machine learning and statistical forecasting, supporting planners and business stakeholders in shaping future outcomes by making data-driven decisions.

Who is it for?

Use case examples

Demand planning
Improve the accuracy of your demand forecasts and quickly get to a consensus demand plan with collaborative planning.

Marketing and promotional spend optimization
Improve the ROI and develop more effective marketing programs programs by understanding the drivers that impact demand uplift.

Financial planning
Quickly generate OpEx forecasts and analyze new scenarios, variances, and opportunities so board members can speed decisions.

Assortment planning
Reduce SKU proliferation by designing the optimal assortment for each segment, channel, and category based on internal and external signals.

Workforce planning
Reduce operating costs and plan for the right levels of staffing and skills by accurately predicting volume and variety of tasks.

Inventory optimization
Reduce inventory carrying costs by optimizing inventory levels across the supply chain without affecting service levels.

Now, let's go under the hood

To get started

We recommend taking the E-learning course and reading these articles:

1. Anapedia
2. Design and build your item list for forecasting
3. Probabilistic forecasting using forecast quantiles
4. How to use item attributes to refine your forecast
5. Join our User Group dedicated to PlanIQ

Once you get more comfortable

Let’s dig a little deeper with the following:

1. PlanIQ – Considerations Before Starting to Forecast
2. New product introduction - all you ever wondered about starting your forecast from scratch
3. Algorithm selection by item - Mix and Match your Forecast!
4. How to manage NULL values?
5. Dealing with outliers
6. PlanIQ: Introduction to forecast evaluation
   

Deep dive on algorithms

If you are interesting learning more about the different algorithms available in PlanIQ, please look at this article:

• Deep dive on the algorithms under the hood

Support

Finally, to support you best in your experience, please refer to the Support self-service article

• PlanIQ error messages and warnings

Got feedback on this content? Let us know in the comments below.

Contributing authors: Nitzan Paz, Christophe Keomanivong, Frankie Wolf, Timothy Brennan, Andrew Martin, and Evgenya Kontorovich.

Background

Customers are frequently wondering what can be used as related data. When thinking about related data, we can think about different types of “internal” time dependent inputs:

• Historical and future promotions
  
• Stockouts
  
• Price fluctuations
  
• Holidays and special events

Actual data may be different dependent on a specific use-case, for example when forecasting cloud or software license costs, related data can be in form of number of engineers or number of employees.

External “external” factors are also use-case dependent and include, among other things:

• Weather
  
• Traffic and mobility data
  
• Currency exchange rates
  
• Interest rates
  
• Commodity prices
  
• Consumer price indexes

Additionally, it may be helpful to provide additional related information in order to help algorithms learn from the past better.

Lastly, it may be useful to provide related data as an additional input even if historic seasonality may be sufficient to predict similar future behavior. We will further explore this scenario in one of the examples in this article.

Before we dive into the examples I would like to remind you that MVLR and Anaplan Prophet algorithms provided by PlanIQ are actually looking at several types of related data:

• Related data provided by the user – when provided.
  
• Holidays calendar provided by the user in form of related data or in form of built-in PlanIQ holiday calendars.
  
• Historical behavior of the particular item (time series) – historical trend and seasonality components.
  

On top of this information, PlanIQ performs automated analysis of the data in order to identify leading and lagging indicators and all that information is the used as part of feature selection where the algorithms pick and choose only relevant information in order to use it as part of the forecast. You can review this article* for detailed information about this topic.

From the perspective of this article, we need to remember that algorithms are looking at both your historical data, as well as related data and historical behaviors.

Another thing to remember is that it is important to provide related data not only for the known past but also into the future. Most of PlanIQ algEorithms (except for CNN-QR) would ignore related data if it’s not provided in the future for the entire length of forecast horizon.

It is not difficult to provide future information for events that are “internal” to your business, for example promotions or changes in price. It is more difficult to do it when external factors are being used. In this case, our recommendation is to first forecast the external factor and then use the forecast as input into the main forecast. Even if your forecast is only directionally accurate it will allow you to generate a forecast based on an assumption that you control and it also allows you to perform what if analysis where you can adjust the external data forecast and observe the impact of the changes on the main forecast.

Dataset

For the discussion in this article I used the following champagne sales data (as a single time series) provided by this Kaggle dataset*. It covers monthly sales of champagne between 1964 and 1972. I have modified the timestamps in the data in order to “move it” into present and used most recent 5 years of this data (from 2018 to 2023).

The experiments

When looking at historical data it’s evident that the data has a clear yearly seasonality pattern with sales peaks around December and drops of sales around August.

It can be observed that overall trend is positive with a peak in end of 2021 and then there’s a certain drop or downward trend. In general, 2021 had higher actuals than years before and after. This is also evident in the Seasonality and Trend analysis data provided by PlanIQ.

We will explore and observe impact of several experiments:

• Forecasting based on historical data only.
  
• Forecasting based on both historical and related data.

During our experiments I started the forecasts from January 2021 and used actuals until December 2020. The forecast horizon for this experiment was 12 months. During several iterations I added 12 months of actuals and projected additional 12 months of forecast.

Experiments with MVLR

We will observe last period of forecast, between January 2023 and December 2023. When based on historical data only MAPE of P2 forecast (0.5 quantile) was 10.72% which is pretty accurate. Looking at explainability, we can see that the algorithm rightfully decided that yearly seasonality identified by PlanIQ is the most impacting factor in our forecast. Second impactful factor is “exponential upwards trend” that is identified by PlanIQ.

Another experiment that I performed included related data that I generated. I used two types of related inputs:

• Holidays – indicating the holiday peaks in December and drops in August. In this case I used numeric values where regular months had a value of 1, December had a value of 100 and August a value of ‘-20’. The numbers are arbitrary.
  
• Covid19 – indicating months of COVID pandemic. Months of 2020 and 2021 had value of 100 and other months had value of 0.

When using this related data to perform same forecasts I was able to reduce MAPE rate of P2 (0.5 quantile) to 5.21% which is a significant improvement.

Even visually, we can see that P2 curve is much closer to actuals during 2023 period. In terms of explainability, seasonality is still the most impacting factor but we can see much bigger impact of other inputs including the impact of COVID. It is possible that introduction of COVID related time series is making the algorithm more sensitive to trend changepoint that happened after 2021.

Experiments using Anaplan Prophet

When forecasting based on the same data using Anaplan Prophet we see a similar behavior. When looking at MAPE P2 forecast (0.5 quantile) based on historical data only between January 2023 and December 2023 the value of MAPE is quite high, it’s ±44%.

The most important factor here is yearly seasonality but we can see that the forecast in 2023 is pretty much on the same level as in 2021 and this is causing the high MAPE value. In this case, the customer could decide to use values between P1 (0.1 quantile) and P2 (0.5 quantile) as forecast baseline.

When introducing related data as specified above, MAPE of P2 is reduced to 27%. In this case, impact of August “lows” and December “highs” is higher, and it seems that the algorithm is giving related holidays data more weight than to seasonality alone. Additionally, it seems that COVID related data also helps the algorithm to be more sensitive to trend changepoint and by that reduces the over-forecasting in the majority of 2023 months. The customer could decide to use values between P1 (0.1 quantile) and P2 (0.5 quantile) as forecast baseline based on the specific use-case.

Comparing MVLR and Anaplan Prophet

In this experiment, we saw that for 2023 MVLR has outperformed Anaplan Prophet both with and without related data. However, if we look at 2022 data, Anaplan Prophet was more accurate than MVLR. In both cases the algorithms over-forecasted. This is probably because 2021 actuals were much higher than previous years and both algorithms clearly tried to continue the trend. However, while MVLR probably gave more “importance” to trend vs Anaplan Prophet that was more “reserved” in its predictions and eventually provided more accurate forecast.

Summary

In this article we have explored various impacts that related data can have on your forecast. Few points can be highlighted:

• If your historical data has strong seasonal patterns, it may still be useful to amplify the impact of those patterns with an additional related data (just like I did with holidays in this example) – in case of several patterns in data, where both seasonality and trend exist.
  
• Even though we’re not in COVID era anymore, some of your historical data may be impacted by COVID and it may make sense to flag months / weeks impacted by COVID and lockdowns in order to indicate to the engine that certain parts of history were not the norm.
  
• Different algorithms may perform differently at different time periods. Some of our customers choose a winning algorithm based on past performance across several time periods. This helps them insure that over time selection of winning algorithm is more stable and predictable.

The experimentation was based on MVLR and Anaplan Prophet specifically because we wanted to explore impact of related data. In larger data collections you are invited to try out DeepAR+ and CNN-QR as well as Anaplan AutoML and Amazon Ensemble.

*Note: you must join the PlanIQ group to view the starred articles. Feel free to join!

PlanIQ produces probabilistic forecasts. This means that PlanIQ algorithms output a distribution of possible values, rather than a single point forecast typically outputted by other solutions. This distribution can be divided into quantiles.

Applied to forecasting, quantiles help to address uncertainty in forecasted values. Quantiles define a prediction interval within which the actual value is likely to fall with a given probability. For example, a P10 quantile (P denotes Probability) indicates that the true observed value is expected to be lower than the forecasted value 10% of the time (i.e. probability of 10%), while a P90 quantile indicates that the true value is expected to be lower than the forecasted value 90% of the time. The difference between the P10 and P90 in this example defines an interval of 80%, which means that the probability of the true value falling between the forecasted values associated with the P10 and P90 quantiles is 80%. Increasing the difference between the quantiles would increase the interval and the probability.

Note that in the case of the median, or P50, 50% of the distribution falls on either side of the cut point. For quartiles, 25% of the distribution is in each interval at P25, P50, P75.

Selection of quantiles should be informed by business considerations about the relative costs associated with over and under forecasting.

Selecting quantiles 

The lower quantile (P10 in the example) can be used in instances where the cost of over-forecasting outweighs the cost of under-forecasting, such as when there are high costs associated with overproduction or overstocking. Use cases could include a manufacturing setting where there is a high cost of capital, as well as a contact center with labor-cost saving objective.

The upper quantile (P90 in the example) can be used in cases where the cost of under-forecasting outweighs the cost of over-forecasting. For example, in a retail setting with sufficient inventory space, the lost sales due to being understocked outweigh the cost of being overstocked, so forecasting at a higher quantile may be useful.

A P50 quantile, indicating that the true value is expected to be lower than the forecasted value 50% of the time, provides a balance between concerns of over-forecasting and under-forecasting.

In cases where the historical data is highly volatile or doesn’t seem to follow any pattern, it could be challenging to fit a forecast model that produces accurate predictions. Users may find the P50 quantile forecast to consistently under- or over-predict throughout the forecast horizon. In such instances, using the lower or upper quantile forecast as the point forecast of choice could produce more accurate predictions. However, if you choose to use a quantile forecast other than P50, it is recommended to reassess this selection in subsequent forecast refresh cycles.

Using Quantiles with PlanIQ

PlanIQ supports 3 forecast quantiles generated with a single forecast action. The middle quantile stays fixed at P50. The lower and upper quantiles can be configured by the user. By default, PlanIQ sets the lower and upper quantiles to P10 and P90.

There are two steps to include quantiles in PlanIQ output:

1. The forecast results module and associated import action must include line items “P1”, “P2”, and “P3”.

2. The quantiles must be specified in the Forecast Action step.

Got feedback on this content? Let us know in the comments below.

Contributing authors: Nitzan Paz, Christophe Keomanivong, Frankie Wolf, Timothy Brennan, Andrew Martin, and Evgenya Kontorovich.

Companies often develop plans and business strategies to drive decisions and actions around finance, operations, supply chain and other areas. They do this based on how they perceive the future, under the conditions of uncertainty and unknowns. Assessing the future and developing these strategies is done so that actions can be taken in the present to better prepare for the future. To this end, companies can collect and analyze past data and other types of data to generate predictions of the future. Failing to plan for the future or creating predictions with a high rate of inaccuracies can be costly in terms of under-forecasting or over-forecasting, misallocation of resources such as time and capital, missed opportunities etc. The benefits of accurate forecasts include, for example, reducing waste, cutting costs, minimizing storage expenses, maximizing resource utilization, and ensuring that no sale opportunity is lost due to insufficient inventory. 

In this article, we will review the algorithms that power modern time series forecasting with PlanIQ. These algorithms range from traditional statistical algorithms such as Autoregressive Integrated Moving Average (ARIMA), to those based on complex neural network algorithms like DeepAR+.

Before describing the algorithms in detail, it is important to understand the types of datasets that these algorithms can accept. In addition to historical values, datasets can also include related time series data and item attributes. Related time series is time-dependent data that has some correlation with the target values and may help improve the accuracy of the forecast. Examples include features such as price, promotions and weather. Item attributes are categorical features that provide valuable context for the items in a historical data. Unlike related time series datasets, item attributes datasets provide static. That is, the data values remain constant over time, like item category or type.

Baseline time series algorithms

Baseline time series forecasting algorithms include ARIMA and ETS (Exponential Smoothing). They are commonly-used statistical algorithms for time-series forecasting and are especially useful for simple datasets with under 100 distinct periods. These algorithms work by attempting to ‘explain’ a given time series based on its own past values, so that the resulting equation can be used to forecast future values. The advantages of these baseline algorithms are that they are relatively quick and can establish a performance baseline. They are relevant when simple concepts of trend and seasonality are likely to explain most of the variance in the time series data. Since these models work on an item-based level, they do not support the use of related data, calendar or attributes. Other disadvantages are that they are not applicable in cold-start scenarios (forecasting with no historical data), and they do not perform hyperparameter optimization.

Another point worth mentioning is that ETS is giving more weight to most recent history, so if the data patters have recently changed, ETS may be more sensitive to those changes.

Advanced statistical time series algorithms

MVLR (Multi-variate linear regression) is a type of advanced statistical forecasting algorithm. It trains a model using a historical dataset and establishes a linear relationship between the input features.

The underlying assumption in the multivariate analysis is that the time-dependent features not only depend on their historical values but also exhibit dependency between them. Using these dependencies, MVLR models can not only generate fast and accurate feature-based forecast models, but also provide insights on how and which drivers most impact forecast results.

Under the hood, the input features MVLR employs are historical data, related data (optional), calendar (optional), and synthetic data (automatically created by PlanIQ, based on either historical or related data). Examples of synthetic data includes trends such as exponential and linear, seasonality effects, as well as lagged values. 

Flexible local algorithms

Another category of algorithms is flexible local algorithms, which includes Anaplan Prophet. It is based on an additive modeling procedure where non-linear trends are fit with yearly, weekly, and daily seasonality. It works best with time series with strong seasonal effects and several seasons of historical data, and is compatible with holidays or other previously known important, but irregular, events. Its advantages are that it is suitable for what-if analysis and is computational inexpensive. While Prophet supports some missing observations or outliers, it is not suitable for sparse datasets.

Anaplan Prophet models can not only generate fast forecast models, but also provide insights on how and which drivers most impact forecast results.

Under the hood, Anaplan Prophet employs historical data, related data (optional), calendar (optional) and synthetic data (automatically created by PlanIQ, based on either historical or related data). Examples of synthetic data includes linear trends, seasonality effects, as well as lagged values.

Neural network algorithms

The last category of time series forecasting algorithms is those based on artificial neural networks, including DeepAR+ and CNN-QR. They work by using deep learning architectures such as RNNs (recurrent neural networks) or CNNs (convolutional neural networks) to identify patterns in the historical datasets and predict the future. They work best with larger historical datasets, containing hundreds of time series. DeepAR+ accepts forward-looking related time series and item attributes. CNN-QR accepts item attributes and is the only forecast algorithm that accepts related time series data without future values. Advantages of the algorithms include the ability leverage data from similar time series while forecasting, and to use related datasets, as well as item attributes to identify and learn underlying structures. Furthermore, these algorithms are suitable for advanced forecasting scenarios such as sparse datasets, what-if analysis, and cold start scenarios. The disadvantages of these algorithms, however, are that they require larger historical datasets and may take longer to train and predict. 

Optimizing Anaplan algorithms: incorporating related time series and holiday information

In order to make the most out of Anaplan algorithms like Anaplan MVLR or Anaplan Prophet, it is recommended that users incorporate as many related time series as possible (up to the maximum limit). Additionally, when applicable, incorporating holiday information can further enhance the accuracy and reliability of the forecasts generated by these algorithms. By following these recommendations, users can optimize the performance of their chosen algorithms for more precise forecasts.

Summary

The time series forecasting field has recently generated interest, with many technology companies developing and open-sourcing algorithms. Like the recent significant improvement in performance in fields such as computer vision or natural language processing, we see that the performance of the time series forecasting algorithms has significantly improved and can be applied to a wide variety of real-world business applications. 

PlanIQ employs open-sourced and proprietary algorithms to allow customers to generate accurate forecasts. Users can choose a specific algorithm for their use case or use Anaplan AutoML, which compares the performance of all algorithms automatically and picks the best performing algorithm for most of the items.

It should be noted that algorithm performance depends on the specific use case, datasets, context and historical patterns, so no single algorithm is better than the other. Therefore, we compare and find the best one for specific customer’s datasets, use cases, and items.

Do you have feedback on this content? Let us know in the comments below.

Contributing authors: Nitzan Paz, Christophe Keomanivong, Frankie Wolf, Timothy Brennan, Andrew Martin, Oren Tevet, and Evgenya Kontorovich.

Below are the main topics to consider before starting to forecast with PlanIQ. Please regard the business problem you are trying to solve as you work your way through the list.

Forecast Horizon

When deciding on the length of the forecast horizon (the length of time into the future for which a forecast is generated), avoid simply basing it on the allowed horizon for the chosen algorithm. While it is a good starting point, be sure to consider both the business use case and the available data. Start with the forecast horizon needed to achieve your goal and work from there. If projections for one year out are needed, verify that there is enough historical data to support it. If using related data, it's recommended to include forward-looking data that covers the forecast horizon. Reference the forecast horizon guide on Anapedia to learn more about the supported horizons for each algorithm.

Related Data and Attributes

Related data (variables that correlate with or affect the forecasted time series) and attributes (categorical information that groups historical data by their shared characteristics) are valuable information that could improve the quality and accuracy of forecasts. Before deciding whether to use such data as part of your forecast, consider the following.

Data Value

The most important thing is the data itself. Consider if the data adds information that is relevant for the forecast. For instance, related data which displays patterns that correspond to historical data could lead to more accurate forecasts. However, having constant or near-constant values in related or attribute data would have limited value and might limit the algorithms you can use. PlanIQ's deep learning algorithms, CNN-QR and DeepAR+, could also be negatively impacted by too few or too many unique attribute values. The number of related data or attributes also matters, where more isn't always better. More information on using attributes can be found on Anaplan Community.

Mean and Variance of Related Data

The nature of mean and variance of the related data is also worth considering. Is the entire dataset available representative of your business now and in the future? Old patterns do not necessarily represent more recent behavior of the forecast items, and therefore including all available related data might result in a less accurate forecast. If there have been significant changes in your operation that affect related values, it might make sense to only use more recent data.

Zeros in Related Data

Also worth considering is the number of zeros in the related data. Too many zeros will prevent a forecast from being generated or will result in a poor forecast. It's important to understand the cause for the high portion of zeros to counter with the appropriate solution. If the data is too sparse (time series where many of the values are zero), consider the frequency and/or granularity of the data and the desired forecast. Please refer to the sections on data frequency, data granularity, and incomplete data for more information.

Data Frequency

The forecast frequency should also be decided before starting with the forecasting process in PlanIQ. The business goal and whether the historical and related data sets support the frequency needed should both be considered. If the forecast frequency needed is different from what the data set can support, consider the minimum (lowest resolution) frequency the forecast is needed at to still be able to achieve your business goal. Other considerations for data frequency include:

Sparsity of Data Set

If a data set is too sparse at the current frequency, either overall or for certain forecast items, this can lead to lower quality forecast and difficulty in using certain evaluation metrics. If aggregating the data to a lower frequency is a possible alternative given your business needs, forecasting with more dense data will generally reduce variance and improve the forecast quality. If only certain forecast items are affected by a sparsity issue, consider removing the items or create a separate forecast model for them.

Frequency Mismatch of Historical and Related Data

In the presence of related data, PlanIQ will forecast at the frequency of the related data. If the historical and related data are on different time scales, PlanIQ is only able to aggregate the historical data to match the related if the latter is on a less frequent time scale. PlanIQ will not change historical data to match the related data otherwise. Consider what is the business goal for the forecast. Will forecasting at the related data frequency make sense or are you able to break out the historical data. If neither are options, does forgoing related data produce acceptable forecasts. Note that related data can be aggregated in Anaplan before being brought into PlanIQ.

For more on managing null values, refer to the best practice guide on Anaplan Community.

Data Granularity

The granularity of the data sets presents similar challenges as data frequency. The main difference being focused at the level at which items are forecasted at instead of the time scale or frequency. Instead of considering forecasting at a week versus month level, granularity looks at the SKU or product category level. The main issue is the same, whether the data supports forecasting at the level the business use case calls for, and if not, can the granularity of the data sets be changed while still achieving the forecasting goal. It is important to keep in mind that unless you can use the forecast, lower granular data or transform it back into a useable higher granularity, it would be best to keep the data as is.

Incomplete Data

Incomplete data, like sparsity, deals with missing data. The difference is the issue centers around missing data due to unavailable data or changes in business. Common causes for incomplete data are new product introduction/cold start scenarios, obsolete items, and lack of related data in the forecast horizon.

New Product Introduction/Cold Start

Forecasting for new products or other items with no historical data will limit the algorithms that can be used to either DeepAR+ or CNN-QR. It would also call for the use of attributes and other requirements. For more on details, see the new product introduction page on Anaplan Community.

Obsolete Items

Whether to include obsolete products or items no longer necessary in historical data should be based on several factors:

• Potential value of the historical data for the item
• Reason the item is no longer needed
• If there are similar or replacement items to be forecasted

Also, obsolete items will count toward quota consumption. These factors should be considered as including these items could produce a weaker forecast by adding non-relevant information. Generally, unless there are replacement items for the obsolete products, it's better to remove them than keep them in the data.

No Related Data for Forecast Horizon

Related data can be a valuable source of information that adds to the accuracy and quality of the forecast. Not all algorithms can take advantage of it, and except for CNN-QR, those that do support it require forward-looking (future) related data values.

Outliers

Historical and related data sets with many outliers will be difficult to predict. A quick check for outliers would be to plot the data sets and search for outliers visually. How to address outliers once identified will depend on the cause of the outlier. A data entry error versus something that occurs due to a known factor, such as a holiday, will require different ways to handle the value or values. For more on how to detect outliers and suggestions on adjustments, see the dealing with outliers article on Anaplan Community.

Forecast Evaluation

Evaluating the forecast accuracy will help determine the best algorithm to use and if data adjustments are needed to support the business use case. It can also be used to compare the performance of PlanIQ against historical baselines — if those are available. When evaluating a forecast, please note the following:

• Backtesting: Test data taken from historical data to evaluate how well the forecast performed.
• Accuracy metric: Deciding on an appropriate metric based on business use case and data considerations. Learn more about forecast evaluation on Anaplan Community.
• Weighting of SKUs, product lines or categories: Consider the relative importance of the forecast items in the data set. For instance, if 20% of the items drive 80% of the revenue, you may want to focus on those items when performing the forecast evaluation.

Instead of using all the historical data available up to the moment of building a model, you can withhold a portion of the most recent data from the model to see how well the model compares to the real data over that period. That period for which data is held back becomes the backtest. 

Note: You have the option to 1) use the automatically-generated backtest results exported from PlanIQ, or 2) build a custom process to evaluate forecast performance. The following sections describe how and when to use either approach. 

Out of the box: PlanIQ automated backtest period and accuracy metrics 

PlanIQ automatically generates some forecast accuracy metrics as soon as you build a forecast model. The automated metrics are there to help quickly check overall accuracy, especially when building multiple forecast models in succession with the same data set. These metrics include RMSE, MAPE, MAE, MAAPE, sMAPe, and MASE, and an overall model quality metric, which are described in more detail in Anapedia Understand Advanced Metrics.

The backtest window for these automatically generated metrics is the length of the PlanIQ forecast horizon. The automated metrics are calculated and aggregated across all items and all periods in the backtest window. 

However, you likely want to know the accuracy period and item to compare relative performance across items and over time, which is not captured in the automatically generated metrics. 

Therefore, it is highly recommended that you calculate these metrics within Anaplan to ensure the forecast performance evaluation is aligned to the business use case, especially if you are dealing with multiple items that may perform differently depending on the underlying data. This way you can ensure the model forecast accuracy is consistent across time periods and items. 

To be able to drill down to the forecast performance at the item time period level, you can import the same backtest results that are used to generate the automated accuracy metrics, and then calculate variance between forecast versus actuals (backtest results). See Anapedia Import a Backtest for step-by-step instructions. 

Custom built: User defines back-test window and timeframe  

Instead of relying on the automatically-generated backtest results, you may also create your own backtest window and calculate forecast accuracy metrics yourself in Anaplan. There are several reasons to do this. First, you may want to customize the length of the backtest window to be shorter or longer than the automatically-generated results. However, keep in mind we recommend using a backtest window that is the same length as the forecast horizon as a baseline. That way you can evaluate performance across the entire forecast window. Especially if there are major seasonal or cyclical trends that impact business, you should try to capture the full cycle in the backtest window. 

Another reason to construct your own backtest window is because the automated back test results are created at the time the model is built and not updated thereafter. Therefore, we recommend that you independently calculate accuracy metrics on an ongoing basis for reporting purposes until the point that you can use the real PlanIQ forecast results instead of the backtest simulation results. 

After obtaining the backtest results, the next step is to calculate accuracy metrics based on a comparison of forecast versus actuals (backtest results). For more guidance on how to select which accuracy metrics are best suited to the business use case, see Anapedia Understanding Advanced Metrics.

How to build a backtest window for forecast accuracy testing: 

1. Identify the period that covers important seasonality while still conserving sufficient historical data to forecast; for example, the last year. 
2. Build a PlanIQ module filtered view as if it were a given past date; for example, as if it were a year prior. Save a new export action based on a saved filtered view and create a new PlanIQ data collection.
3. Move forward in single period increments to simulate a real-world forecast. For example, monthly, quarterly forecasts for the past year, or weekly forecasts over a period. 
   • You can do this manually by adjusting the saved view referenced by the data collection forecast action.
   • Or, you can partially automate this by creating a module that manages a time filter that is referenced in a saved view. 
4. You can use a similar approach to monitor performance on an ongoing basis.
5. Now that you have forecast versus actuals (backtest results), you can compare multiple accuracy metrics. 
6. Review the different considerations for selecting which accuracy metrics are best suited to your use case. It is ideal to test multiple metrics. (For more details about how to select which statistical accuracy metrics to use see Anapedia Understanding Advanced Metrics.) 
7. If possible, translate into real-world units such as quantity or dollar value. This will provide the best approximation of the most likely cost of accuracy for different scenarios.

Example forecast accuracy dashboard

When should you assess forecast accuracy?

Throughout the process of building an operational PlanIQ forecast there are several occasions when you will likely need to assess forecast accuracy:

• Comparing the alternative models and selecting the best one
• Estimating the expected performance of the future forecast
  • It is important to assess the risk of over- or underperformance, and to be able to plan for variance from the point forecast.
  • Establishing accuracy metrics can also help identify if the forecast deviates from expected performance.
  • Monitoring forecast performance over time and determining when to re-train with new data as old model becomes deprecated.
• Identifying potential areas with poor performance (e.g., times of year, regions, product groups, etc.) before deployment and take corrective measures
  • Performance may improve with addition of related time series or metadata.
  • You may detect extreme and unpredictable values that bias the model, and you can remove them by using the “exclude_value” option (for more on the exclude value function, see Anapedia).

Which factors impact forecast accuracy?

For business users, one of the main challenges to evaluating the performance and accuracy of time series forecasts is that it is extremely dependent on unique business context and use case.

There are many factors to consider as you establish a forecast evaluation process including:

1. Seasonality and other cyclical influences
2. Length of training period
3. Forecast horizon duration
4. Business cost of over- vs under- forecasting
5. Sensitivity to user inputs and assumptions
6. Periodicity of the dataset (monthly, weekly, daily, hourly, etc.)
7. Industry and particular line of business
8. Single-item forecast variable (SKU, Group, etc.) vs. multiple items
9. Nature of item hierarchy, at which level of hierarchy are business decisions made
10. Other factors specific to your industry or use case, such as business evaluation criteria that are impacted by forecast results. Consider that different stakeholders or business groups may be relying on the same forecast results for different purposes and those different groups may be interested in different metrics or over different time frames (short- vs long-term performance).

It is recommended that you note the above factors for your use case and align with business stakeholders on which metrics are most important for decision making.

Evaluating multiple approaches

Different approaches to forecast accuracy will be better suited to your specific use case. Many use cases require a combination of methods and metrics to provide accurate performance evaluation.

It is important to consider multiple approaches to calculating forecast accuracy when implementing a new forecasting project with PlanIQ. When appropriate metrics are identified, performance should be monitored on an ongoing basis. As shown below, we recommend benchmarking the PlanIQ forecast against any existing forecasts.

Example comparison of actuals vs PlanIQ and other forecasts

In this article, I explore functionalities that PlanIQ offers using a sample data set and the preparation steps outlined in the Anaplan Academy's Course: Setting up Anaplan data for PlanIQ. The sample data contains three years of weekly sales volume by product item along with the item price and related data that indicates whether there is a promotion during the week. I also created a report module and some UX pages for easy visualization of the functionalities.

Preparing the Anaplan models:

Before we can configure PlanIQ to make predictions, we need to have several things ready in the Anaplan model.

Anaplan modules

In general, there are two groups of modules that need to be built in Anaplan.

1. Modules that will store the source data, which can be further categorized into:
   • Historical data - main set of data used to generate the forecast (ex. sales volumes)
   • Related data - additional drivers that can impact the forecasted values such as product price or promotion schedule. Related data is optional.
   • Attributes - characteristics of the data that can help identify patterns like product size. Attributes data is also optional.
2. Module that will store the forecast results

Below is the model map of those modules and a custom report module that combines the forecast results with the actuals for easier presentation on the UX pages.

Anaplan actions

There are two groups of actions that need to be built in Anaplan.

1. Export actions - to load the source data from Anaplan the model to PlanIQ (one action for each source data module)
2. Import action - to load the forecast results from PlanIQ back into the Anaplan model

Configuring PlanIQ:

Once we have the Anaplan model ready, we need to set up our objects in PlanIQ:

1. Data collection - data set that will be forecasted
2. Forecast model - algorithm that will be used to train forecast model
3. Forecast action - to generate predictions based on the forecast model and import the results back to Anaplan model

Data collections

Here, I created a data collection named Historical Sales Data Collection_2

with the following properties:

Here is a quick look at the data for a particular product item:

Forecast models

I chose to create a forecast model using Anaplan Auto ML which is supposed to find the best algorithms that fits our data set among all machine learning algorithms within PlanIQ. A list of the machine learning and other algorithms can be found here. 

PlanIQ will train the model that we have specified using the data set provided. During model training, the data set will be divided into two sets: a training and a validation set. The training set is used to build the model parameters while the validation set is used to find the optimal hyper-parameters based on forecast accuracy metrics.

Below is an example of the forecast accuracy metrics from the training in addition to the overall model quality. To know more about each metric, please click here.

However, reading the accuracy metrics alone hardly give us an insight into the performance of the forecast model. In this case, PlanIQ provides us with the ability to import backtest data so that we can compare the forecast result with the actuals of the data set that is used to train the model. Importing backtest data can be done via the Overview tab of our forecast model. In my opinion, it would be helpful if PlanIQ indicates which algorithm AutoML selected to the user has more information. 

Reviewing the backtest data allows us to consider whether we should try different forecast algorithms. Below are backtest data from two other algorithms that I also setup: ARIMA which is part of statistical algorithms and Deep AR Plus which is one of the machine learning algorithms available in PlanIQ.

It appears that ARIMA generates a smoother forecast, which is due to the moving average component of the algorithm. Both Anaplan Auto ML and ARIMA look like they perform well. They are able to track the actuals curve. This can be confirmed by their metrics. AMIRA and Anaplan Auto ML's are very different except the MAPE. However, unlike Anaplan Auto ML, ARIMA cannot take advantage of related data and attributes. On the other hand, as Anaplan Auto ML needs to run through different algorithms, training time takes longer. Deep AR Plus does not seem to fit the data well and the metrics are also much higher (lower is better) than the other algorithms.

In addition to producing the forecast, PlanIQ also helps us understand the trend and seasonality of our data, which is useful when working with time-series data. In the picture below, though the data set shows a flat yearly trend for the past 53 weeks, we can see that there is a clear quarterly seasonal pattern in the data. 

Forecast actions

Forecast actions generate the predictions for future periods based on the model parameters built during the training. They also import the results back into the Anaplan model.

PlanIQ generates a probabilistic forecast range by allowing us to specify the lower and upper quantiles, in this example 0.1 (10%) and 0.9 (90%) respectively. Under this confidence interval, we can expect the forecast lower and upper bounds to include the true observed values 80% of the time.

Below is an example the forecast range produced. P1 indicates the lower bound of the forecast at 10% quantile, P2 the median, and P3 the upper bound at 90% quantile.

As PlanIQ provides a number of forecast algorithms, we can compare how different algorithms produce the forecasts. Below is a P3 forecast comparison of three algorithms on the same data set.

We have seen that PlanIQ provides a wider range of forecasting capabilities to the business users, expanding options beyond simple approaches like using the last available actuals or an average over the past few periods. Please refer to Anapedia for more detailed information. 

Start here: PlanIQ overview and resources

Are you are interested in starting time series forecasting with PlanIQ on Anaplan, but you don’t know where to start? You have come to the right place!

Here are two resources to get you started:

• Start your journey with Anaplan PlanIQ!
  What is PlanIQ, who is it for, use case examples and all the relevant documents and articles you should be familiar with when starting your journey with PlanIQ!
• Join the PlanIQ User Group!
  In the user group, we discuss all things PlanIQ. You will find a forum and helpful resources.

PlanIQ Best Practices

In the articles below, you will find many of PlanIQ's best practices.

Learn how to deal with outliers, how to manage NULL values, how to use forecasting quantiles and many more!

	PlanIQ - Deep dive on the Algorithms under the hood Learn more about Baseline time series algorithms, Flexible local algorithms and Neural network algorithms.
	PlanIQ - Dealing with outliers Outliers in time series data are values that differ greatly from the rest of the time series. Learn how to handle them as part of PlanIQ!
	PlanIQ - How to use item attributes to refine your forecast Metadata attributes are static, non-time dependent categorical text features that describe the items in the historical time series. Learn how to use this data to improve your forecast!
	PlanIQ - How to manage NULL values Null represent missing values for specific points in time. Learn how to manage them within your data set.
	PlanIQ - Probabilistic forecasting using forecast quantiles PlanIQ algorithms produce probabilisitic forecast quantiles, which refer to a distribution of possible values.  Learn more about those values and how to use them wisely based on your use case!
	PlanIQ - Algorithm selection by item: Mix and Match your forecast! What is algorithm selection by item?  Learn more about "model blending" - the use of more than one algorithm to produce optimal forecasts across multiple items.
	PlanIQ - New Product Introduction All you ever wondered about starting your forecast from scratch!
	PlanIQ - Design and build your item list for forecasting What is the best practice for concatenation in Anaplan?  A step-by-step overview for creating a concatenated item list.

There are three main types of outliers:

1. Point (global) outliers: a value is far from the rest of the values in the time series.
   
2. Contextual outliers: the point in time at which the value appears makes it an outlier when measured against other points in time.
   
3. Collective outliers: a collection of values that as a group deviate from the rest of the values in the time series.
   

Point outliers and contextual outliers are more common forms of outliers, and therefore will be the focus of this article.

Outliers can be detected in many ways. For instance, one could visualize the time series data and see if any outliers stand out visually. Another common method is to calculate the number of standard deviations from the mean value of the time series, and flag values that are several standard deviations away from it. Outliers can also be detected by creating a forecast model using the time series data, predicting the values in each time period and identifying the times where the prediction errors are substantial.

Once an outlier has been detected, it’s important to investigate what has caused the outlier to occur, if there is an option to do so. After establishing the reason for the presence of the outlier, consider whether any adjustments need to be made to the data.

If the outlier originates in an error, it’s recommended to take out the value from the time series by removing the observation from the data, or by replacing the value with a more suitable value.  This could be done in several ways. For instance:

1. Leverage the Exclude Value functionality, which will replace the outlier a value based on an automated logic
2. Create a forecast model and predict the value for the period in which the outlier is present, then replace the outlier with the forecasted value
3. Replace the outlier manually

If the outlier represents a real observation (e.g. fluke, rare anomaly), consider whether the time series forecast model should account for such occurrences. For instance, it is generally recommended to remove outliers that are caused by one-time isolated events, as the forecast model would most likely not be able to predict those going forward. The outliers can also be replaced according to the methods mentioned above. In addition, related time series data (drivers) such as special event or holidays can also be used to better guide the algorithms in forecasting when outliers may occur.

Note that keeping outlier values in the time series would typically result in the forecasting of values greater or smaller compared to other predicted values. When tested against backtest windows, one can compare the accuracy of the forecasts with and without the presence of outliers, and determine which approach results in greater accuracy.

Questions? Let us know in the comments.

Contributing authors: Nitzan Paz, Christophe Keomanivong, Frankie Wolf, Andrew Martin.

This article serves as a guide to help PlanIQ users deal with scenarios in which their datasets include missing values or ‘empty’ cells.

It is important to differentiate between a true zero and a missing value. Moreover, a dataset with many missing values (a sparse dataset) is different than a cold start scenario where little or no data exists because a certain product is new to the market. For further information, please refer to the New Product Introduction article.

Many missing values in a dataset may impair the forecast accuracy. This is especially true for more recent (later) data in the time series. Our recommendation is to not have more than 30% of missing values per time series (per item). PlanIQ limits the missing values per item to 50% in the historical data. If a dataset contains more than 50% missing values, PlanIQ will display an error message indicating that too many values are missing.

PlanIQ assumes that datasets that originate in Anaplan modules with records set to zero are true zeros and therefore will be treated as such. In addition, in instances where a custom time dimension is used (i.e. where the time dimension based on a list of timestamps), records with missing timestamps will be treated as zeros as well (rather than missing).

There are multiple ways to deal with missing values. For instance:

• Use the ‘___exclude_value’ column so that missing values are not interpreted as zero (see Anapedia for more details). If a value is marked with 1 using the ‘___exclude_value’ column, PlanIQ will automatically fill it in with the mean of the values around it.
• Manually review and fill in missing values.
• Aggregate the data by using reducing its frequency (e.g. instead of a daily frequency, aggregate the data to a weekly level).
• Aggregate multiple distinct items to a category of items based on item hierarchy or other dimensions such as location (for example, combining multiple cities to a state level).

One more way to deal with sparse datasets is to use robust forecasting algorithms such as CNN-QR and DeepAR+. These algorithms take longer to train and typically require more historical data, but are better suited to handle sparsity in time series data. In scenarios where it is unclear which algorithm to choose, it is recommended to choose Anaplan AutoML; PlanIQ will then explore different algorithms and select the one producing the most accurate forecasts for most of the items.

Finally, when using related data, and future values are missing, CNN-QR should be chosen as the forecasting algorithm since other algorithms expect each related time series to contain future values (with a length equivalent to the forecast horizon).

While time series datasets will often contain many missing values, PlanIQ enables users to maximize the value of their datasets and generate accurate forecasts using the aforementioned approaches.

Got feedback on this content? Let us know in the comments below.

Contributing authors: Nitzan Paz, Christophe Keomanivong, Frankie Wolf, Timothy Brennan, Andrew Martin, and Evgenya Kontorovich.

Algorithm selection by item is the use of more than one algorithm to produce optimal forecasts across multiple items. Instead of using forecasts generated by a single algorithm for all items in the historical data module, where the algorithm is selected based on best overall performance – multiple algorithms are selected based on forecast accuracy by item (or groups of items).

Each forecast model trained in PlanIQ allows for a single algorithm to be used in order to generate forecasts for the items in the historical dataset. In order to use more than one algorithm, multiple forecast models and actions should be configured and executed. Then, the accuracy of the forecasts should be calculated per algorithm and per item; The algorithm which produces the most accurate forecast is then selected for each item, resulting in the final forecast.

Evaluation

First, define a backtesting window, which is a past period of time that is included in the forecast horizon. Doing so would allow for forecasts to be evaluated against past observations (actuals). Then, calculate an accuracy metric of choice (e.g. RMSE, MAPE) at the individual item level over the backtest period. Finally, compare the metric across all algorithms to identify the best performing algorithm for each item in the back testing window.

Requirements & Considerations

All algorithms can theoretically be used for algorithm selection by item, though the number of records and/or the desire to use related data or attributes may determine which algorithms can be leveraged for this purpose. For instance, data collections must contain at least 300 historical values to meet the neural network machine learning algorithms (DeepAR+ and CNN-QR) data requirements. In another example, if related data should be taken into account for forecasting, then only Prophet and the neural network machine learning algorithms would apply.

The level at which the algorithm selection should be made can be typically determined by the data; Selection at the individual item level will provide more granularity but could lead to overfitting compared to group/category level algorithm selection.

How To

Below is an example of an algorithm selection by item process:

1. Create a data collection
2. Create forecast models based on the new data collection for all algorithms that the dataset captured in the data collection is suited for and you wish to evaluate
3. Create a forecast action for each forecast model created
4. Calculate the forecast accuracy by item for each algorithm
   1. For each algorithm, calculate the forecast metric for each item
   2. Compare the forecast accuracy across the different algorithms for each item
   3. Select the best performing algorithm for each item based on forecast accuracy
5. Create a line item populated with the final selected forecasts for each item

Got feedback on this content? Let us know in the comments below.

Contributing authors: Nitzan Paz, Christophe Keomanivong, Frankie Wolf, Timothy Brennan, Andrew Martin, and Evgenya Kontorovich.