Predictive Insights (PI) - How to Refresh a Predictive Model

NatalieC

What is a Model Refresh?

Predictive Insights (PI) leverages machine learning models to help users target their total addressable market more intelligently. The machine learning model uses historical account data alongside PI data to build knowledge on how to rank your accounts. Over time, the data set can change and the performance of the model can degrade. A model refresh is the action of retraining a Predictive Insights model with more recent data to add new accounts, update attribute data, and improve performance.

When to refresh your model

A PI model is trained on the account dataset loaded to it at the time it was built. Overtime, account data starts to change due to many factors such as: account targeting, sales plans, salesman incentives, and much more. Therefore, a model refresh should be initiated under the following situations to help mitigate these factors:

1. Following Performance Tracking, a user notices that the model is underperforming.
2. You wish the model to be trained on more recent accounts to adjust attributes and rankings. In this case, we recommend a model refresh if your prospect and customer data has grown by at least 30% in overall number of accounts.
3. A substantial volume of prospects converted into customers.
4. The model has been running for approximately 6 months. The model may be refreshed more frequently than this based on your business use case and the effects that refreshing the model may have on the processes that use it.

How to refresh your model

A model refresh is a simple process with PI. In order to refresh the model you will need to gather your customers and prospects in the module that was used to build your PI model previously. Most importantly, include the data used previously in your model plus the new data collected since the last time the model was built in both the account list. You will rebuild the model by using the rerun button that appears when clicking the triple ellipses on the “Predictive Insight Models” page.

The process should follow this order:

1. Add any recent accounts or additional accounts you wish to include in the rebuild to the module you previously used to create the model training dataset.
2. In Predictive Insight, go to the “Predictive Insights Models” page. Then find the Training model connected to the dataset in which you just added accounts for and click the Rerun option in ellipse menu. This will initiate the model refresh process.

Changes after a Model Refresh

Following a model refresh, you can expect to see that the scores and ranking distribution of accounts has shifted, where the extent of the change will vary. If you see that many accounts have changed in score or ranking, consider what has changed within the business in the time between when the model was first built and the time at which you refreshed the model. Given the updated volume of accounts, it may be possible to build a separate model using only the more recent data to produce a better optimized model.

In the case where you are refreshing the model based on the 6-month guideline, you may not see a substantial difference in the model; This is not a cause for concern as it means the more recent data closely resembles the data the model was originally trained on and has not made the model optimize itself much differently.