Creating a Data Integration Schema for your Models

ryan_kohn

A data integration schema describes the integration and flow of data across systems. It will typically consist of a combination of diagrams, tables, and other visual components that provide the ability to quickly understand the flow of data. In an Anaplan context, a data integration schema outlines the movement of data from model to model, and to/from source systems. 

Here's an example of what a data integration schema might look like:

Unfortunately, there's no “easy button" for generating a data integration schema for your entire Anaplan environment. Solving this ultimately comes down to a combination of 

1. following best practices for data integration governance during the model building process, 
2. gathering details about your data flows from what is available in the Anaplan modeling experience, and 
3. leveraging the gathered information to build a data integration schema outside Anaplan. 

 

1. Governance and best practices 

Following best practices for action management in your models will make it easier to maintain details about your data integrations. Before building any new documentation, it’s a good idea to review your existing models and ensure that these best practices are being followed. 

1. Remove any import data sources, actions, and processes that you don't need. This will make it easier both to find things as well as prevent you from documenting details about mappings that aren't being used. 
2. Follow standard naming conventions for imports (Planual rule 5.01-01), exports (Planual rule 5.05-01), processes (Planual rule 5.02-01), and import data sources (Planual rule 5.06-01). You can read through Anaplan's recommended naming conventions here: Name conventions - Anaplan Technical Documentation 
3. By following numerical naming conventions, you can then order your actions so that related actions are together. This will make it easier to follow the sequence of actions. For example, grouping actions from the same process together in the order they occur will make it easier to build and maintain any external documentation for those action mappings. You can read through OEG’s best practices on action management here: OEG Best Practice: Import action management best practices. 
4. Whenever you deploy changes to production, be sure to update any external documentation so that it stays up to date. 

 

2. Collecting relevant data from your models 

To generate a map of your inbound and outbound data sources, you can combine data that's available within the model settings. Unfortunately, there are no APIs available today to get this information. You will need to export the information manually and paste it into an external tool for reference (e.g. a spreadsheet or a database). 

Export the list of Import Data Sources 

Whenever you create an import, you have the choice to use an existing import data source or add a new one (by uploading a file or connecting to another model). The list of import data sources is retained and this is your first stop for understanding the possible data sources for a specific model. 

 

Note: There isn't an “Export” button for the list of Import Data Sources, but you can highlight the entire grid to copy it, and then paste it into your spreadsheet. 

Tips: 

• When you see the Source Model listed as “-” and the Source Type is LIST, MODULE, or SAVED VIEW, then this means that the import uses an element from the current model as its source, as opposed to importing from a different model. 
• When the Source Type is FILE, this often indicates that the data is loaded from an external source system, but it can also mean that the data is expected to be manually uploaded by a user 

[If Applicable] Export the list of Source Models mappings 

You will need to pay special attention to situations where you have overridden the Source Model mapping using Edit Mapping. In this case, you will need to revisit and update your Import Data Sources spreadsheet from the prior step. In the Import Data Sources list, the original source model name (the first column in Source Models) is used, and not the overridden mapping (the Mapped To column in Sources Models). See example below. 

If you haven't overridden the mapping for any source models, you won’t need to export the Source Models, since all the information will already exist in the list of Import Data Sources. 

Export the list of Imports 

The list of imports is where you begin to connect everything together. For each import, you can see the Import Data Source ("Source Label"), along with ‌details about the Import Data Source (Source Object and Source Type). 

Note: There isn't an “Export” button for the list of Imports, but you can highlight the entire grid to copy it, and then paste it into your spreadsheet. 

 

Export the list of Actions to map Imports to Processes 

Identifying which actions are part of which processes is a manual step, but it will be easier if you have followed best practices around action management. 

Use the filter tools in your spreadsheet to filter by the Used in Processes column to see the specific actions that are part of a process. Note that it is possible for an action to be part of multiple processes. If you are looking to show a list of all actions in a specific process, you would have to compile that on your own. 

3. Build documentation and generate a data integration schema 

Gather the above details for all your models. Since model-to-model connections in Anaplan are “pull” based, you will need to get these details from the target model side of any model-to-model imports to see the connections between models. 

Once you have all the information, you can use it to build your documentation in a format that's meaningful to your business. 

Example 

Let’s say we have a Territory Planning model with a Data Hub, and the Data Hub imports data from a CRM system. 

For this example, our exported data might look like the below. 

Data Hub 

Import Data Sources 

Imports 

Actions 

Territory Planning PROD 

Import Data Sources

Imports

Actions 

Diagram

Finally, we can put together a diagram that shows the data flows at a high level:

Summary 

It takes manual effort to compile information about data flows across all your Anaplan models into a data integration schema. However, once you do so, you will have an easy-to-access and easily shareable overview of all the data movements in your entire Anaplan tenant, which can support any debugging efforts or data flow audits you may need to participate in. 

If you have any additional suggestions, please add them in the comments below. 

PujithaB

@ryan_kohn, thank you for describing in detail! Very Insightful!