What is Data Management System that was talked about during Anaplan Kick-off this year?

Accepted answers

The Anaplan Data Management System is not currently available and as such all statements below are subject to our standard safe harbour statement and are subject to may change.

The Data Management System is aimed at a technically savvy business user that is comfortable building models in Anaplan and working in Excel. They have a good knowledge of the business problem to be solved and the data that is available to do it, but are not necessarily Master Anaplanners, experts in SQL or database management. We call them Data Modellers.

Connect

Data Modellers configure connections to common on-premises and cloud source systems through intuitive interfaces tailored to the data source. Modellers will authenticate, be guided through the selection of source data defaulting to delta loads where that is supported by the source application and schedule execution. Executions will be logged to support failure alerting, monitoring, summary reporting and housekeeping.

The modeller will be able to connect to Anaplan models as an inbound data source to support inter model data transfer and reporting and analysis capabilities outside of Anaplan.

The approach is to build a connector framework into which connectors for specific data sources can be introduced quickly and without significant effort. In the longer-term partners should be able to certify and register additional connectors.

Convert

Data Modellers restructure the extracted data into the format required for modelling and planning by configuring a repeatable sequence of transforms through an intuitive graphical interface. In addition to basic primitives (filter, lookup, join, calculate, sum, union) there will be extended capabilities for OLAP such as balance and validate hierarchies, aggregation to dimension levels, and data quality and anomaly detection capabilities.

Catalogue

The catalogue exists at a tenant level and Modellers across workspaces in the tenant can consume accessible Data and Master data objects into their models. It also allows the Data Modeller to assign semantics to objects to simplify, accelerate, and automate the mapping for consumption of data into Anaplan models.

The Data Modellers will have access to statistics for objects in the library (for example where these are consumed, how frequently and recently, volatility, quality) for housekeeping and maintenance purposes.

Consume

Data Modellers will be able to create and update lists and data in their models from accessible objects in the DMS catalogue for faster model development and maintenance.

When consumptions are executed, these are logged for success / failure, data volumes, execution time and rejected data so that the Data Modeller can report on statistics of catalogue object use.

In alter releases:

An end user can drill from the UX through to the underlying details in the DMS.
A page builder should be able to surface accessible data from the DMS through the NUX so that this can be displayed to the end user based on them having suitable permission to that data. Combining this with the connector for Anaplan modelling allows for reporting from static plans which have been published into the DMS without a need to open the underlying model.

What the Data Management System is and what it is not

The DMS will be the solution for extracting, transforming, enriching, validating, staging, and orchestrating the incoming and outgoing data for Anaplan.

It is not:

A replacement or alternative persistence layer for Anaplan models.
An enterprise or generic ETL tool.
An enterprise data lake or data warehouse, although it could be considered as a “DataMart for your planning data”.
An alternative modelling or calculation engine, although it can transform and enrich data it does this as part of data integration and not interactively.

All comments

JohnC

The Anaplan Data Management System is not currently available and as such all statements below are subject to our standard safe harbour statement and are subject to may change.

Connect

The modeller will be able to connect to Anaplan models as an inbound data source to support inter model data transfer and reporting and analysis capabilities outside of Anaplan.

Convert

Catalogue

Consume

Data Modellers will be able to create and update lists and data in their models from accessible objects in the DMS catalogue for faster model development and maintenance.

When consumptions are executed, these are logged for success / failure, data volumes, execution time and rejected data so that the Data Modeller can report on statistics of catalogue object use.

In alter releases:

An end user can drill from the UX through to the underlying details in the DMS.
A page builder should be able to surface accessible data from the DMS through the NUX so that this can be displayed to the end user based on them having suitable permission to that data. Combining this with the connector for Anaplan modelling allows for reporting from static plans which have been published into the DMS without a need to open the underlying model.

What the Data Management System is and what it is not

The DMS will be the solution for extracting, transforming, enriching, validating, staging, and orchestrating the incoming and outgoing data for Anaplan.

It is not:

A replacement or alternative persistence layer for Anaplan models.
An enterprise or generic ETL tool.
An enterprise data lake or data warehouse, although it could be considered as a “DataMart for your planning data”.
An alternative modelling or calculation engine, although it can transform and enrich data it does this as part of data integration and not interactively.

abhi4246

@JohnC Thanks for the insight !

NikhilGuntuku1

@JohnC Is there any list of tools or servers to which DMS can connected? And is there any timeline for when we can expect the DMS release?

PujithaB

@JohnC Thank you for Sharing. Excited to Know more about this in future.

JohnC

The current time line is for a Beta in April with Early Adopters being able to leverage Anaplan DMS from July of this year.
In the Beta data can be loaded from S3, local CSV file and Anaplan models.

In the July release we expect to add Azure Blob, GBQ, SFDC, Snowflake and SAP through OData.
Post July we have plans for connectors to Workday and SQL Databases.
Additional connectors will be added based in customer demand.

PujithaB

@JohnC Great to Know this. Looking forward!

alexpavel

@JohnC : thanks for sharing the info. All clear from architecture point of view and looking forward to see it implemented. :)

How will work the "automated" data flows using DMS? Will be similar to CloudWorks integration flows?

In that case, how would you "see" an automated integration using a local CSV file?

Will be still needed of some external tools to automate the upload the file using some REST APIs and triggere the DMS integration flow? If yes, will these REST APIs be released in Beta version ?

thanks

Alex

JohnC

Orchestration and Scheduling will leverage Workflow capabilities - but the steps will be DMS tasks. Upload of local CSV files is manual - until we provide a local file connector. For files in S3 or in another source with an available connector the steps to extract / load the data from that source, transform it and publish into one or more models will be handled through this orchestration capability, either run manually or on a schedule.

The available schedules will be similar to those in CloudWorks but be based on a starting time range and / or throttled to avoid excessive loading on the platform at the top and bottom of the hours.

We will provide Rest API's to allow customer to push data into the DMS in some future release. Our priority is on avoiding the need for the use of API's at this stage.

Nasabulkk

It's like a behind-the-scenes maestro coordinating tasks seamlessly. Right now, uploading local CSV files is a bit hands-on, but they're working on a local file connector to automate that. For files in S3 or other sources, the DMS takes charge of extracting, transforming, and loading data into models, either manually or on a set schedule.

Oh, and they're planning to introduce Rest APIs down the line for customers to push data into the DMS. It's all about making things easier without diving into complex APIs just yet.

By the way, speaking of systems, have you ever wondered, what is ERP and how does it work? It's like the backbone of many businesses, managing everything from finances to HR seamlessly. Pretty neat, right?

NikhilGuntuku1

@JohnC For early adaption of the Beta version to test the functionality and workability, it is available for all customers or is it only for a few customers?

JohnC

The Beta will be made available to a small number of existing Anaplan customers. Please work with your Anaplan Account team and Business Partner if you are interested in getting involved.

Quick Links

Can you help?

Unanswered questions

ERROR 16020 |-- Anaplan API: Invalid key path
Hello everyone, We are trying to use Anaplan Connect in order to push data from a dwh to Anaplan Data Hub. To do so, we need an authentication by token. We can't make it work, we receive the following error: ERROR 16020 |-- Anaplan API: Invalid key path Any suggestions? Thank you all in advance for the help! Michele
Azure SQL Connection, database name with Hyphens in Data Orchestrator
I am attempting to connect to an azure SQL database vis at the MSSQL connector in data orchestrator but am having issues and I believe it is due to hyphens in database name (which is a fairly standard naming convention I will note!) Example below (fake credentials) - has anyone else encountered similar and is there a way…
AnaplanXL Difficulties
Has anyone found success using Anaplan XL with Polaris models in a shared drive with or without concurrent users? My team has experienced a variety of errors, login failures, etc. sometimes it works, sometimes it doesn't. We've been recommended to save and edit files locally, which helps but certainly with its limitations…