New Contributor

Delta Load

Is it possible to do a delta load in the Anaplan model? Lets say one flat file is uploaded at 10.00 AM with 5 million records and the same file is uploaded at 11.00 AM with 6 million records. I just want to upload the new 1 million records into my model. How can we achieve that? 

 

Thank you in advance!

3 REPLIES 3
Highlighted
Master Anaplanner/Community Boss

Re: Delta Load

Hi,

 

Assuming you have 2 models, i.e. Data Hub and Spoke Model.

 

You do the initial load into Data Hub, there's now 5 million transactions ID in the Transaction# List, and you have 2 modules, i.e.

1. Transaction# Properties with line items of CODE, NAME, and any properties of Transaction#

2. Transaction Module with the transaction data of Sales, Cost, etc etc

 

You can add 2 line items in the Transaction# Properties Module, i.e.

1. True Flag = TRUE, format = Boolean

2. Not Exported?, format = Boolean

 

Create a Saved View in the Transaction# Properties Module called Delta, with a filter where 'Not Exported? = TRUE'.

Create a Saved View in Transaction Module, with a filter where 'Transaction# Properties.Not Exported? = TRUE'.

 

Then perform the 3 actions below:

1. Import Transaction# List to Spoke Model using Saved View 'Transaction# Properties.Delta'.

2. Import Transaction data to Spoke Model using Saved View 'Transaction Module.Delta'.

3. Import 'True Flag' to 'Not Exported?' in 'Transaction# Properties' module.

 

The idea is that any data that has been sent from the Data Hub to the Spoke Model will be flagged, so that only new data will have a blank 'Not Exported?' and is filtered out in the Saved View ready for the next export.

 

Having said that, you should consider 2 items below:

1. 5 million data point sounds like a lot of data, is it daily transactional data? Do you REALLY need this in Anaplan? (Follow 'N' of PLANS, is it necessary?)

2. If you only need an aggregation, do have a discussion with your database colleague, and get them to aggregate before sending the DELTA to Anaplan, so that you don't even need to do the steps above.

 

For more info on the best practices on Data Hub, you can refer to https://community.anaplan.com/t5/Knowledge/Building-a-Data-Hub/ta-p/33593

 

Thanks,

LipChean

Highlighted
New Contributor

Re: Delta Load

Hi LipChean,

 

Thank you for your reply. 5 M records were just the assumption as I was trying to understand the concept. It worked for me. Thank you again!

Highlighted
Master Anaplanner/Community Boss

Re: Delta Load

Hi,

 

Cool. Please mark as solution, so that this topic is closed.

 

Thanks