incremental synchronization

Is there any way to implement incremental synchronization of data loads?

which will only synchronizes the data in the application that has changed since the last sync from the data source?

Accepted answers

There are a few ways to go about only loading deltas. If you can track this in your data source (MySQL, SAP, etc), it offloads the burden to your database. If not, you can do a full load into a data hub, where locking the model during an import isn't consequential because users wouldn't be performing any work in that model - from there it's easy to perform a calculation to look for changes from the previous load. Create a boolean line item that is true when there are changes, and a saved view based on that from which you import into your spoke model(s).

Regards,

Jesse

rob_marshall

To add on to what Jesse is saying, create a new row in the data load set to True or 1. The process to load the data would be:

Run and action to set the boolean to False
Load the data (thus it will switch the boolean to a true where new records are created)
Load only the trues to the spoke.

Hope this helps

Rob

All comments

jesse_wilson

Regards,

Jesse

rob_marshall

To add on to what Jesse is saying, create a new row in the data load set to True or 1. The process to load the data would be:

Run and action to set the boolean to False
Load the data (thus it will switch the boolean to a true where new records are created)
Load only the trues to the spoke.

Hope this helps

Rob

abhay.kanik

@jesse_wilson @rob_marshall

Thanks for the Solution, It helped!

anildudam

@abhay.kanik,

Currently there is a requirement to implement incremental data loading from SQL DB to Anaplan through Anaplan Connect.

Could you please share some document or step by step process to implement.

Thanks & regards,

Anil

david.savarin

Ok, question there : what happens when a new load into the datahub is performed ?

Do you add the new ones as true as well ? Or only the newest data is flagged as true ?

If so, how you make sure that spoke models get loaded with the data before a new data load is performed ?

kartiendasolventoph

Hello @david.savarin , I have the same question about incremental data loading from Postgre SQL to Anaplan using Batch scripts. The new ones will add as well, and if there is similar data to anaplan from database, will get the updated ones from database and update to anaplan

Quick Links

Can you help?

Unanswered questions

12_ [COMPLETE GUIDE] Uphold Support Number
At Uphold customer support ⭐+1-888-355-2348 accounts may be temporarily disabled or restricted for various reasons. Raise a concern on priority basis at Uphold support ⭐+1-888-355-2348 if you notice any suspected malicious activity, or a problem during account recovery. To contact Uphold support directly at…
New UX Filtering for Hierarchy sibling
I have A6 as the parent list and two sibling lists: A6.5 and A7. The A7 list includes an attribute that maps to A6.5. In the UX, the user will select A6.5 using a content selector, and the grid (dimensioned by A7) should display only the A7 items associated with that selected A6.5. Since the selector is a page context and…
A reminder of Bulk Copy - Current year to Prior year version for YoY analysis
A quick reminder of the Bulk Copy functionality. Bulk Copy allows you copy large volumes of data from one slice of a model to another in a single, optimised operation, instead of using formulas or imports. Use case: copy a version (RF1) into a prior year version (PY RF1) using a versions list to allow for year-on-year…