Exporting multiple modules using API takes too much time

AHmedLattali

Hello Community, 

 

I'm trying to export multiple modules (7 modules) using the API. but it takes up to 4 hours to do that.

The size of the biggest module is about 600Mo.

 

My questions are the following :

 

Is there way to do a full export of the Anaplan modules (like creating a dump) ?

Is there some possibilities to accelerate the extraction using the API ?

 

Any help will be appreciated.

 

Best regards

Ahmed

 

jairamkamath

@AHmedLattali Kindly first revisit and validate if an entire module has to be exported. If not mandatory, below are few options you can consider:

Option 1: Something that we have implemented is split the models & data hubs into 15 smaller models. 
This allows APIs to run parallel job on all 15 models thereby reduced the run time by approximately 15-20 times; as in some cases it run much faster. 

Option 2: If you are continuing to use only one model - another options would be to use boolean and filter our any ZERO values in the export saved view. Also ensuring you do not export summaries and only one level from each list. And then exporting data through saved view. If there is lot of sparsity in the data being exported, this process can improve and may complete within an hour.

Option 3: If the model is bulky or memory intense, the model opening time may be also included in the total time of export. You may also want to check if you can use a dummy job and have model up before export job is triggered, that way saving another 15-20 minutes.

Option 4: Check if you can reduce the list values in export module by creating a new module by using list subsets. This can also reduce the exports by good numbers (we have seen unto 50% benefit due to reduced sparsity/cell count).

 

Option 5: The last option to save time and if the data is not very important; where if it is just being exported as a backup is to archive the model itself using "Copy and Archive" feature which can complete in 2 minutes on 100GB model.

There may be more options available, these are something I can think on top of my mind.