Data fields are offset when retrieving data in Chunks via API

AW25 · February 2020

I'm retrieving a large export via the API and because of the size of the file, Anaplan breaks up the export into multiple chunks. The issue is the column structure in the first and last record of each chunk isn't consistent. For example, in the attached image, the first chunk ended on the first row mid-way through the Version field and continued in the first row of the following chunk. Since a chunk can cut off at any point in the row, I don't see any practical way of piecing the data back together. Is there any workaround for this?

JaredDolich · February 2020

@AW25

I've honestly never seen that happen before.

Two suggestions:

First it might help is to set the chunk size.

You can read this post, scroll down to @Misbah solution. See if that helps.

This assumes you're using Anaplan Connect though.

The other, which is really unlikely but worth mentioning is to make sure you don't have any control characters that might be interpreted as a delimiter from your source module. Sometimes people use special characters, like the emoji set. These cause all kinds of problems.

Can you give us more specifics about which export method you're using?

Honestly, I've used almost all of them and never had this problem. You might also consider posing this to [email protected].

AW25 · February 2020

Hi @JaredDolich,

Thanks for the reply. I'm using the method outlined here in the API v2 documentation. I've attempted to get the data via Alteryx and Postman and had the same issue with both.

There aren't any special characters in this dataset. This break actually happens in the middle of number strings as well. The documentation says that the chunk size for Exports is set to 10MB but I couldn't find any information on changing that.

karank_*4578 · February 2020

@AW25

This is expected behaviour . What i normally do is

I open a file(this is my final output file) in write mode and keep it open and [Loop]then make a API request for chunk 0 , read content and write to the open file[LOOP] . I loop this until i reach the end of chunk count . Then i close and save the file . This gives me 1 complete file without breakage

I dont know how to achieve above in either Alteryx or Postman . Hoping someone posts more solutions here