Data fields are offset when retrieving data in Chunks via API
I'm retrieving a large export via the API and because of the size of the file, Anaplan breaks up the export into multiple chunks. The issue is the column structure in the first and last record of each chunk isn't consistent. For example, in the attached image, the first chunk ended on the first row mid-way through the Version field and continued in the first row of the following chunk. Since a chunk can cut off at any point in the row, I don't see any practical way of piecing the data back together. Is there any workaround for this?
The other, which is really unlikely but worth mentioning is to make sure you don't have any control characters that might be interpreted as a delimiter from your source module. Sometimes people use special characters, like the emoji set. These cause all kinds of problems.
Can you give us more specifics about which export method you're using?
Honestly, I've used almost all of them and never had this problem. You might also consider posing this to email@example.com.
Thanks for the reply. I'm using the method outlined here in the API v2 documentation. I've attempted to get the data via Alteryx and Postman and had the same issue with both.
There aren't any special characters in this dataset. This break actually happens in the middle of number strings as well. The documentation says that the chunk size for Exports is set to 10MB but I couldn't find any information on changing that.
This is expected behaviour . What i normally do is
I open a file(this is my final output file) in write mode and keep it open and [Loop]then make a API request for chunk 0 , read content and write to the open file[LOOP] . I loop this until i reach the end of chunk count . Then i close and save the file . This gives me 1 complete file without breakage
I dont know how to achieve above in either Alteryx or Postman . Hoping someone posts more solutions here