Integration issue- the last line is split across two files (chunk 1 and chunk 2)

CommunityMember111275
CommunityMember111275
in Integrations and extensibility

Hi Team,

while calling through API for export for integration, if the file size is large, it is divided into chunks of 10MB each (e.g. if the file size is 890 MB, it will be divided into 89 chunks). As a result, we are getting incomplete record on the last row of every chunk and the rest of the records are being stored in the next chunk due to the size limitation of 10 MB. Please suggest some ways to get rid off this issue so that each chuck contains complete rows (even if the size of the chunks will be slightly less than 10 MB) and the partial row in chunk1 can get shifted to chunk2 as a complete row.

Tagged:

Answers

  • rob_marshall
    rob_marshall

    @CommunityMember111275 

     

    I must admit, I am not up to date on the API's, but I did experience this with earlier releases of Anaplan Connect and believe it was solved in 1.4.4.

  • AnyaS
    AnyaS
    edited January 4

    We encountered the same issue as @CommunityMember111275: during the export, Anaplan chunks the data into several files 10Mb in size each, while splitting the record row in the middle and, what's more troublesome, even in the middle of field. We've attempted to use Snaplogic to merge the automatically produced 10Mb chunks but were not successful and had to implement other the workaround. I am curious, if anyone had found a better solution for this issue.

Categories