Anaplan restful api uploads with zip files

tomz

Hello,

I have a question about using the Anaplan integration API V2 guide.  Is it possible to upload a csv document inside a .zip file to Anaplan?  I'm able to manually upload a zip file with a csv in it, which works, but cannot figure out if its possible to do it with the API.   I'm trying to do this with Postman but the data being received in Anaplan is all binary.  I'm basically trying to accelerate my uploads via the api using large zipped files.  Do I need to post with a certain header? I'm not sure if this is possible or not?

 

My only headers are:

'Authorization: AnaplanAuthToken bla'

'Content-Type: application/octet-stream'

 

 

 

Thanks,

Tom

karank_*4578

your URL looks like you are uploading file as single chunk ? I cannot see the entire URL but i can make out with 80% accuracy as you have file/some file ID

 

If this is TRUE then you cannot compress the file and load . It defeats the purpose . Uploading a file as single chunk means your file size is small(less than 1 mb) therefore you dont need to compress and you PUT it onto the server by streaming it as an octet stream.

 

you need to Chunk the file into smaller parts and then use gzip application  method 

 

in other words your URL should look like below 

 

/workspaces/{workspaceID}/models/{modelID}/files/{fileId}/chunks/{chunkId} -X --upload-file chunk-aa

 

if you need more info on chunking please refer to 

https://anaplanbulkapi20.docs.apiary.io/#reference/upload-files under Upload a file in chunks

 

 

 

Misbah

@tomz 

 

It is not possible with Anaplan Connect. Below is the excerpts from Anaplan Integration guide. But I am not aware how REST API behaves. I believe it should fall on the same lines

 

 

Thanks,

Misbah

karank_*4578

@tomz 

 

First of all Excellent question ! 

 

You need 

 

1. compress using gzip format
2. change your header to Content-Type:application/x-gzip

 

Hope this helps

 

 

tomz

I compressed the file to g-zip and tried to send with the header you provided and I get the message: "The server refused this request because the request entity is in a format not supported by the requested resource for the requested method."

karank_*4578

I hope your request method is POST 

 

Perhaps share your postman screenshots so that i can debug 

tomz

 Was using a PUT, which was working with a plain CSV.  Just tested with POST, same bad response.  Maybe its the way i'm uploading the file?

 

Headers:

 

Body:

 

 

Response:

 

Thanks

tomz

Oh okay I see. Yes I was doing this as a single chunk. I'll split and try again. Thanks.

tomz

I just tested it out again and splitted a file into the chunks and converted into gzip and it worked,  Thank you.  I had also tested another method:

- I gzip'ed a single file, posted the chunk count of 1,

-Then in my upload url I had to switch it to  {fileid}/chunks/0 ,  instead of the /{fileid}  by itself

- Then uploaded it with a single gzip compressed file.

And it worked! 

 

Thanks!

karank_*4578

glad it worked
uploading a single compressed file as 1 chunk will work as well but its not recommended as it puts load on the server and if if ur upload fails you will have to reupload the entire file.

elchen8923

I used the following sample python codes from Jesse wilson (https://community.anaplan.com/t5/How-To/Anaplan-API-2-0-Python-Library/ta-p/38139) to upload a csv file using API v2.0, it works perfectly.

 

To upload a CSV, it reads a CSV file into a buffer and then does stream_upload

 

upload = anaplan.file_upload(conn, "<file ID>",<chunkSize (1-50)> , "<path to file>")

with open('/path/sample.csv', "rt") as f:
buf=f.read()
f.close()
print(anaplan.stream_upload(conn, file_id, buf))
print(anaplan.stream_upload(conn, file_id, "", complete=True))

 

The header used to upload a CSV is -

 

put_header = {
"Authorization":authorization,
"Content-Type":"application/octet-stream"
}

 

I modified the above code to upload a zip file 

 

buf = zipfile.ZipFile ('/path/sample.zip', "r")
logging.info(anaplan.stream_upload_zipfile(conn, file_id, buf))
logging.info(anaplan.stream_upload_zipfile(conn, file_id, "", complete=True))

 

In stream_upload_zipfile function, I used put_header

 

put_header = {
"Authorization":authorization,
"Content-Type":"application/x-gzip"
}

But the zipfile cannot be uploaded, what is my problem here?

 

Thanks

ben_speight

Zip and GZip are different. GZip is a format for compressing a stream of bytes, whereas a Zip file is an archive containing a folder/file structure where the files are compressed using different compression schemes. The API expects gzip-compressed chunk content with Content-Type: application/x-gzip.

elchen8923

Thanks Ben for your help.

 

I was able to get this work by first zipping up the file using python function gzip, then I read the gzip file using "rb" and upload the gzip file  using Content-Type: application/x-gzip in the header

 

with open(file.gzip, "rb") as f:
buf=f.read()