Anaplan restful api uploads with zip files

Hello,

I have a question about using the Anaplan integration API V2 guide.  Is it possible to upload a csv document inside a .zip file to Anaplan?  I'm able to manually upload a zip file with a csv in it, which works, but cannot figure out if its possible to do it with the API.   I'm trying to do this with Postman but the data being received in Anaplan is all binary.  I'm basically trying to accelerate my uploads via the api using large zipped files.  Do I need to post with a certain header? I'm not sure if this is possible or not?

 

My only headers are:

'Authorization: AnaplanAuthToken bla'

'Content-Type: application/octet-stream'

 

 

 

Thanks,

Tom

Best Answer

  • your URL looks like you are uploading file as single chunk ? I cannot see the entire URL but i can make out with 80% accuracy as you have file/some file ID

     

    If this is TRUE then you cannot compress the file and load . It defeats the purpose . Uploading a file as single chunk means your file size is small(less than 1 mb) therefore you dont need to compress and you PUT it onto the server by streaming it as an octet stream.

     

    you need to Chunk the file into smaller parts and then use gzip application  method 

     

    in other words your URL should look like below 

     

    /workspaces/{workspaceID}/models/{modelID}/files/{fileId}/chunks/{chunkId} -X --upload-file chunk-aa

     

    if you need more info on chunking please refer to 

    https://anaplanbulkapi20.docs.apiary.io/#reference/upload-files under Upload a file in chunks

     

     

     

Answers

  • @tomz 

     

    It is not possible with Anaplan Connect. Below is the excerpts from Anaplan Integration guide. But I am not aware how REST API behaves. I believe it should fall on the same lines

     

    Misbah_0-1580408674020.png

     

    Thanks,

    Misbah

  • @tomz 

     

    First of all Excellent question ! 

     

    You need 

     

    1. compress using gzip format
    2. change your header to Content-Type:application/x-gzip

     

    Hope this helps

     

     

  • I compressed the file to g-zip and tried to send with the header you provided and I get the message: "The server refused this request because the request entity is in a format not supported by the requested resource for the requested method."
  • I hope your request method is POST 

     

    Perhaps share your postman screenshots so that i can debug 

  •  Was using a PUT, which was working with a plain CSV.  Just tested with POST, same bad response.  Maybe its the way i'm uploading the file?

     

    Headers:

    tomz_0-1580429253254.png

     

    Body:

    tomz_1-1580429303215.png

     

     

    Response:

    tomz_2-1580429335754.png

     

    Thanks

  • Oh okay I see. Yes I was doing this as a single chunk. I'll split and try again. Thanks.
  • I just tested it out again and splitted a file into the chunks and converted into gzip and it worked,  Thank you.  I had also tested another method:

    - I gzip'ed a single file, posted the chunk count of 1,

    -Then in my upload url I had to switch it to  {fileid}/chunks/0 ,  instead of the /{fileid}  by itself

    - Then uploaded it with a single gzip compressed file.

    And it worked! 

     

    Thanks!

  • glad it worked
    uploading a single compressed file as 1 chunk will work as well but its not recommended as it puts load on the server and if if ur upload fails you will have to reupload the entire file.
  • I used the following sample python codes from Jesse wilson (https://community.anaplan.com/t5/How-To/Anaplan-API-2-0-Python-Library/ta-p/38139) to upload a csv file using API v2.0, it works perfectly.

     

    To upload a CSV, it reads a CSV file into a buffer and then does stream_upload

     

    upload = anaplan.file_upload(conn, "<file ID>",<chunkSize (1-50)> , "<path to file>")

    with open('/path/sample.csv', "rt") as f:
    buf=f.read()
    f.close()
    print(anaplan.stream_upload(conn, file_id, buf))
    print(anaplan.stream_upload(conn, file_id, "", complete=True))

     

    The header used to upload a CSV is -

     

    put_header = {
    "Authorization":authorization,
    "Content-Type":"application/octet-stream"
    }

     

    I modified the above code to upload a zip file 

     

    buf = zipfile.ZipFile ('/path/sample.zip', "r")
    logging.info(anaplan.stream_upload_zipfile(conn, file_id, buf))
    logging.info(anaplan.stream_upload_zipfile(conn, file_id, "", complete=True))

     

    In stream_upload_zipfile function, I used put_header

     

    put_header = {
    "Authorization":authorization,
    "Content-Type":"application/x-gzip"
    }

    But the zipfile cannot be uploaded, what is my problem here?

     

    Thanks

  • Zip and GZip are different. GZip is a format for compressing a stream of bytes, whereas a Zip file is an archive containing a folder/file structure where the files are compressed using different compression schemes. The API expects gzip-compressed chunk content with Content-Type: application/x-gzip.

  • Thanks Ben for your help.

     

    I was able to get this work by first zipping up the file using python function gzip, then I read the gzip file using "rb" and upload the gzip file  using Content-Type: application/x-gzip in the header

     

    with open(file.gzip, "rb") as f:
    buf=f.read()