Hi All -
I am working on a small POC to do the following:
- Create a simple list in anaplan
- Import data into the list via file upload.
- Permission on import/upload file will be sent to "Admin Only".
- Import/upload will be done initially with my user account (ie damian.shameer@xcompany.com)
- Create a systematic/batched process within my org to periodically do the following:
- Upload data to Anaplan into file created in #2
- Trigger the import process created in #2 to load the data from the upload file
- Upload and Import will be done via the REST APIs and not Anaplan Connect.
- A special data integration account, ie my_aplan_integ@xcompany.com, will be used for #5(I cannot used my login account for security reasons). The data integration account will be added as a Work Space Admin to the applicable Workspace/Model.
I have completed #1-4 with out any issues.
#5 is also complete. I can upload a file via the REST API and trigger the Import Task created in #2 with my integration account.
How the POC was tested:
*Special integration account used for all REST API calls
1. Perform import with 100K rows via Anaplan UI with my login id. Verify list contains 100K rows
2. Create an Action to truncate the list, ie, "Delete All Data From List"
3. Execute "Delete All Data From List" via the REST API call to clear list--> completed with 200 return code . Verified in Anaplan as well
4. Create a text file with 2 rows (structurally the same as original upload file).
5. Upload file via REST API using the File ID (retrieved by getting list of all Server Files) --> completed with 204 return code
6. Trigger Import task via REST API --> completed with 200 return code but not as expected
Expected result: List will now contain 2 rows
Actual result: Import process picked up the file I uploaded via the User Interface.
13:25:26.916 [main] INFO com.fw. - successRowCount - 166598
13:25:26.916 [main] INFO com.fw.x - successCreateCount - 166598
What could be missing here?
@Ernie_Goff
@Fwolf
Additional Details since Anaplan will not accept my replies (made 3 replies, none showing up):
This is a PUT request hence the response (as per standards) should be 204 and not 200. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/204
I am following this API Request: Upload a single chunk
Example from Documentation. I also tried the option to upload by chunks (Set Chunk count, upload 1 chunk, set upload as complete...Still didn't work).
curl -X PUT \
htt ps://api.anaplan.com/2/0/workspaces/8a8196b15b7dbae6015b8694411d13fe/models/75A40874E6B64FA3AE0743278996850F/files/113000000008 \
-H 'authorization: AnaplanAuthToken {auth_token}' \
-H "Content-Type:application/octet-stream" --upload-file Tests.txt
My API call:
Request: URL: htt ps: //api.anaplan.com/2/0/workspaces/x/models/x/files/113000000039
File to be uploaded:
File uploaded via browser had 160K rows. THis file will be smaller
ID,Timestamp,Email,Description
1,7/1/2019 9:55,test.user2@comap.com,Change Line Item
2,7/1/2019 9:58,test.user2@comap.com,Change Module
19,7/1/2019 11:03,test.user2@comap.com,Change Line Item Write Access Driver
20,7/1/2019 11:03,test.user2@comap.com,Change Line Item Read Access Driver
21,7/1/2019 11:03,test.user2@comap.com,Change Line Item Read Access Driver
22,7/1/2019 11:03,test.user2@comap.com,Change Line Item Write Access Driver
Upload file details (from REST API call)
ServerFileData: id: 113000000039 name: Activity_log_4_160K-20190915.csv code: null chunkCount: 1 origin: null format: txt language: null country: null encoding: ISO-8859-1 separator: , delimiter: " headerRow: 1 firstDataRow: 2 files[2]:
Import Details:
imports[1] id: 112000000038 | name: Activity Log from Activity_log_4_160K-20190915.csv | code: null| importType: HIERARCHY_DATA | importDataSourceId: 113000000039
Java Code:
static void uploadFile(CloseableHttpClient client, String apiURL, String authToken, String fileNameWithPath) {
// My URL: URL: htt ps://api.anap lan.com/2/0/workspaces/x/models/x/files/113000000039
// Sample (Docs, Unload as single chunk): htt ps://api.anapla n.com/2/0/workspaces/8a8196b15b7dbae6015b8694411d13fe/models/75A40874E6B64FA3AE0743278996850F/files/113000000008
System.out.println("URL: " + apiURL);
StopWatch stopWatch = new StopWatch();
stopWatch.start();
try {
HttpPut request = new HttpPut(apiURL);
request.setHeaders(getHeadersForUpload(authToken, false));
File source = new File(fileNameWithPath);
if (source == null || !source.exists() || !source.isFile() || !source.canRead()) {
throw new RuntimeException("Input file must be a valid file. File must be read-able. Check Source file. ");
}
byte[] byte_data = org.apache.commons.io.FileUtils.readFileToByteArray(source);
if (byte_data == null || byte_data.length == 0) {
//TODO determine if this should be allowed
throw new RuntimeException("Source File seems to be empty. No Data to upload. ");
}
System.out.println("Data Size (mb): "+byte_data.length /1024);
// create data stream and set to request
InputStreamEntity entity = new InputStreamEntity(new ByteArrayInputStream(byte_data), byte_data.length);
request.setEntity(entity);
System.out.println("Begin File upload...");
HttpResponse response = client.execute(request);
stopWatch.stop();
System.out.println("Upload request done. Took: " + stopWatch.toString());
int responseCode = response.getStatusLine().getStatusCode();
String respText = response.getEntity() != null ? EntityUtils.toString(response.getEntity()) : "[no response text]";
System.out.println("Response Code: " + responseCode);
if (responseCode != 204) {
System.out.println("Error: " + respText);
throw new RuntimeException("Upload may have failed. API Return code was: " + responseCode + ". Expect Code 204");
} else {
System.out.println("File Uploaded!!");
}
} catch (IOException e) {
//TODO add proper logging and custom exception
e.printStackTrace();
throw new RuntimeException("IOException thrown. File upload failed. Error: " + e.getMessage());
}
}
Response:
URL: http s://api.anaplan.com/2/0/workspaces/x/mode ls/x/files/113000000039
contentTypeValue: application/octet-stream
Data Size: 443
Begin File upload...
Upload request done. Took: 00:00:02.722
Response Code: 204
File Uploaded!!
Results from Import Task:
Expecting only 6 rows to be upload. Import task picked up the file i uploaded via the browser (166,598 rows)
8:12:56.858 [main] INFO com.fw.x.x.anaplan.restapi.client.logging.LogUtils - - - - - - - - - - - - - - - - - - - - - -
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - Activity Log: 166598 (0/166598) rows successful, 0 ignored
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - hierarchyName - Activity Log
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - successRowCount - 166598
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - successCreateCount - 0
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - successUpdateCount - 166598
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - warningsRowCount - 0
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - warningsCreateCount - 0
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - warningsUpdateCount - 0
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - failedCount - 0
08:12:56.859 [main] INFO com.fw.x.anaplan.restapi.client.Task - ignoredCount - 0
08:12:56.860 [main] INFO com.fw.x.anaplan.restapi.client.Task - totalRowCount - 166598
08:12:56.860 [main] INFO com.fw.x.anaplan.restapi.client.Task - totalCreateCount - 0
08:12:56.860 [main] INFO com.fw.x.anaplan.restapi.client.Task - totalUpdateCount - 166598
08:12:56.860 [main] INFO com.fw.x.anaplan.restapi.client.Task - invalidCount - 0
08:12:56.860 [main] INFO com.fw.x.anaplan.restapi.client.Task - updatedCount - 166598
08:12:56.860 [main] INFO com.fw.x.anaplan.restapi.client.Task - renamedCount - 166598
08:12:56.860 [main] INFO com.fw.x.anaplan.restapi.client.Task - createdCount - 0
08:12:56.864 [main] INFO com.fw.x.anaplan.restapi.client.Task - Task ID 407A8B7967A94CFEA00E8491CBE98F35 executed by anaplanfrsysdev Complete. Took: 00:00:01.316
