How to use Talend with CA Certs and Transactional APIs

Options
AnaplanOEG
AnaplanOEG
edited January 2023 in Best Practices

In this article, we will illustrate an example of real-life scenario involving Anaplan's Transactional APIs with Talend and CA Certs. If you haven't already, we recommend to start your CA Certs journey here and then there.

Pre-requisites

  • Talend Open Studio Knowledge
  • Anaplan REST API
  • REST API & JSON
  • Python

Supporting assets provided with this article

  • Disclaimer:. Python scripts and Talend exports are provided "as-is" and are for reference purposes only.  They are not production grade and are not supported by Anaplan Support.  They are meant to serve as examples to build your own integrations.  
  • Python scripts (Python Scripts Talend.zip) to generate Anaplan Authentication Token using Basic & CA Certificate Authentication.
    • getAuthTokenBasic.py
    • getAuthTokenCACert.py
  • Talend Open Studio export (Anaplan_Community_Blog.zip) consisting of following jobs:
    • Blog_getTokenBasic_TransAPI_CSV
    • Blog_getTokenBasicPython_TransAPI_CSV
    • Blog_getTokenCACertPython_TransAPI_CSV
    • Blog_getTokenCACerts_TransAPI_CSV
    • Blog_getTokenBasic_TransAPI_JSON
    • Blog_getTokenBasicPython_TransAPI_JSON
    • Blog_getTokenCACertPython_TransAPI_JSON
    • Blog_getTokenCACerts_TransAPI_JSON

Introduction

Anaplan provides native connector to several iPaaS (Informatica Cloud, Mulesoft, Snaplogic, Dell Boomi, OneCloud) and Business Analytics (ex: Microsoft PowerBI) technologies. In Data Integrations, Anaplan customers, often, have a need to integrate into Anaplan using technologies for which there may not be a native connector available. Anaplan REST API are foundations for all integrations, including above mentioned native connectors.

Most iPaaS (ex: Talend) and Business Analytics vendors provide REST client support. 

“Talend Open Studio for Data Integration is the leading open source solution for data integration.  Its capabilities include:

  • Connectors to packaged applications (ERP, CRM, etc.), databases, mainframes, file, Web Services, and so on to address the growing disparity of source.” (Source: Talend)

Currently, there is no native connectivity to Anaplan in Talend. However, as stated above, Talend does provide connectors to invoke Web Services, including REST. In this article, we will show how powerful integration capabilities of Talend can be combined with Anaplan REST APIs to build integration pipelines.

Objective of this article isn’t to teach audience how to build data integration Jobs using Talend Open Studio, but rather share tips on how different components in Open Studio can be configured to authenticate and integrate with Anaplan platform. Our expectation is that audience has sufficient knowledge on building jobs in Talend Open Studio.

First step of any Anaplan integration is generation of authentication token using Anaplan Authentication Services. Anaplan token is, then, passed in header of any integration or transactional API service request.

Recapping from previous section, there are three utilities you can leverage to create required strings for Anaplan token generation:

  • Python Scripting
  • Custom Java application (vEncrypt_0_4.jar)
  • certAuthGenerator Windows tool (certAuthGenerator.bat)

This article presents two distinct integration job designs in Talend Open Studio that perform following steps:

  • Use CA Certificate information to generate Anaplan Token
  • Pass Anaplan Token to read data from a saved view in Anaplan using Transactional API.

You will learn how to authenticate to Anaplan in Talend Open Studio using two methods:

  • REST API call
  • Python script

REST API call to generate Authentication Token

Objective:  To create and successfully run a Talend Job that performs following tasks:

  • Authenticate to Anaplan and generate an authentication token.  We will refer to this token as AnaplanAuthToken here on out.
  • Assign AnaplanAuthToken to a global variable.
  • Pass AnaplanAuthToken to an Anaplan Transactional API request via a global variable in its header.
  • Write response (JSON) of Anaplan Transactional API request to log console.

Steps:

  1. In Talend Open Studio, create a new Job named Blog_getTokenCACerts_TransAPI_JSON.
  2. Add following components: tREST, tExtractJSONField, tSetGlobalVar, tRESTClient, tMap, & tLogRow
  3. Connect above components as depicted in the figure below.

annejulie_0-1621987665878.png

  1. In the next several steps, we will configure each component.
  2. tREST_1:  This component will be used to make a POST request to Anaplan Authentication Service (https://auth.anaplan.com/token/authenticate).  Response will be a JSON structure with 2 fields:  Body, ERROR_CODE. 

Note: Configuring this step is where you will likely spend most amount of time.  This is because Talend requires values to be double quoted (“).  Since, generated JSON string (encodedData and encodedSignedData) contain strings with double quotes, you will need to use escape character around each string value of JSON structure. You will notice escape character “\” around double quotes in the example below.

Under Basic setting, provide following information:

  • URL: https://auth.anaplan.com/token/authenticate
  • HTTP Method:  POST
  • HTTP Headers
    • “Content-Type”: “application/json”
    • “Authorization”: “CACertificate <public cert base64encoded>”.  You should have public cert base64 encoded string from utilities described in the last section.
  • HTTP Body:  JSON structure generated from the utilities described in the last section

annejulie_1-1621987665888.png

Schema should have two fields:  Body & ERROR_CODE.

annejulie_2-1621987665891.png

  1. tExtractJSONFields_1:  This component will parse JSON response from tREST_1 to extract a key:pair value tokenValue:<value>.  Sample JSON response from Anaplan Authentication Service is shown below.

annejulie_3-1621987665900.png

  • Create two fields in the schema for tExtractJSONFiels_1 component:  token_name & tokenValue.
  • Configure the component specifying following:
    • Read By:  JsonPath
    • JSON Field:  Body
    • Loop Jsonpath query:  “$.tokenInfo”
    • Mapping
      • token_name = “tokenValue”
      • tokenValue = “$.tokenValue”

annejulie_4-1621987665903.png

 

  1. tSetGlobalVar_1:  This component is used to create and set value for global variable AnaplanAuthToken.  Value for this global variable will be the value for field “tokenValue” from previous component (tExtractJSONFields_1).  Once this global variable AnaplanAuthToken is set, it’s value can be accessed & used in other components.

Add a global variable AnaplanAuthToken and set its value to row2.tokenValue.

annejulie_5-1621987665904.png

 

  1. tRESTClient_2:  This component makes a GET request to Anaplan Transactional API to read from a saved view passing page selectors as input parameters in the request.  This request can return data from a saved view in either CSV or JSON format.  In this example, we’ve chosen a JSON response.  Export of Talend project provided with this article will have sample jobs for both CSV and JSON.

Note: This request is not limited to Transactional API. You could have easily made a POST request call to Anaplan Bulk APIs to execute an Action (import/export) or a Process.

 

Pre-requisite: 

  • You will need to be familiar with Anaplan Transactional API.
  • Saved View created in Anaplan with at least one page selector (optional).
  • ModelId, ViewId, ids for any dimensions & list items in page selectors.

Provide following information in the component:

  • URL:  “https://api.anaplan.com/2/0”
  • Relative Path: “/models/<modelId>/views/<viewId>”
  • HTTP Method:  GET
  • Accept Type:  JSON
  • Query Parameters:
    • “format” : “v1”
    • “pages” : “dimensionId:listItemId” (Optional)

annejulie_6-1621987665908.png

 

Under Advanced Settings,

  • Uncheck “convert Response to DOM Document”
  • Specify HTTP Headers as shown below.  This request requires two headers:
    • Content-Type:  Set this to a string value of “application/json”
    • Authorization:  String Value of “AnaplanAuthToken” concatenated with global variable AnaplanAuthToken created in earlier step.  We’ll use the syntax (String)globalMap.get(“AnaplanAuthToken”) to access value of the global variable.

annejulie_7-1621987665911.png

 

  1. tMap: We will use tMap component to select fields from input and map them to output.

Response from this API request will have three fields: statusCode, body, and string. We are only interested in the fields statusCode & string. Field “string” will contain the result from API request, which will be logged to the console. We will use tMap component to select fields from input and map them to output.

annejulie_8-1621987665913.png

 

  1.  tLogRow_2:  Logs data from previous component to the console in Talend Open Studio.

annejulie_9-1621987665916.png

  1. Save & Run the Job. Log console should display successful connection to Anaplan for each request and a JSON response with data from Anaplan Saved View.

annejulie_10-1621987665919.png

 

Python script to generate Authentication Token

Another, simpler & more flexible, option might be to execute a Python script that generates AnaplanAuthToken and passes the token in a global variable to tRESTClient that makes Anaplan API request.

 

Note: Sample Python script provided with this article is intended for reference purpose only. It is not production grade and will not be supported by Anaplan Support.

 

Objective: To create and successfully run a Talend Job that performs following tasks:

  • Authenticate to Anaplan and generate AnaplanAuthToken via Python script.  
  • Execute Python script using tSystem component.
  • Access AnaplanAuthToken from tSystem’s global variable, OUTPUT.
  • Pass AnaplanAuthToken to an Anaplan Transactional API request via a global variable in its header.
  • Write response (JSON) of Anaplan Transactional API request to log console.

Steps:

  1. In Talend Open Studio, create a new Job named Blog_getTokenCACertPython_TransAPI_JSON.
  2. Add following components: tSystem, tRESTClient, tMap, & tLogRow
  3. Connect above components as depicted in the figure below.
  4. annejulie_11-1621987665924.pngIn the next several steps, we will configure each of the components.
  5. tSystem_1:  This component allows you to execute an OS command.  This example depicts a linux shell.  You may need to modify the command to suit your operating system.
    • Provide command to execute your python script that generates AnaplanAuthToken.

annejulie_12-1621987665927.png

  • Select Standard Output to “to global variable”
  1. tRESTClient_1:  This component makes a GET request to Anaplan Transactional API to read from a saved view passing page selectors as input parameters in the request.  This request can return data from a saved view in either CSV or JSON format.  In this example, we’ve chosen a JSON response.  Export of Talend project provided with this article will have sample jobs for both CSV and JSON.

Note:  This request is not limited to Transactional API.  You could have easily made a POST request call to Anaplan Bulk APIs to execute an Action (import/export) or a Process.

 

Pre-requisite: 

  • You will need to be familiar with Anaplan Transactional API.
  • Saved View created in Anaplan with at least one page selector (optional).
  • ModelId, ViewId, ids for any dimensions & list items in page selectors.

Provide following information in the component:

  • URL: “https://api.anaplan.com/2/0”
  • Relative Path: “/models/<modelId>/views/<viewId>”
  • HTTP Method:  GET
  • Accept Type:  JSON
  • Query Parameters:
    • “format”: “v1”
    • “pages”: “dimensionId:listItemId” (Optional)

annejulie_13-1621987665931.png

 

Under Advanced Settings,

  • Uncheck “convert Response to DOM Document”
  • Specify HTTP Headers as shown below.  This request requires two headers:
    • Content-Type:  Set this to a string value of “application/json”
    • Authorization:  String Value of “AnaplanAuthToken” concatenated with global variable tSystem_1_OUTPUT created by tSystem component in earlier step.  We’ll use the syntax (String)globalMap.get(“tSystem_1_OUTPUT”) to access value of the global variable.

annejulie_14-1621987665934.png

8. tMap: We will use tMap component to select fields from input and map them to output.

Response from this API request will have three fields:  statusCode, body, and string. We are only interested in the fields statusCode & string. Field “string” will contain the result from API request, which will be logged to the console. We will use tMap component to select fields from input and map them to output.

annejulie_15-1621987665936.png

 

9. tLogRow_2:  Logs data from previous component to the console in Talend Open Studio.

annejulie_16-1621987665939.png

 

10. Save & Run the Job.  Log console should display successful connection to Anaplan for each request and a JSON response with data from Anaplan Saved View.

 

annejulie_17-1621987665942.png

 

Got feedback on this content? Let us know in the comments below.

 

Contributing authors: Pavan Marpaka and Christophe Keomanivong.

Tagged:

Categories