Anaplan Connect 2.0 - Approaches to Anaplan integration using On-Prem integration platforms

AnaplanOEG

This article is the last one on our series about Anaplan Connect 2.0. 

To refresh your memory on Part 1 & 2, go here and there.

 

There are several On-Prem integration platforms that are popular amongst Anaplan customers. These include Informatica Power Center, IBM InfoSphere DataStage, and Talend Studio. However, none of these platforms have a native Anaplan Connector available. What are our options to integrate with Anaplan using these platforms?

In the community article titled, “How to use Talend with CA Certs and Transactional APIs”, we learned how to integrate with Anaplan platform using Anaplan’s Transactional APIs and Python Scripts. 

You can, certainly, make Anaplan’s REST API requests using REST connectors in these platforms to authenticate and automate integrations. As we presented in the article above, Anaplan authentication & integration logic can be encapsulated in a script (ex: python). This script can be executed from the integrations. 

Although, both approaches provide several benefits such as flexibility, modularity and re-usability, there are some minor drawbacks:

• Following functionality needs to be implemented explicitly
  • Authentication via CA.
  • Refresh Anaplan Token every 30 min for long running imports & exports.
  • Ability to re-try an API call in case of a failure.
  • Automatic chunking while uploading data to Anaplan platform.
  • Capturing rejected records from import actions and processes.

There is an alternate solution that simplifies integration solution even further yet provides above functionality with no additional coding. This solution involves Anaplan Connect. Anaplan Connect is a Java based command line utility that facilitates automation of Anaplan integrations using either batch (windows) or shell (Linux) scripts.  We will not dwell into basics of Anaplan Connect in this article.

Please refer to following resources on Anaplan Connect:

• Data Integration – Part 1
• Data Integration – Part 2
• Introduction to Anaplan Connect
• Anaplan Connect – Data from a Flat File
• Anaplan Connect – Data from a JDBC Database
• Anaplan Connect - Guidelines
• How to leverage sample codes to generate authentication strings from CA Certs?

Anaplan Connect and On-Prem Integration Platforms

Most On-Prem integration platforms provide the ability to execute operating system commands in integration flows. Output from any operating system command (ex: ls, sh, etc…) execution can be written to either a log or an output file. 

In order to run Anaplan Connect scripts from these On-Prem platforms, following pre-requisites must be satisfied:

• Install latest Anaplan Connect
• Whitelist api.anaplan.com & auth.anaplan.com
• Install Java 8

For this article, we will use Talend Studio to illustrate how Anaplan Connect scripts can be executed directly from integration platforms to take advantage of built-in authentication & integration functionality.

Anaplan Connect & Talend

In this example, we will execute an Anaplan Connect script from a Talend Job.  This script will execute an Anaplan Process that will import data from a flat file (Accounts.csv) into a model (List & Module). Output from Anaplan Connect script execution is written to log console in Talend. Components used in this job are:

• tSystem:  Execute Anaplan Connect Script and write stdout to a global variable
• tFixedFlowInput: Create an internal variable to hold output from tSystem stdout.
• tLogRow:  Write contents of internal variable from tFixedFlowInput to log console.

Following instructions outline detailed steps:

• Create a job in Talend (ex: Blog_runACScript_ProcessExample) by connecting following components:  tSystem, tFixedFlowInput, and tLogRow.

• Label components as shown above (Optional).
• runACScript:  Provide absolute path to Anaplan Connect script for “Command” and set Standard Output to “to global variable”.

• As a best practice, Anaplan Connect scripts should not contain values hard-coded into the scripts. For example, any sensitive information (username/passwords) or values that could be dynamic (ex: Model Id, ProcessName) should be handled via environment variables. Using environment variables in your script will provide following added benefit:
  • Sensitive information is not hard coded into your script but passed during run time by integration platform.
  • A single “generic” script can be used to execute multiple Anaplan actions/processes. For example, if you have a multiple processes that need to be automated via Anaplan Connect script, it is simpler and manageable to maintain a single Anaplan Connect script than multiple scripts with hard coded values.
  • Provides flexible & configurable design approach.
• In this example, I have an Anaplan Connect script using environment variables instead of hardcoded values for Anaplan User, Anaplan Password, workspaceId, ModelId, ProcessName, FileName, FilePath, and ErrorDump.

• We will pass values to these environment variables in Anaplan Connect script from tSystem component in Talend. tSystem provides us the facility to create and assign values to environment variables which will be passed to the script during execution. Figure below shows environment variables created in tSystem component. Notice naming of environment variables is identical to those that are defined in Anaplan Connect script.

• tFixedFlowInput_1: Create a variable called “ACOutput” and set its value to (String)globalMap.get(“tSystem_1_OUTPUT”).  This expression will write stdout to internal variable ACOutput from global variable OUTPUT.

• tLogRow_1:  Nothing to configure. Take defaults.
• Save and run the job.
• Contents of log console should look very familiar if you have worked with Anaplan Connect scripts in the past.  You will see output from Anaplan Connect script execution. Of course, if you choose to, you may re-design your job to write the log to an output file.

Summary

Running Anaplan Connect script from an integration platform provides following benefits:

• Shorter development time.
  • Developing Anaplan Connect scripts (using provided examples) is much faster than “re-inventing” authentication & integration logic in another scripting language (ex: python/java).
  • Using environment variables, a single Anaplan Connect script can support multiple integrations.
• Anaplan Connect is supported by Anaplan. Custom built Python/Java application using Anaplan API may not be supported by Anaplan. Anaplan will support REST APIs but not the actual custom application.
• No need to worry about granual APIs, IDs for different objects (imports, processes, tasks, files, etc…).
• No need to explicitly implement token refresh, retry, chunking, or dump files. These are automatically handled by Anaplan Connect.

If your integration use case involves an On-Prem integration platform for which there is no native Anaplan Connector, we hope you will give Anaplan Connect a strong consideration before embarking on re-engineering the same logic using Anaplan APIs.

 

Got feedback on this content? Let us know in the comments below.

Contributing authors: Pavan Marpaka and Christophe Keomanivong.