Integration is one of the key implementations for most enterprise solutions. It involves connecting different applications and transforming a huge amount of data on a daily or ad-hoc basis.
Different iPass tools are available in today's market to make the integration ease of use without many codings. It provides a good user interface to configure and run the integrations. Most of the application connectors are readily available and easily configurable. Another possible solution would be to use look at the Open Source Integration Frameworks.
Before going through technical concepts. One should be familiar in below technologies to have better understanding .
What is Integration Framework?
Integration framework helps to integrate applications in a standardised way using several integration patterns, where It reduces efforts and cost. Every developer will easily understand what you did (if he knows the used framework).
Spring Integration, Apache Camel & Mule ESB are widely used Integration Frameworks in Enterprise Application Integration. Each Framework has its own pros and cons, depending on the use case. In this article, we will focus on Apache Camel.
What is Apache Camel?
Camel is an Open-Source Integration Framework based on Enterprise Integration Patterns. Camel empowers to define routing and mediation rules in a variety of domain-specific languages, including a Java-based Fluent API, Spring, or Blueprint XML Configuration file. It makes integration easier by connecting different endpoints and transforming the data from one endpoint to another endpoint by providing support for multiple protocols and data types.
In camel we connect over a wide range of applications through available connectors and even perform HTTP calls. Few of the major connectors such as SalesForce, WorkDay, AWS. Currently, Camel provides over 300+ connectors. Use this link to learn more about the available connectors
Below is the Camel architecture which was taken from the Apache Camel website. To learn more, click this link.
When Can Apache Camel Be a Good Fit?
As mentioned previously, below are the few points we have to consider when choosing Apache Camel as your integration approach.
If your integration does not have multiple connections to send/receive data.
It does not involve huge data or much complexity in the transformation layer.
If you do not require sophisticated monitoring and analysis of the data.
Finally, Camel is free and open source. It has great community support which is necessary for any open source project.
As we discussed above on how can we integrate Anaplan with different applications using Apache Camel? Below is the Example I have taken where you can integrate Salesforce and Anaplan by Syncing Accounts from SFDC to Anaplan.
We are using Anaplan REST API 2.0 to connect Anaplan and perform upload and Process operations. Camel provides an HTTP connector to call and consume REST SPI services.
To connect to Salesforce, we can do it two ways. one way where we can directly use Salesforce Rest API (or) Salesforce Connector provided by Camel. In the Current Article we are going with Salesforce Rest API approach .
Before going to code explanation lets Setup Required Data in Salesforce and Respective Actions in Anaplan.
Below is the sample Account data created in salesforce where Account id will be the Unique id.
Salesforce Provides Salesforce Object Query Language (SOQL) where we can search the org database, which is like Structured Query Language (SQL) but is designed specifically for Salesforce data. We will use using SOQL to retrieve Account data .
In Anaplan, Let's Create two Import Actions and Add it to Process.
One Action Is to import Account Id to List
Another Action is to import Remaining data fields into Module.
You can generate the project with the required configuration and dependencies and download it as a zip file at start.spring.io. Below is the snapshot:
We can use Eclipse (Or) Spring Tool Suite as IDE for the Project. Export the file as a Maven Project.
In Camel, a 'route' is a sequence of steps, executed in order, that consume and process a message.
It is the best practice to provide credentials in the property file and use it across the application wherever required. Replace workspace Id, Model Id, File Id, Process Id, Export Id and Import Id in the respective places.
Below are the steps that have been executed in the route anaplansync.
1) Fetch Token from Salesforce
2) Run SOQL Query and fetch the Account Results
3) Transform the Json Payload to CSV Format
4) Upload File CSV file into Anaplan.
5) Run Process which will import data into list and modules.
A complete sample project has been attached to this article for reference.
Here we have used Salesforce as an external application. In a similar way, we can have different available Camel connectors in the place of the salesforce and transform the data into CSV while uploading into Anaplan. We can use the above implementation as an alternative approach for an integration solution, instead of using an integration tool. We can even make more customised based on the required, and we'll have an exception login scenario too.