Automating Cloud to Predictive Insights Integration

Options
AnaplanOEG
AnaplanOEG
edited August 2023 in Best Practices

Introduction

The following document describes the steps needed to automate integration between cloud data sources and Predictive Insights using CloudWorks.

In the document, we will use Amazon S3 integration as an example, but other cloud data sources are also supported. For simplicity, account data will be referred to, but contacts and leads can be enriched in the same way.

The overall flow described by this document can be depicted like this:

image.png

For this procedure, we assume that the user has access to Amazon S3, and is assigned the Integration Admin or Restricted Integration Admin role in Anaplan to configure CloudWorks. The user will also need to be assigned a Workspace Admin(1) role to access Predictive Insights.

(1) Predictive Insights role will be available in future.

Preliminary steps

Amazon S3 and CloudWorks

  1. Create an Amazon S3 bucket in AWS, including a folder for source files and another folder for destination files.
  2. Create an Amazon S3 connection in CloudWorks connecting to the new S3 bucket created in step 1.
  3. Prepare a text/csv file containing Accounts from your CRM that will be used to import into Anaplan.
  4. Place a copy of the Accounts file in the source folder in the S3 bucket, and keep a copy locally for the creation of initial import actions.

Anaplan Modules and Lists

  • Create a list in Anaplan for accounts. We will import accounts into this list from the text/csv file prepared earlier. For this example, we have called the list “Accounts Source List for Scoring and Enrichment”.
  • Create an import action to populate the “Accounts Source List for Scoring and Enrichment” List with accounts. Use the file in the same format as you plan to place in the source S3 bucket. For example:
image.png

For this example, we have renamed the import to “IMP Accounts List for Scoring and Enrichment”.

  • Create a Module that will contain account information, and ensure it is dimensionalised by the “Accounts Source List for Scoring and Enrichment” List. Create Line Items for the accounts data you are importing from the file, for example: Account ID, Website, Account Type.
image.png

In the below example, we are leveraging the “Account Type” CRM field to calculate the Is Customer field using a formula:

IF
    COMPARE(Account Type, "Customer - Channel") = 0 OR COMPARE(Account Type, "Customer - Direct") = 0
THEN
    TRUE
ELSE
    FALSE

You can do it differently of course, this is provided as an example.

  • Using the same file as above, create an import action into the module. We have called this import action “IMP Account Data for Scoring and Enrichment”.
image.png

  • Set default file to ‘Admin Only’ or ‘Everyone’.
image.png

  • Map the “Accounts Source List for Scoring and Enrichment” List to the column that contains the List identifier, and map the Line Items to the column headers of the file.
image.png

  • In the Line Items mapping tab, map the individual fields from the file to their corresponding line items in the module. Make sure to ignore unmapped fields.
  • Create a second module that will contain the results of the Predictive Insights scoring action. We have called this module “Account Scoring Module”. This module should also be dimensionalised by the “Accounts Source List for Scoring and Enrichment” list and should contain line items for the scoring and ranking results:
image.png

Predictive Insights Configuration

Create the following Predictive Insights objects. You may already have them in your system.

  1. Create a Predictive Insights dataset.
  2. If you plan to run both enrichment and scoring operations, create a Predictive Insights model using the dataset from the previous step with the following actions:
    1. Create a Score accounts action.
    2. Create an Enrich accounts action.

Configuring CloudWorks Integrations

We will now configure CloudWorks integrations for the actions created in the previous steps. These will be used in a later section to create a CloudWorks Integration Flow.

Bringing updated account information from Amazon S3 to Anaplan

  1. Create an import integration in CloudWorks to populate the “Accounts Source List for Scoring and Enrichment” list. You can use the same file that contains detailed account information for this integration. Use the import action, “IMP Accounts List for Scoring and Enrichment” that you created earlier.
  2. Create an import integration in CloudWorks to populate the module with source account information that you created before. Use the import action “IMP Account Data for Scoring and Enrichment”.

Invoking Predictive Insights flows

  1. Create a Predictive Insights integration in CloudWorks to invoke account scoring.
  2. Create a Predictive Insights integration in CloudWorks to invoke account enrichment.
  3. Run both integrations to verify their correct operation. You can run the integrations from CloudWorks à Integrations and Processes list.
image.png
image.png

Writing updated information from Anaplan to Amazon S3

  1. Create an export action from the “Account Scoring Module” that we previously created. This action will be used by CloudWorks to export the predictive scores to Amazon S3.
  2. Create an export action from the account enrichment data module. This action will be used by CloudWorks to export the account enrichment information to Amazon S3.
  3. Create an export integration in CloudWorks to export the predictive scores to Amazon S3.
  4. Create an export integration in CloudWorks to export the enrichment data to Amazon S3.
  5. Run both integrations to verify their correct operation. Inspect the results in the Amazon S3 bucket. You can run the integrations from CloudWorks à Integrations and Processes list.

Creating an End-to-End Integration Flow

You are now ready to create an end-to-end integration flow using a CloudWorks Integration Flow to run all of the actions created. Integration flows ensure that all steps are executed in the desired order, and can be scheduled to create a periodic scoring and enrichment process. The flow we create will include the following logical steps from the previous section:

  1. Bring updated account data from Amazon S3 to Anaplan
  2. Run Predictive Insights enrichment and scoring of the updated accounts.
  3. Export the results to back to Amazon S3.

Make sure you have the following integrations ready in CloudWorks:

image.png

When creating an integration flow, you need to add each integration step in the correct order and decide if you want the flow to stop upon a failure or on partial success of one of its actions.

image.png

The completed integration flow will look like the below and you can now run or schedule it.

image.png

Optional: Can I run this using code?

Yes, CloudWorks provides a set of APIs that can be used by developers in order to automate and streamline the integration. Among other capabilities, the API allows the invocation of the integration flows. This allows the integration administrators more flexibility with regard to execution of Predictive Insights flows.

The administrator has, therefore, two main options:

  1. Scheduling of the integration flow
  2. Invoking the integration flow from outside of Anaplan

The latter provides more flexibility if you have an upstream system that needs to be integrated in the entire process.

image.png

This is an example of Python code that allows the invocation of an integration flow. It relies on the requests python library. Make sure to replace the relevant placeholders in the code. Anaplan does not take any responsibility or warranty for the code provided below.

image.png
image.png

Tagged:

Categories