[Part 2] Enhancing Anaplan Audit Log Data Extraction with a Streamlined Python Solution

AnaplanOEG
AnaplanOEG
edited August 31 in Community Knowledge Base

The Anaplan Audit History project, consisting of several Python modules and a configuration file, works synergistically to streamline the process of fetching, formatting, and loading Anaplan audit log data into a preconfigured Anaplan Model. The project leverages the Anaplan REST API, the Python Pandas library for efficient data conversion from web services JSON format to tabular, and SQLite for advanced data blending and transformation capabilities.

SQLite, a highly versatile and lightweight database engine, offers substantial benefits for transforming and blending various related datasets. Its capacity to execute complex SQL queries and join operations enables users to merge datasets in diverse ways, establishing itself as a powerful tool for data integration and enrichment. Utilizing SQLite's comprehensive set of built-in functions, the project seamlessly combines multiple Anaplan datasets and performs data transformation operations such as filtering, sorting, and reshaping within the SQLite environment. The simplicity of SQLite's file-based storage system facilitates easy deployment and management directly within the Python code, making SQLite an efficient, scalable, and accessible solution for data blending, transformation, and summarization. In context to this project, this SQL code combines various Anaplan metadata, including Users, Workspaces, Models, Actions, Processes, and CloudWorks Integrations via an SQL join with the raw audit data to synthesize audit data in a reportable format. 

In order to supply all this Anaplan content, the following Anaplan REST APIs need to be leveraged:

  • OAuth Service API - to authenticate and refresh the access_token based on the client_id and refresh_token.
  • Authentication API - to authenticate and refresh theaccess_tokenbased on a valid username and password combination or valid certificate in the PEM format.
  • Audit API - to fetch the audit records.
  • Integration API - to fetch metadata about Anaplan objects such as data sources, Processes, and Actions. Additionally, to refresh content to the target Anaplan Audit Reporting Model, the bulk API is used to upload the report-ready audit data, and the transaction API is leveraged for updating the latest timestamp.
  • SCIM API - to fetch Anaplan user metadata.
  • CloudWorks API - to fetch CloudWorks integration metadata.

This project exemplifies how Python can effectively integrate and automate Anaplan operations, using modern OAuth services for Anaplan authorization in tandem with the rich capabilities of the Anaplan REST API. Additionally, the project highlights several Python best practices, such as:

  1. Organizing code into packages and modules: Segmenting the code into multiple modules based on functionality improves maintainability and simplifies the process of locating and resolving future issues.
  2. Enhancing error handling: Effective error management is vital when working with external APIs. Implement try-except blocks to handle exceptions that may occur during API calls or file I/O operations.
  3. Utilizing Python's logging module: Opt for the built-in logging module instead of print statements for debugging, providing better control over log verbosity and streamlining log output management.
  4. Leveraging environment variables or configuration files: Avoid hardcoding sensitive information like API keys or credentials and store this information using environment variables or configuration files instead.
  5. Adding comments to the code: Include annotations in complex or non-obvious code sections to improve comprehensibility for both yourself and others.

Anaplanners can use this project as a basis for building tailored integrations.


The source code and supplementary details, such as requirements, deployment instructions, and helpful videos, are available on GitHub.

Next, learn how to use this data in an Anaplan Model by reading the third installment of this series: [Part 3] Anaplan Audit History Data in an Anaplan Reporting Model.

Author: Quin Eddy, @QuinE - Director of Data Integration, Operational Excellence Group (OEG)

Tagged:

Comments

  • Liampratt
    Liampratt

    Thanks for the information..

  • Dorothymullins
    Dorothymullins

    Very useful for me. Thanks for the sharing information.

  • QuinE
    QuinE

    Exciting news! The Anaplan Audit History Python solution is now cross-platform, supporting Windows, macOS, and Linux. 🎉 To upgrade, first uninstall the old libraries: pip uninstall jwt and pip uninstall pyjwt. Then, reinstall pyjwt with pip install pyjwt for enhanced functionality. #happyauditing #anaplan #python

  • QuinE
    QuinE
    edited August 31

    Exciting News Update! In response to your valuable feedback and requests for enhancements, we're thrilled to unveil two new improvements in this project. Here's what's new:

    1. Expanded Authentication Options: We've listened to your needs and now offer enhanced support for authentication. You can now choose between basic username and password authentication or utilize certificate-based authentication.
    2. It is your choice - Names or IDs: You now have the freedom to input either the workspace and model names or their respective IDs.

Categories