Data Hub Imports without manual uploads, how?

In the Data Hub training it says 'DO NOT' use manual uploads to the data hub. Use ETL Tools. We receive data in flat files (csv) for many of our data sources, and have no 'link' to their systems. So how do I utilise my data hub it if am not supposed to manually upload these files?

Accepted answers

All comments

elena56brown

To utilize your data hub without manually uploading files, you'll need to use ETL (Extract, Transform, Load) tools. These tools automate the process of extracting data from your flat files (CSV), transforming it into the required format, and loading it into the data hub. Here are some steps to get started:

Choose an ETL Tool: Select an ETL tool that supports your data sources and data hub. Some popular options include Apache NiFi, Talend, and Informatica subway bogo footlong

Extract Data: Configure the ETL tool to extract data from your CSV files.

Transform Data: Apply any necessary transformations to the data, such as cleaning, filtering, or aggregating.

Load Data: Load the transformed data into your data hub.

By automating this process, you can ensure consistent and efficient data integration without manual uploads.

DeveloperCYT

Hi, thanks, do these tools all have a cost associated with them?

alexpavel

Indeed, this is about automating the uploads in Anaplan.

In your case, where you have the CSV created from the source system, a costless solution (from the ETL tool point of view) is to use Anaplan Connect which is an utility that upload and import available file.

You can find more info here:

https://help.anaplan.com/anaplan-connect-e3a9f00c-3924-4cfb-aed0-1ec14233821b

In the same Anapedia area, you can alsocheck other ETL tools used in data integration.

Hope it helps

Alex

DeveloperCYT

Hi Alex,

I already have written and used approximately 100 Anaplan Connect scripts, and I would class that as 'manual' uploading, but if that's not classed as manual, then I will continue to use Connect to import the data.
The trouble we have is we are uploading customer orders maybe 20 times a day, as we are FMCG we need this live order processing.
We receive CSV files of the orders, and then they need uploading.
Users cannot run Anaplan Connect scripts.
So currently they upload directly into the main Model, using an Import action.
I wanted them to upload to the HUB, process the orders and then send clean data to the model.
This is not possible unless we allow manual uploads to the HUB.

thanks

Quick Links

Can you help?

Unanswered questions

import monthly data with months in columns
is there a way to import data with months in columns (usual excel table view)? I have to transform the data so there are rows to cover each period in the upload data form, but I am wondering if there is a better way to just import the data as is with the months in the columns. please let me know what you think. thank you,
Cloudworks Big Query Integration Fail
Hello, I am receiving an 'Anaplan Upload Failed' Status Description when testing my integration with a Big Query dataset. The integration imports data from BQ to our Anaplan model. No other details given in the error log. I suspect that Cloudworks is not even picking up the file but am not sure what we did wrong on the set…
Looking for Anaplan End-Users to take a short survey!
We are looking for Anaplan end-users to provide feedback on their experiences with the Excel add-in. Interested individuals will respond to this 5-minute survey to help us understand personal needs and behavior when using the add-in. The feedback provided by survey takers is essential to the roadmap of Anaplan's products.…