Data load for duplicate rows

Hi All,

I was trying to load data into a module, but my data file has duplicate rows.

when I load the file, I was expecting the duplicate rows to be ignored or flagged as duplicate record value but it getting added in the member value.

Example:

Dimension Line item(amount)

row1 ABC 10

row2 DEF 20

row3 ABC 30

row4 DEF 40

When I load the above file, my values for ABC becomes 40(10+30) and DEF becomes 60(20+40)

Is this is the way anaplan will load a data file or is there any way I can eliminate the duplicate rows.

Regards,

Usha

Accepted answers

VIGNESH.M

Hi Usha,

Try this,

Load the file into the one Staging module and use first occurrence to find the first valid record and load it from the saved view to the target module.

Regards,

Vignesh M

kavinkumar

@usha.cherukuri,

Simple!!!

Get the data as 'Text' and then convert that to 'Number'. So in this way, only the first occurrence will be loaded and the other will be ignored. I hope this helps!

Regards,
Kavin.

All comments

VIGNESH.M

Hi Usha,

Try this,

Load the file into the one Staging module and use first occurrence to find the first valid record and load it from the saved view to the target module.

Regards,

Vignesh M

kavinkumar

@usha.cherukuri,

Simple!!!

Get the data as 'Text' and then convert that to 'Number'. So in this way, only the first occurrence will be loaded and the other will be ignored. I hope this helps!

Regards,
Kavin.

nathan_rudman

that's a clever one 🙂

usha.cherukuri

Hi Kavin,

sorry but I didn't actually got this, say I have loaded the data in as a text line item and convert that to number format in second line item.

still I may require the firstoccurance filter view to ignore the duplicate line right? as suggested by @vignesh above.

Thanks ,
Usha

usha.cherukuri

Hi Vignesh,

Thanks for you quick response.

but only one Q I have here , If we have 5-6 module for data load.
then i end up in making 5-6 staging tables for each module right?

Thanks
Usha

VIGNESH.M

Hi Usha,

You don't need the FIRST OCCURENCE to applies kavin's solution. when we load data into the text format line item, the system will take the first unique record only and ignore the duplicate records., So You don't need the FIRST OCCURENCE.

To implement my solution, Yes, you need staging modules.

Let say you have 3 data loading modules, then you need 3 staging modules.

Regards,

Vignesh M

usha.cherukuri

Thank you Vignesh..

Quick Links

Can you help?

Unanswered questions

Where can I find export file after running export process using machine task in a workflow?
I am testing export actions using workflow but its not generating any file when I am running the workflow, but the same process is generating export file when ran from the Actions→Process section. Is their any issue with the workflow functionality? Also, If I want to generate a file after completion of workflow and pull…
Automating Flat File Uploads to Anaplan Data Orchestrator (ADO) – Best Practices and Methods
Hello Anaplan Community, I am seeking guidance on the most effective ways to automatically upload flat files to Anaplan Data Orchestrator (ADO). Currently, our process involves loading flat files to an Anaplan Datahub using Anaplan APIs and custom Python scripting. With Anaplan Data Orchestrator's capabilities for data…
How to Design Simulation Planning in Anaplan with Multi-Level User Input and Step-by-Step Logging?
Hi Anaplan Community, I'm working on a simulation planning model where users need the flexibility to input pricing adjustments at any level of the product hierarchy, and I’d love your guidance on best practices. 📌 Scenario Overview: We have a 4-level product hierarchy: Brand Category Sub Category SKU (lowest level) The…