Highlighted
Contributor

Data load for duplicate rows

Hi All,

 

I was trying to load data into a module, but my data file has duplicate rows.

when I load the file, I was expecting the duplicate rows to be ignored or flagged as duplicate record value but it getting added in the member value.

 

Example:

         Dimension   Line item(amount)

row1 ABC             10

row2 DEF              20

row3 ABC              30

row4 DEF               40

 

When I load the above file, my values for ABC becomes 40(10+30) and DEF becomes 60(20+40)

 

Is this is the way anaplan will load a data file or is there any way I can eliminate the duplicate rows.

 

Regards,

Usha

7 REPLIES 7
Highlighted
Master Anaplanner/Community Boss

Re: Data load for duplicate rows

Hi Usha,

 

Try this,

 

Load the file into the one Staging module and use first occurrence to find the first valid record and load it from the saved view to the target module.

1.png

 

Regards,

Vignesh M

Highlighted
Contributor

Re: Data load for duplicate rows

Hi Vignesh,

Thanks for you quick response.

but only one Q I have here , If we have 5-6 module for data load.
then i end up in making 5-6 staging tables for each module right?


Thanks
Usha
Highlighted
Master Anaplanner/Community Boss

Re: Data load for duplicate rows

@usha.cherukuri,

Simple!!!

Get the data as 'Text' and then convert that to 'Number'. So in this way, only the first occurrence will be loaded and the other will be ignored. I hope this helps!

Regards,
Kavin.

Regards,
Kavin.
Community Boss

Re: Data load for duplicate rows

that's a clever one 🙂
Nathan Rudman
Anaplan Model Builder
Highlighted
Contributor

Re: Data load for duplicate rows

Hi Kavin,

sorry but I didn't actually got this, say I have loaded the data in as a text line item and convert that to number format in second line item.

still I may require the firstoccurance filter view to ignore the duplicate line right? as suggested by @vignesh above.

Thanks ,
Usha
Highlighted
Master Anaplanner/Community Boss

Re: Data load for duplicate rows

Hi Usha,

 

You don't need the FIRST OCCURENCE to applies kavin's solution. when we load data into the text format line item, the system will take the first unique record only and ignore the duplicate records., So You don't need the FIRST OCCURENCE.

 

To implement my solution, Yes, you need staging modules.

Let say you have 3 data loading modules, then you need 3 staging modules.

 

 

Regards,

Vignesh M

 

 

 

Highlighted
Contributor

Re: Data load for duplicate rows

Thank you Vignesh..