Ways to identify duplicates for list import other than RANK or ISFIRSTOCCURRENCE?

Just wondering is there a way to find duplicates where the cell count is more than 50 million and I know in Polaris engine it is possible. Any other way to do it in the classic engine ?

Accepted answers

M.Kierepka

I'd say the problem is you need to frequently import more than 50 million items to a list. Consider rethinking your processes, and if it's always necessary to import so many items — it's worth checking the source of this data, if maybe you can do something that will filter out duplicates there.
For an infrequent one (especially if it's admin-only), if it doesn't cause incorrect results, I'd just not worry about this warning.
If you really need a solution in Anaplan, the only solution I can think of is to add subsets to this list based on something (like first char in/length of the name of an item). That should allow you to split the items into buckets, each having less than 50m items, and then use ISFIRSTOCCURRENCE().

All comments

M.Kierepka

It depends on the situation, format of analyzed value and what type of result you need. I.e. If you just want to know if there are some duplicates in format of some list items, and which items are duplicated, then you can just put something like dummy formula"1" and later [SUM:] by list(s) you expect multiple values for. If in the result you will get 1 for some item, it means no duplicates, if you get more than 1 then you see even how many (and 0 means no occurrence).

tharuns98

what can be done for list creation purposes to avoid the warnings when we import it?

M.Kierepka

Quick Links

Can you help?

Unanswered questions

Importing Ragged Hierarchies
Hello - I have a 2 level ragged hierarchy (yes, it's a chart of accounts), that for reporting purposes is used in 4 different models (yes, it's a P&L). What is the best way set this to allow one to be the "source of truth" and then import between the four models without creating actions that include ignores? It seems like…
Where can I find export file after running export process using machine task in a workflow?
I am testing export actions using workflow but its not generating any file when I am running the workflow, but the same process is generating export file when ran from the Actions→Process section. Is their any issue with the workflow functionality? Also, If I want to generate a file after completion of workflow and pull…
Automating Flat File Uploads to Anaplan Data Orchestrator (ADO) – Best Practices and Methods
Hello Anaplan Community, I am seeking guidance on the most effective ways to automatically upload flat files to Anaplan Data Orchestrator (ADO). Currently, our process involves loading flat files to an Anaplan Datahub using Anaplan APIs and custom Python scripting. With Anaplan Data Orchestrator's capabilities for data…