Validate imported data

[kunal3591]

Currently building a model which involves import of data at various stages.

Wanted to implement a functionality where i can keep a check on the imported data to be validated after each import is run.

Any suggestion would be great help.

Regards,

Kunal

[LipChean_Soh]

Hi Kunal,

Can you further narrate what you mean by "keep a check on the imported data to be validated after each import is run."?

Thanks,

LipChean

[kunal3591]

Hi Lip Chean,

So there are multiple import actions in the model that i have made.

I need to publish a dashbaord that would prompt if there are any errors or mismatches or if the actions have run successfully.

This is to make sure that the files being uploaded/imported into the model have  identical names and with the dimensions/list items and numbers in the model.

Hope this gives the required details.

Thanks,

Kunal

[LipChean_Soh]

Hi Kunal,

You can refer to the Data Hub best practices link below, particularly the 'Reconcile' segment.

https://community.anaplan.com/t5/Knowledge/Building-a-Data-Hub/ta-p/33593#toc-hId--333796648

Cheers,

LipChean

[zafkamar · Accepted solution]

@kunal3591,

As @LipChean_Soh has pointed to, the best practices here is to utilize the FINDITEM() function to determine where your data loads have resulted in metadata mismatches between your source system and Anaplan.  First off, you should be importing your source data into text-formatted line items in a flat, 2-dimensional module so that they don't immediately kick out (since text will always be imported kickout-free).  Second, create a new list-formatted line item that points to the text-formatted line item.  Third, create a validation boolean-formatted line item that can show where there are invalid line items.

Line Item .                               Format                                  Formula

COST_CENTER                      Text                    

FI Cost Center                          List [Cost Centers]                FINDITEM('Cost Centers', 'COST_CENTER')

Valid CC?                                 Boolean                                 IF ISBLANK('FI Cost Center') THEN FALSE ELSE TRUE

There are more advanced techniques that you can utilize to further enhance validations as well as ways to make this process more efficient, but this is the basic premise. Beyond this, you can then filter your module on invalid instances of the 'Valid CC?' boolean-formatted line item and publish that filtered view onto a validation dashboard.  Hope this helps.

Thanks,

Zaf

[rob_marshall · Accepted solution]

Zaf,

I agree with everything you stated above except for importing into a two dimensional module as metadata and transactional data should be kept separate.  By keeping them separate, you will save space as the metadata should not change over time except if you are doing something similar to employee department transfers.

Thanks,

Rob

[kunal3591]

Thanks a lot Zaf..This is great help!!did try something similar...should work!!

[kunal3591]

Hi Zaf,

I had created a numbered list and got a list of mismatched items for various dimensions by using a similar methodology.I want to further add a functionality for the user to go and rectify the mismatched items/errors on the tool on the go.
For that i published the list on the same dashboard where the mismatched items/errors are being displayed .
The user can search the list number and make changes on the go ,once completed ,on clicking refresh the error that was being displayed doesnt appear anymore.
Is it a good practice or is there another workaround.
Also i wanted to see if a similar error is occuring mutliple times so if i could display it just once and the number of times it has occured.
If the user wants to get into the details,they can click on the mismatched item to show wherever it has occured and furthermore rectify it in the list published below on the same dashboard.
Any leads on this would be highly appreciated.
Thanks,
Kunal