How to roll back a production model to recover lost data
Author: Jessie Zhao is a Solution Architect and Certified Master Anaplanner at Anaplan.
Working with a model in ALM has a lot of benefits. These include building new features in a development model anytime without affecting production users, securely managing production lists by identifying structural data that needs to change in the deployment environment without risking an overwrite, and achieving model synchronization through formal revision control by leveraging the tagging of model changes. But there is one challenge — rolling back the production model to recover lost data is more complex in ALM models because of the potential compatibility issue between the development model and production model.
Recently, it happened that I needed to roll back one production model to restore lost data caused by deletion of list members from a production list. There had been three more revision tags added and a few hundred new inputs added to the production model after the deletion was run. My plan was to roll back to the history ID before the deletion happened to confirm the cause and get a copy of the lost data, and roll forward to the latest user change ID so that the few hundreds user edits between the problematic action and the latest user change wouldn’t be impacted, while I could use the copy to add back the lost data.
Steps I took (hint: don't follow these steps!):
- Change production model mode from deployed to standard offline.
- Roll production model back to the history ID before the deletion action with “unlink revision created after this restore point” flagged.
- Confirm prod model at the ID has lost data and make a copy (I know there'll be a copy revision tag added but will also be removed after Step 4).
- Roll model forward to the latest user change history ID with ““unlink revision created after this restore point” flagged.
- Add a new revision tag in Dev model and sync to Prod.
Everything looked good as the latest revision tag showed to be compatible to the DEV model initially, but it returned to an error stating that the target model was not compatible to the source model. It was hard to figure out what — except the copy model action — could cause the incompatibility.
For time’s sake, I gave up reconnecting the production model to the original DEV model and started to create a new DEV model from the latest revision tag existing in the production model. But, when I synced the revision tag after DEV model was created, it would return the same incompatibility error. It’s wasn't until I added a new revision tag in the production model first, and created DEV model from this newly created revision tag, did I successfully sync the revision tag between the new DEV and prod model.
To find out what caused the incompatibility, I recreated the issue with a test production model and a test development model and went through the same steps again.
Attempt #2
With more time to compare changes between models, I found that flagging “unlink revision created after this restore point” (revision tag 1) when rolling back and forward the model removed revision tag 2-4 from the production model, but keeping the structural changes made in revision tag 2-4 in production model, which in fact unsaved structural changes in production model. The available “add revision tag” and “revert to last revision” after rolling forward model also confirmed the idea (screenshot 1). At this point, reverting to last revision tag, meaning reverting to revision tag 1 “run delete action for account 7-9", would restore the compatibility (screenshot 2) but changes made in revision tag 2-4 as well as user inputs after the history ID associated with revision tag 1 would be erased. Given that, saving all changes by creating a new revision tag in this production model and creating a new development model would be the solution at this point.
Screenshot 1:
Screenshot 2:
Knowing what "unlink" does to the revision tag, I became curious about what would happen if I unflag “unlink revision after this restore point” when rolling back and forward model. I tested the idea and found that revision tag 2-4 will stay, but a new revision tag of copying model as well as history ID associated with it would be added. The copy model revision tag makes the prod model incompatible again and it would need to take the same solution as able to create a new development model, or use the revert feature which potentially could cause user input after revision tag 4 erased.
After the above two tests, I found that unflagging “unlink revisions created after this restore point” when rolling back the prod model to keep structural revisions while flagging the “unlink revision created after this restore point” when rolling forward the production model to eliminate copy model revision tag would be the best way to keep compatibility between the original Dev and Prod models.
The correct steps should be:
- Change production model mode from deployed to standard offline.
- Roll production model back to the history ID before the deletion action with “unlink revision created after this restore point” UN-flagged.
- Confirm prod model at the ID has lost data and make a copy.
- Roll model forward to the latest user change history ID with “unlink revision created after this restore point” flagged.
- Add a new revision tag in Dev model and sync to Prod.
Hope my experience of rolling back production model could be helpful for you when you face the same situation!
Questions? Leave a comment!