Slow Data Uploads Issue to Anaplan Storage Server
Hi Everyone
I hope this post finds you well. I wanted to share an issue I've been
facing recently and seek your insights and experiences on how to tackle
it.
I've been encountering slow data uploads to the Anaplan storage server.
It seems like the data is taking much longer to upload than usual,
affecting our workflows and overall productivity. As a team, we heavily
rely on timely data updates, and this slowdown is becoming a significant
obstacle to achieving our goals.
Observations:
- The issue is not related to our internet connection, as we've tested and verified our network speeds to be sufficient for data uploads.
- It's not isolated to a specific model or workspace; the slowdown occurs across various models.
- We haven't made any significant changes to our data upload processes or Anaplan configurations recently.
Please share your experiences, insights, or any potential solutions you might have. We're looking forward to hearing from you and working together to overcome this challenge.
Thank you!
Answers
-
We have had experienced issues with progressive slowdown of models in various forms. Key question would be to identify what is causing the performance degradation.
- We had functionality where a user could add a list item record real-time. It was working fine in the hundreds but the moment the list got into the thousands the model started experiencing exponential performance degradation each time a new list item was added. The root cause of the issue was calculations that were being performed in a multi dimensioned module that had time period. Each time a list item was added it would perform the calculations. Problem was it wasn't only calculating for the new list item added but recalculated across the full list because summaries against the module and other stuff. E.g. Existing list had 800 list items. Upon adding 1 extra list item it was not just calculating values for the 1 list item it was recalculating for the 800 list items as well. We fixed it by not having the module to calculate at time of list insertion. It was feasible for us because the calculation was not really required at time of insertion but could be calculated at a later stage and we isolated the calculation to be performed specific to the one list item being added. Lessons learnt for me was that a model that was "extremely" connected could cause performance degradation. I suggest you first identify what is causing the performance to degrade. In our case, we performed trial and error by disconnecting one component at a time to measure how much improvement we would get. To assist in expediting identification, sequencing from largest modules to smallest modules for disconnection is a good approach
- With batch jobs, we found that some overnight batch jobs were taking longer to execute. What I learnt was that if your module is not used for X period of time (it's between 30 minutes and an hour from recollection) Anaplan will move your model from memory to disk. When a user or batch job tries to access the model after it has been saved to disk you would find it takes longer than usual for the model to load. In general, when you execute your overnight jobs, your model has been inactive for a few hours so it would have been loaded to disk already. How long it would take for your model to load back to memory from disk depends on the complexity of your model. Worst one I've seen takes 10 minutes to load. To minimise this risk, we created a dummy Anaplan process that runs every 30 minutes so it's always in memory
Hope this helps
0 -
What architecture are you using to import data into Anaplan? (Cloudworks, Anaplan Connect, other? )
Is this slowing down related to when the import actions are triggered in Anaplan (long processing time) or is it related to the upload file step itself?
In order to try to pinpoint the issue related to the actions, you could verify the latest launch time of every import action and verify what action takes more time.
Hope it helps
Alex
0
