Running multiple export actions in parallel
I am using the API the execute multiple export actions and download the resulting output. I notice that if I start the first export with the call to
/workspaces/{workspaceId}/models/{modelId}/exports/{exportId}/tasks
Then the request will return in under a second as the export starts. However, if I call the endpoint again for my second export the request won't return for roughly the amount of time it takes for the original export to be completed. My impression is thus that a single user cannot be running multiple simultaneous exports. Is this accurate, and if so is there any workaround?
Answers
-
Exporting data is a locking action for the model as the data must remain static while the export file is being created - I'm not aware of a workaround to have parallel processing.
0 -
@rob_marshall connected to my question on data hub in another thread. So if a reading action doesn't cause the model lock. So can you comment on this particular observation. This is a write to file I understand but its only performing a read against the model
0 -
Is the export action from a view? Actually, I don't think it matters if it is a view or not on an export action. If so, it is having to create that view as views are only created when called (they get created in memory at model open like modules and lists). It is not the read that is blocking you, it is the creation or spinning up of the view that is getting you because the system locks to make sure the values in the view are stagnant as the line items in the view could be coming from other modules.
So, going back to the previous answer in the other thread, if you are using views (and I am 99% positive you are), you could have concurrency issues kicking off processes from multiple spokes at the same time to get data from a single data hub.
Does that help?
0 -
@rob_marshall Thanks it's very clear now. This was the 1st thread I found when searching for whether a model read would lock a model which contradicted your comment on the other thread. I vaguely recall a post that had a table which showed what actions/activities would cause a model to lock several months ago but I'm unable to find it now.
0 -
Correct, if the action was writing data. Back in the day, an action would lock the model (both if doing an import from a hub), but that has been changed to just writing of data.
0
