Peak load test
20 – 30 minutes
Peak load duration:
1 hour
Aim
This test is used for measuring the average response times and the throughput of the model(s), at given numbers of concurrent users.
Scenarios
Simulating the anticipated peak hour of the typical business day.
Simulating the anticipated peak hour of an exceptional day, e.g. majority of users completing their transactions at end of financial period.
Results
Statistical measures for response times are used such as average, median, standard deviation, 90th, 95th, 99th percentile, minimum, and maximum. To assess the success of the model in the peak load scenario, the 90th percentile transaction response times are compared against the target response times. This test will also identify the top slowest transactions.
Stress test
1 – 3 hours
Peak load duration:
5 – 10 minutes
Aim
This test is used for observing other performance characteristics of the model(s), such as how well it scales to a growing number of users.
Scenarios
Simulating loads beyond those of the peak load test, e.g., double or triple the number of concurrent users.
Results
Although we still measure the response times and throughput, the focus of the test is about exploring how a model reacts to an increasing number of users. The results can show whether the model is resilient, i.e., can handle excess number of users, or the results could show the model concurrency "limit," i.e., when the model becomes unreasonably slow.
Endurance test
20 – 30 minutes
Peak load duration:
8 hours or more
Aim
This test is used for observing characteristic changes of model(s) over time. The main aim of this test is to find out if there are any long-term stability issues, e.g., does the model grow over time, and are response times for the same actions affected?
Scenarios
By completing the User Journeys at an accelerated rate, for an extended time, we can observe the trend results and extrapolate. We can begin to understand and build a picture of what a production model may be like in the future, e.g., in 12 months’ time.
Results
When the model experiences load for an extended time through the endurance test, it simulates the near-future state of the model. The endurance test will provide you the size of the model from start to end of the test, in context to the number of actions processed. These findings may also be extrapolated to assess future risks.
Custom test
Variable
Peak load duration:
Variable
Suitability
This is suitable for creating complex scenarios which do not fall into the peak load, stress, or endurance test load profiles.
If a custom test is required, the aim/objective of the test must also be clearly defined.
Scenarios
Simulating an entire day’s anticipated load from beginning to end.
Running imports/exports or processes at specific times of the working day.
Observing performance impacts when a temporary spike of users begin interacting with the model(s).
Please contact model.concurrency@anaplan.com to assess the viability of your custom test.
