Peak load test

peak_load_graph.pngUser ramp-up period:

20 – 30 minutes

Peak load duration:

1 hour

Aim

This test is used for measuring the average response times and the throughput of the model(s), at given numbers of concurrent users.

Scenarios

Simulating the anticipated peak hour of the typical business day.

Simulating the anticipated peak hour of an exceptional day, e.g. majority of users completing their transactions at end of financial period.

Results

Statistical measures for response times are used such as average, median, standard deviation, 90th, 95th, 99th percentile, minimum, and maximum. To assess the success of the model in the peak load scenario, the 90th percentile transaction response times are compared against the target response times. This test will also identify the top slowest transactions.

 

Stress test

stress_graph.pngUser Ramp-up Period

1 – 3 hours

Peak load duration:

5 – 10 minutes

Aim

This test is used for observing other performance characteristics of the model(s), such as how well it scales to a growing number of users.

Scenarios

Simulating loads beyond those of the peak load test, e.g., double or triple the number of concurrent users.

Results

Although we still measure the response times and throughput, the focus of the test is about exploring how a model reacts to an increasing number of users. The results can show whether the model is resilient, i.e., can handle excess number of users, or the results could show the model concurrency "limit," i.e., when the model becomes unreasonably slow.

 

Endurance test

endurance_graph.pngUser ramp-up period:

20 – 30 minutes

Peak load duration:

8 hours or more

Aim

This test is used for observing characteristic changes of model(s) over time. The main aim of this test is to find out if there are any long-term stability issues, e.g., does the model grow over time, and are response times for the same actions affected?

Scenarios

By completing the User Journeys at an accelerated rate, for an extended time, we can observe the trend results and extrapolate. We can begin to understand and build a picture of what a production model may be like in the future, e.g., in 12 months’ time.

Results

When the model experiences load for an extended time through the endurance test, it simulates the near-future state of the model. The endurance test will provide you the size of the model from start to end of the test, in context to the number of actions processed. These findings may also be extrapolated to assess future risks.

Custom test

 

custom_graph.pngUser ramp-up period:

Variable

Peak load duration:

Variable

Suitability

This is suitable for creating complex scenarios which do not fall into the peak load, stress, or endurance test load profiles.

If a custom test is required, the aim/objective of the test must also be clearly defined.

Scenarios

Simulating an entire day’s anticipated load from beginning to end.

Running imports/exports or processes at specific times of the working day.

Observing performance impacts when a temporary spike of users begin interacting with the model(s).

Please contact model.concurrency@anaplan.com to assess the viability of your custom test.

Latest Articles
Model Concurrency Testing Reporting
Model Concurrency Testing Knowledge Base
10-09-2018
Model Sanitization Guide
Model Concurrency Testing Knowledge Base
03-05-2018
Model Concurrency Test Types
Model Concurrency Testing Knowledge Base
03-01-2018
Model Concurrency Testing FAQ
Model Concurrency Testing Knowledge Base
02-05-2018
Model Concurrency Testing Process
Model Concurrency Testing Knowledge Base
01-23-2018
Labels (1)
0 Kudos