The requirements for automated testing are included here. Note that getting ready for automated testing takes time and should be included in the project schedule.
According to Anaplan security policies, all models placed into the testing environment must be sanitized. Sanitization involves the manipulation of data in a model to values that do not identify any company, persons, precise locations, company plans, or sensitive financial data.
Make a copy of the model, sanitize it and provide login access to the Customer Performance Testing Team. If there is insufficient workspace to do a model copy, L3 Support can assist by providing an isolated workspace to carry out the sanitization.
While it is best to sanitize all data, there may be situations where that is not possible due to time and effort constraints. The chart below ranks the priority for sanitization.
|1||Company Name(s)||Model Name / Workspace||Anaplan||Yes||Performance Team|
|2||Other Company Name(s)||General Lists||Accounts, Suppliers, Clients, Distributors||Yes||Business Partner/ Model Builder|
|3||Financial Data||Data Input Modules||Salaries, Revenue, Expenses, Sales Tax %||Yes||Business Partner/ Model Builder|
|4||Real Person Name(s)||General Lists||Employees, Partners||Yes||Business Partner/ Model Builder|
|5||Locations||General Lists||Sales Offices, Retailers||No||Business Partner/ Model Builder|
|6||Products||General Lists||Biscuit Brands, Drink Brands||Yes - Brand specific names
No - Generic
|Business Partner/ Model Builder|
|7||Services||General Lists||Dental, Advertising, Housing||No||Business Partner/ Model Builder|
The Customer Performance Testing team provides additional information and techniques for sanitization on their Confluence page. Sanitization techniques include:
- Modify numbered lists
- Temporary hardcoding of values
- Direct copy and paste
- Use import and export functionality
Test Scripts or Users
Test scripts can also be referred to as discrete users in performance testing. These are step-by-step instructions that can be followed by the simulated user.
The Customer Performance Testing team requires the finer details on test scripts/users, roles and selective access when the model's business processes become clear. A video that demonstrates the user role and its steps would give them the material to start evaluating whether these scripts are fit for performance testing. If a video recording cannot be created, an arranged meeting (and screen share) will be sufficient to talk through the steps. It is important that these details are captured accurately.
The ideal number of scripts is dependent on each unique model, but the team would typically expect to have multiple scripts where each script/user has a specific set of tasks related to their role. Multiple scripts enable greater control over the distribution of the work load, reflecting the load/usage patterns as though real users were using the system. Additionally, the Customer Performance Testing team should know where the user base will be geographically located.
Targets and Customer Requirements
The team needs the customer requirements of model performance:
- 90th or 95th Percentile target response times for each transaction
- Expected load volumes of the model by end users (pacing)
- Expected scenarios by end users
- Concurrency level of the user base (typically 15% to 20%)
These requirements are included in the questionnaire that the team has developed to capture the information they need to perform testing. It is available on the Customer Performance Testing Confluence page.
How long will performance testing take?
There is no simple answer to this question; every project has many variables that impact performance testing duration. The range is from a week and a half to four weeks.