Testing Requirements

100% helpful (1/1)

The requirements for automated testing are included here. Note that getting ready for automated testing takes time and should be included in the project schedule.

 

Model Sanitization

According to Anaplan security policies, all models placed into the testing environment must be sanitized. Sanitization involves the manipulation of data in a model to values that do not identify any company, persons, precise locations, company plans, or sensitive financial data.

Make a copy of the model, sanitize it and provide login access to the Customer Performance Testing Team. If there is insufficient workspace to do a model copy, L3 Support can assist by providing an isolated workspace to carry out the sanitization.

While it is best to sanitize all data, there may be situations where that is not possible due to time and effort constraints. The chart below ranks the priority for sanitization.

 

#
Data Location Examples Sanitization
Mandatory?
Responsible
Team
1 Company Name(s) Model Name / Workspace Anaplan Yes Performance Team
2 Other Company Name(s) General Lists Accounts, Suppliers, Clients, Distributors Yes Business Partner/ Model Builder
3 Financial Data Data Input Modules Salaries, Revenue, Expenses, Sales Tax % Yes Business Partner/ Model Builder
4 Real Person Name(s) General Lists Employees, Partners Yes Business Partner/ Model Builder
5 Locations General Lists Sales Offices, Retailers No Business Partner/ Model Builder
6 Products General Lists Biscuit Brands, Drink Brands Yes - Brand specific names
No - Generic
Business Partner/ Model Builder
7 Services General Lists Dental, Advertising, Housing No Business Partner/ Model Builder

 

Sanitization Techniques

The Customer Performance Testing team provides additional information and techniques for sanitization on their Confluence page. Sanitization techniques include:

  • Modify numbered lists
  • Temporary hardcoding of values
  • Direct copy and paste
  • Use import and export functionality

 

Test Scripts or Users

Test scripts can also be referred to as discrete users in performance testing. These are step-by-step instructions that can be followed by the simulated user.

The Customer Performance Testing team requires the finer details on test scripts/users, roles and selective access when the model's business processes become clear. A video that demonstrates the user role and its steps would give them the material to start evaluating whether these scripts are fit for performance testing. If a video recording cannot be created, an arranged meeting (and screen share) will be sufficient to talk through the steps. It is important that these details are captured accurately.

The ideal number of scripts is dependent on each unique model, but the team would typically expect to have multiple scripts where each script/user has a specific set of tasks related to their role. Multiple scripts enable greater control over the distribution of the work load, reflecting the load/usage patterns as though real users were using the system. Additionally, the Customer Performance Testing team should know where the user base will be geographically located. 

 

Targets and Customer Requirements

The team needs the customer requirements of model performance: 

  • 90th or 95th Percentile target response times for each transaction
  • Expected load volumes of the model by end users (pacing)
  • Expected scenarios by end users
  • Concurrency level of the user base (typically 15% to 20%)

These requirements are included in the questionnaire that the team has developed to capture the information they need to perform testing. It is available on the Customer Performance Testing Confluence page.

 

How long will performance testing take?

There is no simple answer to this question; every project has many variables that impact performance testing duration. The range is from a week and a half to four weeks.

Version history
Revision #:
2 of 2
Last update:
‎03-23-2017 02:37 PM
Updated by: