Unit Testing is for finding bugs, not show correctness
Having worked projects with hundreds of bugs, the natural question is how did this happen?
The solution passes User Acceptance Testing means that it was adequately tested.
This could be a paradigm shift for some in thinking about unit testing. Doing a quality job of it adds time to get a solution to Production. However, development is the best and optimal time within the software development life cycle to spend the time, providing most value to a client.
This discussion thread is written with great comradery and respect for our Anaplan Community to express support and gratitude for those that put time into quality unit testing. It benefits all of us and usually goes unrecognized.
Software testing principles are published by the International Software Testing Qualifications Board:
- Testing reduces the probability of undiscovered defects, but if there are no defects found, that is not a validation of correctness.
- Exhaustive testing is impossible so conducted weighing risks and priorities.
- Early testing makes a bug much cheaper to fix.
- 80% of the problems are found in 20% of the modules.
- If the same tests are repeatedly done, eventually, the same set of test cases will no longer identify any new bugs in the system.
- The risk associated with each type of application is different, so using the same method is ineffective.
- Finding and fixing defects does not help if the system is unusable and does not fulfill users’ needs and requirements.
This discussion is addressed to Architects and Builders because a client may never see or validate integration points, understand how scaling will impact a model, recognize how a culmination of formula inefficiencies could impact optimization, have a clear understanding of processes, understand how design decisions will impact maintainability, and the list goes on. Since sufficiently complex models may have an infinite number of possible bugs, it is logically impossible to show the correctness of software through testing.
Those that have heard the phrase that Anaplan is Excel on steroids are sometimes caught off guard by the realization that building and maintaining this platform has a much steeper learning curve than Excel. And we guide them through that journey, reach a good degree of proficiency, and sometimes have complacency in overlooking some fundamental development principles like good unit testing. There are bugs that we may be the only ones able to identify so share the testing responsibility equally with the client.