Scalability is an integral characteristic comprising the following:
- How well volume of test cases can be extended without linear (or even geometric) growth of efforts for creation and maintenance.
- How well the same testing coverage can be applied to different environments.
- How well framework of the solution supports creation of integration of tests, by reusing automation of two integrated applications.
- How well framework of the solution supports creation of tests for a different GUI platform (Desktop GUI, Web GUI, Mobile Device(s) GUI).
Failure criteria:
- Test scripts are built to use only certain volume of data, defined at their creation time.
- Adding more data requires code changes.
- Adding more data linearly increases creation effort.
- Maintenance and operation procedures take time longer than reasonable (too hard to find code module, too hard to find data set, too long to review execution results).
