Testing Pilot and Prototype Deployments

The purpose of testing pilot and prototype deployments is to determine as best as possible under test conditions whether the deployment satisfies the system requirements and also meets the business goals.

Ideally, functional tests should model scenarios based on all identified use cases and a set of metrics should be developed to measure compliance. Functional testing can also involve a limited deployment to a select group of beta users to determine if business requirements are being satisfied.

Stress tests measure performance under peak loads. These tests typically use a series of simulated environments and load generators to measure throughput of data and performance. System requirements for the deployment are typically the basis for designing and passing stress tests.

Functional and stress tests are particularly important for large deployments where system requirements might not be well-defined, there is no previous implementation on which to base estimates, and the deployment requires a significant amount of new development.

Testing can indicate problems with the deployment design specification and might involve several design, build, and test iterations before you can roll out the deployment to a production environment. When testing prototype deployments, you might discover problems with the deployment design, in which case you can iterate back to earlier phases in the solution life cycle to address the problems.

Make sure you have thoroughly tested your deployment design before proceeding to a pilot deployment. A pilot deployment indicates you have already verified the deployment design with earlier series of tests. Problems you uncover during the testing of a pilot deployment must generally be addressed within the parameters of the deployment design.

Because testing never completely simulates a production environment, and also because the nature of a deployed solution can evolve and change, you should continue to monitor deployed systems to identify any areas that require tuning, maintenance, or service.