Developing and Deploying Siebel Business Applications > Testing the NREC Deployment >

Executing Performance Tests

There are three types of performance test cases that are typically executed: response time, stress, and reliability testing. Each is intended to measure different key performance indicators. Performance tests are typically managed by specialized members of the testing and administration organizations, who have ownership of the architecture and infrastructure.

Figure 38 illustrates the process for performance test execution. The first step involves validating recorded user-type scripts in the system test environment.

Figure 38. Diagram of the Execute Performance Tests Process

Execute each script for a single user to validate the health of the environment. Obtain a low user-load baseline before attempting the target user load. Use a baseline to measure scalability by comparing results between the baseline and target loads.

Users need to be started at a controlled rate to prevent excessive resource utilization due to large numbers of simultaneous logins. This rate depends on the total configured capacity of the deployment. For every 1000 users of configured capacity, add one user every three seconds. For example, if the product is configured for 5000 users, you add five users every three seconds.

Excessive login rate causes the application server tier to consume 100% CPU, and logins begin to fail. Randomize think times during load testing to prevent inaccuracies due to simulated users executing transactions simultaneously. Set randomization ranges based on determining the relative think times of expert and new users when compared to the average think times in the script.