15 About the Load Generator

This chapter describes Load Generator, which is a tool used for debugging and benchmarking Inline Services by simulating decision requests. Load Generator is used both for testing Inline Services, and for performance characterization.

You can access Load Generator by opening RTD_HOME\scripts\loadgen.cmd. For a sample Load Generator script, see the etc directory of the Cross Sell example.

Load Generator has five tabs:

• Security: To run a load generator session, you must enter a valid User Name and Password in this tab.

• Run: Runs a load generator session and provides feedback on the performance through measurement and graphs.

• General: Sets the general settings for Load Generator's operation, including the rate at which data is sent to the server and the location of the client configuration file.

• Variables: Used to create script, message, and access variables.

• Edit Script: Used to set up the script that specifies the Integration Point requests to be sent to the server.

This section contains the following topics:

• Section 15.1, "Using Load Generator for Testing"

• Section 15.2, "Using Load Generator for Performance Characterization"

• Section 15.3, "Running a Load Generator Session"

• Section 15.4, "Viewing Performance Graphs"

• Section 15.5, "About the General Tab"

• Section 15.6, "About Variables"

• Section 15.7, "About Actions"

• Section 15.8, "Load Generator CSV Log File Contents"

• Section 15.9, "XLS File Contents"

15.1 Using Load Generator for Testing

Load Generator is used to generate load on the server to test it for performance and scalability. Intelligently random messages are sent to the Inline Service, allowing the models to learn. The capability of your models can be gauged after running Load Generator for a sufficient period of time.

15.2 Using Load Generator for Performance Characterization

Once an Inline Service is configured, Load Generator is used to evaluate how the service performs under load in order to assess how many servers are needed for specific loads. When you want to stress the server, typically one instance of Load Generator running on one client machine is sufficient, because Load Generator can engage many threads of execution to run multiple scripts concurrently. If additional load is desired and Microsoft Task Manager shows that Load Generator is already consuming the majority of the client's processing power, then several instances of Load Generator can be started on several client computers and pointed to one server. They send messages with some intelligently random generated messages in the context of sessions. The clients measure performance statistics, as well as the server.

15.3 Running a Load Generator Session

To start a session, first create a new script, or load an existing one. In the Security tab, enter a valid User Name and Password. Then, select the Run option from the Run menu, or click the Run icon on the toolbar. You can alter the delay between data samples on the General tab.

15.3.1 Measuring the Server Load

The Run tab displays real-time information about the session running. Table 15-1 describes the options on the Run tab.

Table 15-1 Options on the Load Generator Run Tab

Option Name	Description
New Requests	The number of requests that have been closed since the previous data sample was taken.
New Errors	The number of errors, either client or server side, that have occurred since the previous data sample was recorded.
New Default Responses	The number of errors since the last data sample, that occurred for Advisor Integration Point requests (as opposed to Informant Integration Point requests) and a default response was defined by the Inline Service for the Advisor.
Active Scripts	Number of simulated users currently connected to the server from this load generator.
Peak Response Time	The length of time it took to close the oldest request during the current data sample.
Total Requests	The total number of requests that have been closed.
Total Errors	The total number of errors.
Total Default Responses	The total number of default responses.
Total Finished Scripts	The total number of simulated users.
Average Script Duration	The length in milliseconds of an average script's execution, from start to finish.

15.4 Viewing Performance Graphs

By default, the Requests per Second graph is visible. You can hide and show graphs by selecting View > Graphs. To clear the data in the graphs, select View > Clear Graphs, or click the Clear Graphs icon on the toolbar:

If you stop a script and restart it, all recorded data will be cleared. However, if you pause a session and then start it again, the data will not be cleared. The following graphs are available:

• Average Response Time: A histogram depicting the 40 most recent average response times.

• Errors: A line graph depicting the number of errors that occurred within the most recent 12,000 data samples.

• Peak Response Time: A line graph depicting the peak response time, in milliseconds, that occurred within each of the most recent 12,000 data samples.

• Requests Per Second: A line graph depicting the average number of requests per second that occurred within each of the most recent 12,000 data samples.

• Requests Per Second distribution: A histogram depicting the 40 most recent readings for requests per second.

15.5 About the General Tab

The General tab contains variables about Load Generator's configuration, timing, and which Inline Service is being specified. The General tab has five sections: Load Generator, Details, Think Time, Scripts, and Logging.

This section contains the following topics:

• Section 15.5.1, "Load Generator Section"

• Section 15.5.2, "Details Section"

• Section 15.5.3, "Think Time Section"

• Section 15.5.4, "Scripts Section"

• Section 15.5.5, "Logging Section"

15.5.1 Load Generator Section

The Load Generator section of the General tab contains the following options:

• Client Configuration: Describes which endpoints Load Generator should use to contact the server.

• Graphs Refresh Interval in Seconds: Sets the delay between graph and counter updates. Click Apply for settings to take effect while a script is already running.

15.5.2 Details Section

The Details section of the General tab contains the following options:

• Inline Service: The name of the Inline Service to which this script will send requests.

• Random Number Generator Seed: If your script has any random elements in it, this gives you the ability to reproduce, to some extent, the random behavior. Repeatable randomness is not possible when running more than one concurrent script (see Number of Concurrent Scripts to Run in Section 15.5.4, "Scripts Section").

15.5.3 Think Time Section

The Think Time section of the General tab contains the following options:

• Fixed Global Think Time: The number of seconds that all simulated users will wait between requests.

• Ranged Global Think Time: A variable time that simulated users wait between requests. The think time changes by either a random number, or a sequentially increasing number from a set number range.

• Minimum: A nonzero number of seconds to wait at a minimum.

• Maximum: A nonzero number of seconds to wait at a maximum (must be greater than minimum).

• Access Type Sequential: At each access, increase the think time by one until you reach the maximum, when it will reset to the minimum.

• Access Type Random: At each access, choose a value between minimum and maximum, inclusive of each.

15.5.4 Scripts Section

The Scripts section of the General tab contains the following options:

• Number of Concurrent Scripts to Run: The number of simultaneous users to simulate.

• Maximum Number of Scripts to Run: A positive number in this field causes Load Generator to stop running after that number of sessions have completed. Zero means unlimited.

15.5.5 Logging Section

The Logging section of the General tab contains the following options:

• Enable Logging: When this option is selected, Load Generator statistics data is periodically written to a file.

• Append to Existing File: When this option is selected, and logging is enabled, Load Generator will append new statistics data onto the end of an existing log file, if any, or else it will create a new file.

• Log File: The full path to the log file, a tab-separated file whose contents are described in Section 15.8, "Load Generator CSV Log File Contents."

• Logging Interval in Seconds: The number of seconds to wait after appending values onto the log file before writing the next set of values.

15.6 About Variables

Variables allow a load simulation to draw its input from many different sources. Session variables are generated once per session. Subsequent accesses to a session variable use the same value. Message variables are held constant for a single request. Access variables may vary every time they are read. Variables are used in Message Actions.

This section contains the following topics:

• Section 15.6.1, "Using Variables"

• Section 15.6.2, "Variable Types"

15.6.1 Using Variables

To use a variable in a message (in the Edit Script tab) for a value to a message parameter, select it from the drop-down list in the Variable column. However, to use it as part of an concatenated string in the Value column, surround the variable name with braces (for example, C001-{customerNum}).

Note:

Entering simply {customerNum} in the Value column is equivalent to selecting customerNum in the Variable column.

15.6.2 Variable Types

There are five types of variables:

• Constant Value: A constant value.

• Integer Range: Select an integer from a range. For example:

  Minimum: 0, Maximum: 50000, Access Type: Random
  Minimum: 0, Maximum: 1, Access Type: Sequential
  

• String Array: Select a string from the specified array. For example:

  List: [A, B, C], Access Type: Random
  List: [Male, Female], Access Type: Sequential
  

• Weighted String Array: Select from the specified array a string with some likelihood [0,1]. For example:

  List: [[0.3, Interested], [0.3, Accepted], [0.4, Rejected]]
  List: [[0.999, Interested], [0.001, Accepted]]
  

• Text File: Select a line of text from a file. For example:

  c:/data.txt, Access Type: Sequential
  

  This example shows an absolute reference to a file on the C: drive.

  inbox/data.txt, Access Type: Random
  

  This example shows a relative reference to a file in the inbox directory, under the directory containing the script file.

15.7 About Actions

In order to easily simulate multiple clients supplying realistic loads to the server, messages can be generated from patterns specified in metadata that are interpreted by Load Generator at run time. These patterns of actions are defined in the Edit Script tab.

The patterns specify message sequences, with fixed or random inter-message delays (think times), as well as patterns for generating values for message fields. Message field values can be literal strings, with optional embedded random characters, or they can be randomly selected from a set of values associated with the field.

Sessions are supported, allowing certain fields to remain constant across messages of the session, suitable for representing session keys (for example, a customer ID, call ID, or account number).

The patterns allow some flexibility in the sequencing of messages. For example, in a typical session, certain messages will come before others, or a predetermined number of messages of certain kind need to happen, and so on.

15.7.1 Types of Actions

There are two types of actions: Message and Loop.

Message has the following attributes:

• Integration Point name: The name of the Integration Point that will be sent the message.

• Session Keys and values: The values sent to the Integration Point request. Session keys have to be separated from other message fields because the server uses them for routing.

Loop has one attribute, Number of times to execute. This attribute can be a constant value, or a range value. A range value executes either sequentially, or randomly within the range.

15.8 Load Generator CSV Log File Contents

Table 15-2 describes the fields of the comma-separated value (CSV) file containing Load Generator statistics.

Table 15-2 Load Generator CSV File Fields

Field Name	Description
Date/Time	The time of day at which the current row of counters was appended to the file. Millisecond precision is available to facilitate correlations with messages in the server's log file.
Thread Pool Size	The number of threads engaged or available to run scripts. This is an implementation detail of little to interest to most people.
New Requests	The number of requests that have been closed since the previous data sample was taken.
Total Requests	The total number of requests that have been closed.
New Errors	The number of errors, either client or server side, that have occurred since the previous data sample was recorded.
Total Errors	The total number of errors.
New Default Responses	The number of errors since the last data sample, that occurred for Advisor Integration Point requests (as opposed to Informant Integration Point requests) and a default response was defined by the Inline Service for the Advisor.
Total Default Responses	The total number of default responses.
Active Scripts	Number of simulated users currently connected to the server from this Load Generator.
Total Scripts	The total number of simulated users.
Average Response Time (ms)	The average length of time it took to close the oldest request during the current data sample.
Max Response Time (ms)	The maximum length of time it took to close the oldest request during the current data sample.
Average Script Duration (ms)	The length in milliseconds of an average script's execution, from start to finish.
Snapshot Period (ms)	The number of milliseconds during which the current counter values were accumulated.

15.9 XLS File Contents

This section describes the contents of the Microsoft Excel file, lg_perf.xls, included in the etc directory of the installation for the purpose of rendering the Load Generator counters written to log/loadgen.csv.

At the top, cell A1 contains a comment describing how to link lg_perf.xls to the tab-separated counter file as a datasource, as follows:

• To specify the path to the Load Generator performance log, place cursor the in cell A2 and select "Import External Data" > "Edit Text Import" from the "Data" menu, and navigate to the path specified in your loadgen configuration, typically {$install_directory}\log\loadgen.csv. Use default parsing settings when prompted. Data will then be automatically refreshed every 3 minutes. To change interval and other settings, select from the "Data" menu the selection "Import External Data" > "Data Range Properties".

In row 2 are the headers containing the names of each counter. All of the headers from the CSV file, described above, appear here, with values below them.