15.3 Tuning the Thread Pool and Connection Queue

This section contains the following topics:

Section 15.3.1, "About Threads and Connections"
Section 15.3.2, "Reviewing Thread Pool Metrics for an Instance"
Section 15.3.3, "Reviewing Connection Queue Metrics for an Instance"
Section 15.3.4, "Tuning the Thread Pool and Connection Queue Settings"

15.3.1 About Threads and Connections

When a client sends a request to an HTTP listener in an Oracle Traffic Director instance, the connection is first accepted by an acceptor thread that is associated with the HTTP listener. The acceptor thread puts the connection in a connection queue and then waits for the next client request. A request processing thread from a thread pool takes the connection from the connection queue and processes the request. Note that if the thread pool is disabled, acceptor threads themselves process every request. The connection queue and request-processing threads do not exist.

Figure 15-1 depicts the connection handling process.

Figure 15-1 Connection Handling in Oracle Traffic Director

Description of "Figure 15-1 Connection Handling in Oracle Traffic Director "

When an Oracle Traffic Director instance starts, it creates the specified number of acceptor threads for each listener and a thread pool that contains a specified, minimum number of request-processing threads.

If the number of acceptor threads for a listener is not specified, Oracle Traffic Director creates one acceptor thread per CPU on the host.
If the minimum size of the thread pool is not specified, Oracle Traffic Director creates one request-processing thread per processor on the host on which the instance is running.

As the request load increases, Oracle Traffic Director compares the number of requests in the connection queue with the number of request-processing threads. If the number of requests in the queue is more than the number of request-processing threads, Oracle Traffic Director creates additional threads, up to the specified maximum size for the thread pool.

The default value of the maximum number of request-processing threads will never be more than quarter of the maximum number of file descriptors available to the process. If there are 1, 2 CPUs, then the default is 256 and if there are 3, 4 CPUs, the default is 512. If there are more than 4 CPUs, the default is 1024.

The maximum number of threads is a hard limit for the number of sessions that can run simultaneously. Note that the maximum threads limit applies across all the virtual servers in the instance.

15.3.2 Reviewing Thread Pool Metrics for an Instance

You can review the thread-pool information for an instance in the SessionCreationInfo section of the plain-text perfdump report, as shown in the following example.

SessionCreationInfo:
------------------------
Active Sessions 2187
Keep-Alive Sessions 0
Total Sessions Created 4016/4016

Active Sessions is the number of request-processing threads that are currently servicing requests.
Keep-Alive Sessions is the number of keep-alive threads serving responses.
Total Sessions Created
- The first number is the number of request-processing threads created.
- The second number is the maximum threads allowed in the thread pool; that is, the sum of the maximum threads configured in the thread-pool and the number of keep alive threads.

If you observe that the total number of request-processing threads created is consistently near the maximum number of threads, consider increasing the thread limit. Otherwise, requests might have to wait longer in the connection queue; and, if the connection queue becomes full, further requests are not accepted. If the average queueing delay (see Section 15.3.3) is significantly high in proportion to the average response time, that too is an indication that the thread limit needs to be increased.

15.3.3 Reviewing Connection Queue Metrics for an Instance

If the maximum size of the connection queue is not large enough, client requests might be rejected during peak load periods. You can detect this situation by examining the connection queue section in the perfdump plain-text report, as shown in the following example.

ConnectionQueue:
-----------------------------------------
Current/Peak/Limit Queue Length            0/1853/160032
Total Connections Queued                   11222922
Average Queue Length (1, 5, 15 minutes)    90.35, 89.64, 54.02
Average Queueing Delay                     4.80 milliseconds

The Current/Peak/Limit Queue Length line indicates the following:
- Current: The number of connections currently in the queue.
- Peak: The largest number of connections that have been in the queue simultaneously.
  
  If the peak queue length is close to the limit, it is an indication that the connection queue might not be large enough for the given load.
- Limit: The maximum size of the connection queue, which is equal to the size of the thread-pool queue + maximum threads + the size of the keep-alive queue.
Total Connections Queued is the total number of times a connection has been queued. This number includes newly-accepted connections and connections from the keep-alive system.
Average Queue Length is the average number of connections in the queue during the most recent 1-minute, 5-minute, and 15-minute intervals.
Average Queueing Delay is the average amount of time a connection spends in the connection queue. It represents the delay between when a request is accepted by the server and when a request-processing thread begins processing the request. If the average queueing delay is relatively high in proportion to the the average response time, consider increasing the number of threads in the thread pool.

15.3.4 Tuning the Thread Pool and Connection Queue Settings

You can change the thread pool and connection queue settings by using either the administration console or the CLI.

Changing the Thread Pool and Connection Queue Settings Using the Administration Console

To change the thread-pool settings by using the administration console, do the following:

Log in to the administration console, as described in Section 2.3.2, "Accessing the Administration Console."
Click the Configurations button that is situated at the upper left corner of the page.

A list of the available configurations is displayed.
Select the configuration that you want to modify.
In the navigation pane, expand Advanced Settings and select HTTP.

The HTTP Settings page is displayed.
Go to the Thread Pool section on the page.
Specify the parameters that you want to change.

On-screen help and prompts are provided for all of the parameters.

When you change the value in a field or tab out of a text field that you changed, the Save button near the upper right corner of the page is enabled.

At any time, you can discard the changes by clicking the Reset button.
After making the required changes, click Save.
- A message, confirming that the updated configuration was saved, is displayed in the Console Messages pane.
- In addition, the Deployment Pending message is displayed at the top of the main pane. You can either deploy the updated configuration immediately by clicking Deploy Changes, or you can do so later after making further changes as described in Section 4.3, "Deploying a Configuration."

Changing the Thread Pool and Connection Queue Settings Using the CLI

To view the current thread-pool settings, run the get-thread-pool-prop command, as shown in the following example:

tadm> get-thread-pool-prop --config=soa
enabled=true
stack-size=262145
max-threads=20480
queue-size=2000
min-threads=20480

To change the thread-pool settings, run the set-thread-pool-prop command.

For example, to change the connection queue size, run the following command:
```
tadm> set-thread-pool-prop --config=soa queue-size=2000OTD-70201 Command 'set-thread-pool-prop' ran successfully.
```
For the updated configuration to take effect, deploy it to the Oracle Traffic Director instances by using the deploy-config command.

For more information about the CLI commands mentioned in this section, see the Oracle Traffic Director Command-Line Reference or run the commands with the --help option.