15.2 Tuning Connection Handling Settings

This section contains the following topics:

• Section 15.2.1, "Tuning the Thread Pool and Connection Queue"

• Section 15.2.2, "Tuning HTTP Listener Settings"

• Section 15.2.3, "Tuning Keep-Alive Settings"

15.2.1 Tuning the Thread Pool and Connection Queue

This section contains the following topics:

• Section 15.2.1.1, "Connection Handling Overview"

• Section 15.2.1.2, "Reviewing Connection Queue Metrics for an Instance"

• Section 15.2.1.3, "Reviewing Thread Pool Metrics for an Instance"

• Section 15.2.1.4, "Tuning the Thread Pool and Connection Queue Settings"

15.2.1.1 Connection Handling Overview

When a client sends a request to an HTTP listener in an Oracle Traffic Director instance, the connection is first accepted by an acceptor thread that is associated with the HTTP listener. The acceptor thread puts the connection in a connection queue and then waits for the next client request. A request processing thread from a thread pool takes the connection from the connection queue and processes the request. Note that if the thread pool is disabled, acceptor threads themselves process every request. The connection queue and request-processing threads do not exist.

Figure 15-1 depicts the connection handling process.

Figure 15-1 Connection Handling in Oracle Traffic Director

Description of ''Figure 15-1 Connection Handling in Oracle Traffic Director''

When an Oracle Traffic Director instance starts, it creates the specified number of acceptor threads for each listener and a thread pool that contains a specified, minimum number of request-processing threads.

• If the number of acceptor threads for a listener is not specified, Oracle Traffic Director creates one acceptor thread per CPU on the host.

• If the minimum size of the thread pool is not specified, Oracle Traffic Director creates one request-processing thread per processor on the host on which the instance is running.

As the request load increases, Oracle Traffic Director compares the number of requests in the connection queue with the number of request-processing threads. If the number of requests in the queue is more than the number of request-processing threads, Oracle Traffic Director creates additional threads, up to the specified maximum size for the thread pool.

The default value of the maximum number of request-processing threads will never be more than quarter of the maximum number of file descriptors available to the process. If there are 1, 2 CPUs, then the default is 256 and if there are 3, 4 CPUs, the default is 512. If there are more than 4 CPUs, the default is 1024.

The maximum number of threads is a hard limit for the number of sessions that can run simultaneously. Note that the maximum threads limit applies across all the virtual servers in the instance.

15.2.1.2 Reviewing Connection Queue Metrics for an Instance

If the maximum size of the connection queue is not large enough, client requests might be rejected during peak load periods. You can detect this situation by examining the connection queue section in the perfdump plain-text report, as shown in the following example.

ConnectionQueue:
-----------------------------------------
Current/Peak/Limit Queue Length            0/1853/160032
Total Connections Queued                   11222922
Average Queue Length (1, 5, 15 minutes)    90.35, 89.64, 54.02
Average Queueing Delay                     4.80 milliseconds

• The Current/Peak/Limit Queue Length line indicates the following:

  • Current: The number of connections currently in the queue.

  • Peak: The largest number of connections that have been in the queue simultaneously.

    If the peak queue length is close to the limit, it is an indication that the connection queue might not be large enough for the given load.

  • Limit: The maximum size of the connection queue, which is equal to the size of the thread-pool queue + maximum threads + the size of the keep-alive queue.

• Total Connections Queued is the total number of times a connection has been queued. This number includes newly-accepted connections and connections from the keep-alive system.

• Average Queue Length is the average number of connections in the queue during the most recent 1-minute, 5-minute, and 15-minute intervals.

• Average Queueing Delay is the average amount of time a connection spends in the connection queue. It represents the delay between when a request is accepted by the server and when a request-processing thread begins processing the request. If the average queueing delay is relatively high in proportion to the the average response time, consider increasing the number of threads in the thread pool.

15.2.1.3 Reviewing Thread Pool Metrics for an Instance

You can review the thread-pool information for an instance in the SessionCreationInfo section of the plain-text perfdump report, as shown in the following example.

SessionCreationInfo:
------------------------
Active Sessions 2187
Keep-Alive Sessions 0
Total Sessions Created 4016/4016

• Active Sessions is the number of request-processing threads that are currently servicing requests.

• Keep-Alive Sessions shows the number of HTTP request processing threads serving keep-alive sessions.

• Total Sessions Created

  • The first number is the number of request-processing threads created.

  • The second number is the maximum threads allowed in the thread pool; that is, the sum of the maximum threads configured in the thread-pool and the number of keep alive threads.

If you observe that the total number of request-processing threads created is consistently near the maximum number of threads, consider increasing the thread limit. Otherwise, requests might have to wait longer in the connection queue; and, if the connection queue becomes full, further requests are not accepted. If the average queueing delay (see Section 15.2.1.2) is significantly high in proportion to the average response time, that too is an indication that the thread limit needs to be increased.

15.2.1.4 Tuning the Thread Pool and Connection Queue Settings

You can change the thread pool and connection queue settings by using either the administration console or the CLI.

Changing the Thread Pool and Connection Queue Settings Using the Administration Console

To change the thread-pool settings by using the administration console, do the following:

1. Log in to the administration console, as described in Section 2.3.2, "Accessing the Administration Console."

2. Click the Configurations button that is situated at the upper left corner of the page.

   A list of the available configurations is displayed.

3. Select the configuration that you want to modify.

4. In the navigation pane, expand Advanced Settings and select HTTP.

   The HTTP Settings page is displayed.

5. Go to the Thread Pool section on the page.

6. Specify the parameters that you want to change.

   On-screen help and prompts are provided for all of the parameters.

   When you change the value in a field or tab out of a text field that you changed, the Save button near the upper right corner of the page is enabled.

   At any time, you can discard the changes by clicking the Reset button.

7. After making the required changes, click Save.

   • A message, confirming that the updated configuration was saved, is displayed in the Console Messages pane.

   • In addition, the Deployment Pending message is displayed at the top of the main pane. You can either deploy the updated configuration immediately by clicking Deploy Changes, or you can do so later after making further changes as described in Section 4.3, "Deploying a Configuration."

Changing the Thread Pool and Connection Queue Settings Using the CLI

• To view the current thread-pool settings, run the get-thread-pool-prop command, as shown in the following example:

  tadm> get-thread-pool-prop --config=soa
  enabled=true
  stack-size=262145
  max-threads=20480
  queue-size=2000
  min-threads=20480
  

• To change the thread-pool settings, run the set-thread-pool-prop command.

  For example, to change the connection queue size, run the following command:

  tadm> set-thread-pool-prop --config=soa queue-size=2000
  OTD-70201 Command 'set-thread-pool-prop' ran successfully.
  

  For the updated configuration to take effect, deploy it to the Oracle Traffic Director instances by using the deploy-config command.

For more information about the CLI commands mentioned in this section, see the Oracle Traffic Director Command-Line Reference or run the commands with the --help option.

15.2.2 Tuning HTTP Listener Settings

The following are the key HTTP listener parameters that affect performance:

• Listener address

  The listener address consists of an IP address and a port number. The host on which an Oracle Traffic Director instance is running can have multiple network interfaces and multiple IP addresses.

  A listener that is configured to listen for client requests on all network interfaces on the host machine would have 0.0.0.0 as its IP address. While specifying 0.0.0.0 as the IP address for a listener is convenient, it results in one additional system call for each connection. For better performance, consider specifying an actual IP address for the listener.

• Number of acceptor threads

  Acceptor threads receive client requests and put them in the connection queue. When an Oracle Traffic Director instance starts, it creates the specified number of acceptor threads for each listener. If the number of acceptor threads for a listener is not specified, Oracle Traffic Director creates one acceptor thread per CPU on the host

  Too many idle acceptor threads place an unnecessary burden on the system, while having too few acceptor threads might result in client requests not being accepted. One acceptor thread per CPU, which is the default setting, is an acceptable trade-off in most situations.

  For HTTP 1.0 workloads, which necessitate opening and closing a relatively large number of connections, the default number of acceptor threads—1 per listener—would be suboptimal. Consider increasing the number of acceptor threads.

• Listen queue size

  As explained earlier, acceptor threads receive client requests and put them in the connection queue. If the operating system has not yet scheduled the acceptor thread, the operating system kernel maintains TCP connections on behalf of Oracle Traffic Director process. The kernel can accept connections up to the limit specified by the listen queue size.

  HTTP 1.0-style workloads can have many connections established and terminated. So if clients experience connection timeouts when an Oracle Traffic Director instance is heavily loaded, you can increase the size of the HTTP listener backlog queue by setting the listen queue size to a larger value.

The plain-text perfdump report shows the IP address and the number of acceptor threads for each HTTP listener in the configuration, as shown in the following example:

ListenSocket ls1:
------------------------
Address                   https://0.0.0.0:1904
Acceptor Threads          1
Default Virtual Server    net-soa

You can change the HTTP listener settings by using either the administration console or the CLI, as described in Section 10.3, "Modifying a Listener."

15.2.3 Tuning Keep-Alive Settings

This section contains the following topics:

• Section 15.2.3.1, "About Keep-Alive Connections"

• Section 15.2.3.2, "Reviewing Keep-Alive Connection Settings and Metrics"

• Section 15.2.3.3, "Tuning Keep-Alive Settings"

15.2.3.1 About Keep-Alive Connections

HTTP 1.0 and HTTP 1.1 support sending multiple requests over a single HTTP connection. This capability, which was called keep alive in HTTP 1.0, is called persistent connections in HTTP 1.1 and is enabled by default in Oracle Traffic Director.

Keeping a connection active even after processing the original request helps reduce the time and overhead associated with creating and closing TCP connections for future similar requests. However, keep-alive connections over which few or no requests are received are an unnecessary burden on the system.

To avoid this problem, you can specify the maximum number of waiting keep-alive connections. When a keep-alive request is received, if there are more open connections waiting for requests than the specified maximum number, the oldest connection is closed. In addition, you can specify the period after which inactive keep-alive connections should be closed.

15.2.3.2 Reviewing Keep-Alive Connection Settings and Metrics

The plain-text perfdump report shows the current keep-alive settings and metrics, as shown in the following example:

KeepAliveInfo:
--------------------
KeepAliveCount 26/60000
KeepAliveHits 154574634
KeepAliveFlushes 0
KeepAliveRefusals 0
KeepAliveTimeouts 5921
KeepAliveTimeout 120 seconds

The KeepAliveInfo section of the perdump report shows the following:

• KeepAliveCount:

  • The first number is the number of connections in keep-alive mode.

  • The second number is the maximum number of keep-alive connections allowed.

• KeepAliveHits is the number of times a request was successfully received over a connection that was kept alive.

  If KeepAliveHits is high when compared with KeepAliveFlushes, it indicates that the keep-alive connections are being utilized well.

  If KeepAliveHits is low, it indicates that a large number of keep-alive connections remain idle, unnecessarily consuming system resources. To address this situation, you can do the following:

  • Decrease the maximum number of keep-alive connections so that fewer connections are kept alive.

    Note that the number of connections specified by the maximum connections setting is divided equally among the keep-alive threads. If the maximum connections setting is not equally divisible by the keep-alive threads setting, the server might allow slightly more than the maximum number of keep-alive connections.

  • Decrease the KeepAliveTimeout so that keep-alive connections do not remain idle for long. Note that if the KeepAliveTimeout is very low, the overhead of setting up new TCP connections increases.

• KeepAliveFlushes is the number of times the server closed connections that the client requested to be kept alive.

  To reduce keep-alive flushes, increase the keep-alive maximum connections.

  Caution:

  On UNIX/Linux systems, if the keep-alive maximum connections setting is too high, the server can run out of open file descriptors. Typically, 1024 is the limit for open files on UNIX/Linux; so increasing the keep-alive maximum connections above 500 is not recommended. Alternatively, you can increase the file descriptor limit, as described in Section 15.3, "Tuning the File Descriptor Limit."

• KeepAliveRefusals is the number of times the server could not hand off a connection to a keep-alive thread, possibly because the KeepAliveCount exceeded the keep-alive maximum connections. If this value is high, consider increasing the maximum number of keep-alive connections.

• KeepAliveTimeouts is the number of times idle keep-alive connections were closed because no requests were received over them during the last KeepAliveTimeout period.

• KeepAliveTimeout is the duration, in seconds, after which idle keep-alive connections are closed.

  Another parameter that is configurable and affects performance, but is not shown in the perfdump report is the keep-alive poll interval, which, together with KeepAliveTimeout, controls latency and throughput. Decreasing the poll interval and the timeout period reduces latency on lightly loaded systems. Increasing the values of these settings raises the aggregate throughput on heavily loaded systems. However, if there is too much latency and too few clients, the aggregate throughput suffers, because the server remains idle unnecessarily. Therefore, at a given load, if there is idle CPU time, decrease the poll interval; if there is no idle CPU time, increase the poll interval.

15.2.3.3 Tuning Keep-Alive Settings

You can tune the keep-alive settings by using either the administration console or the CLI.

Changing Keep-Alive Settings Using the Administration Console

To change the keep-alive settings by using the administration console, do the following:

1. Log in to the administration console, as described in Section 2.3.2, "Accessing the Administration Console."

2. Click the Configurations button that is situated at the upper left corner of the page.

   A list of the available configurations is displayed.

3. Select the configuration that you want to modify.

4. In the navigation pane, expand Advanced Settings and select HTTP.

   The HTTP Settings page is displayed.

5. Go to the Keep Alive section on the page.

6. Specify the parameters that you want to change.

   On-screen help and prompts are provided for all of the parameters.

   When you change the value in a field or tab out of a text field that you changed, the Save button near the upper right corner of the page is enabled.

   At any time, you can discard the changes by clicking the Reset button.

7. After making the required changes, click Save.

   • A message, confirming that the updated configuration was saved, is displayed in the Console Messages pane.

   • In addition, the Deployment Pending message is displayed at the top of the main pane. You can either deploy the updated configuration immediately by clicking Deploy Changes, or you can do so later after making further changes as described in Section 4.3, "Deploying a Configuration."

Changing Keep-Alive Settings Using the CLI

• To view the current the keep-alive settings, run the get-keep-alive-prop command, as shown in the following example:

  tadm> get-keep-alive-prop --config=soa
  enabled=true
  threads=20
  max-connections=2000
  poll-interval=0.002
  timeout=31
  

• To change the keep-alive settings, run the set-keep-alive-prop command.

  For example to change the maximum number of keep-alive connections, run the following command:

  tadm> set-keep-alive-prop --config=soa max-connections=2000OTD-70201 Command 'set-keep-alive-prop' ran successfully.
  

  For the updated configuration to take effect, deploy it to the Oracle Traffic Director instances by using the deploy-config command.

For more information about the CLI commands mentioned in this section, see the Oracle Traffic Director Command-Line Reference or run the commands with the --help option.