1. Scaleout User's Guide
  2. Recovering from Failure
  3. Client Connection Failover

Client Connection Failover

When constructing a highly available system, you want to ensure that client connections are automatically rerouted when there is a problem.

  • Client application connections are automatically routed to an active data instance for that database.

  • If an existing client connection to a data instance fails, the client is automatically reconnected to another active data instance in the database.

  • If the data instance to which a client is connected fails, then that client is automatically reconnected to another active data instance in the database.

Note:

See Connecting to a Database.

By default, if a connection fails, then the client automatically attempts to reconnect to another data instance (if possible). Consider the following details on how to prepare for and respond to a connection failure:

  • The TTC_REDIRECT client connection attribute defines how a client is redirected. By default, TTC_REDIRECT is set to 1 for automatic redirection. If set to 0 and the initial connection attempt to the desired data instance fails, then an error is returned and there are no further connection attempts. See TTC_REDIRECT in Oracle TimesTen In-Memory Database Reference.

  • The TTC_NoReconnectOnFailover client connection attribute defines whether TimesTen should reconnect after a failover. The default is 0, which indicates that TimesTen should attempt to reconnect. Setting this to 1 specifies that TimesTen performs standard client failover, but without reconnecting. This is useful where an application does its own connection pooling or attempts to reconnect to the database on its own after failover. See TTC_NoReconnectOnFailover in Oracle TimesTen In-Memory Database Reference.

  • Most connection failures tend to be software failures. Reconnecting to another data instance takes some time during which the connection is not available until the client failover process is completed. Any attempt to use the connection during the client failover processing time generates a native error. See JDBC Support for Automatic Client Failover in Oracle TimesTen In-Memory Database Java Developer's Guide or Using Automatic Client Failover in Your Application in Oracle TimesTen In-Memory Database C Developer's Guide.

  • If you receive a native error in response to an operation within your application, your application should place all recovery actions within a loop with a short delay before each subsequent attempt, where the total number of attempts is limited. If you do not limit the number of attempts, then the application may stop responding if the client failover process does not complete successfully. See Application Action in the Event of Failover in Oracle TimesTen In-Memory Database Java Developer's Guide or Application Action in the Event of Failover in Oracle TimesTen In-Memory Database C Developer's Guide for an example on how to write a retry block within your application for automatic client failover.

Configuring TCP Keep-Alive Parameters

One of the ways that a client connection can fail is with a network failure, such as disconnecting a cable or a host that is hanging or crashing. When the client connection is lost, then client connection failover is initiated. However, when a TCP connection is started, you can configure the TCP keep-alive parameters for the connection to ensure reliable and rapid detection of connection failures.

Note:

You can also detect that there is a problem with the connection by setting the TTC_Timeout attribute, which sets a maximum time limit for a network operation that is completed by using the TimesTen client and server. The TTC_Timeout attribute also determines the maximum number of seconds a TimesTen client application waits for the result from the corresponding TimesTen server process before timing out.

TimesTen Scaleout recommends configuring the TCP keep-alive parameters for determining a failed TCP connection in addition to the TTC_TIMEOUT attribute, as some database operations may unexpectedly take longer than the value set for the TTC_TIMEOUT attribute.

Refer to TTC_Timeout in Oracle TimesTen In-Memory Database Reference for more information about that attribute.

You can control the per connection keep-alive settings with the following parameters:

  • TTC_TCP_KEEPALIVE_TIME_MS: The duration time (in milliseconds) between the last data packet sent and the first probe. The default is 10000 milliseconds.

    Note:

    The Linux client platform converts this value to seconds by truncating the last three digits off of the value of TTC_TCP_KEEPALIVE_TIME_MS. Thus, a setting of 2500 milliseconds becomes 2 seconds, instead of 2.5 seconds.

  • TTC_TCP_KEEPALIVE_INTVL_MS: The time interval (in milliseconds) between subsequent probes. The default is 10000 milliseconds.

  • TTC_TCP_KEEPALIVE_PROBES: The number of unacknowledged probes to send before considering the connection as failed and notifying the client. The default is set to 2 unacknowledged probes.

If you keep the default settings, then TimesTen Scaleout sends the first probe after 10 seconds (the TTC_TCP_KEEPALIVE_TIME_MS setting).

  • If there is a response, then the connection is active and the TTC_TCP_KEEPALIVE_TIME_MS timer is reset.

  • If there is no response, then TimesTen Scaleout sends another probe after this initial probe at 10 second intervals (the TTC_TCP_KEEPALIVE_INTVL_MS setting). If no response is received after 2 successive probes, then this connection is terminated and TimesTen Scaleout redirects the connection to another data instance.

For example, you could modify the TCP keep alive settings in the client/server connectable to have a shorter wait time for the initial probe of 50000 milliseconds, and to check for a connection every 20000 milliseconds for a maximum number of 3 times as follows:

TTC_TCP_KEEPALIVE_TIME_MS=50000
TTC_TCP_KEEPALIVE_INTVL_MS=20000
TTC_TCP_KEEPALIVE_PROBES=3

See TTC_TCP_KEEPALIVE_TIME_MS, TTC_TCP_KEEPALIVE_INTVL_MS, and TTC_TCP_KEEPALIVE_PROBES in Oracle TimesTen In-Memory Database Reference for more information on these connection attributes.