About Fast Connection Failover

9.2.1 Overview of Fast Connection Failover

The Fast Connection Failover (FCF) feature is a Fast Application Notification (FAN) client implemented through the connection pool. The feature requires the use of an Oracle JDBC driver and an Oracle RAC database or an Oracle Restart on a single instance database.

Note:

This section only describes the steps that an application must perform when using FCF with Oracle RAC.

9.2.2 What is Fast Connection Failover?

After Fast Connection Failover is enabled, the mechanism is automatic; no application intervention is needed. This section discusses how a connection failover is presented to an application and what steps the application takes to recover, in the following sections:

9.2.2.1 What the Application Sees

By the time an Oracle RAC service failure is propagated to the JDBC application, the database already rolls back the local transaction. The cache manager then cleans up all invalid connections. When an application holding an invalid connection tries to do work through that connection, it is possible to receive SQLException, ORA-17008, Closed Connection.

When an application receives a Closed Connection error message, it should do the following:

Retry the connection request. This is essential, because the old connection is no longer open.
Replay the transaction. All work done before the connection was closed has been lost.

Note:

The application should not try to roll back the transaction. The transaction was already rolled back in the database by the time the application received the exception.

9.2.2.2 How FCF Works

Under Fast Connection Failover, each connection in the cache maintains a mapping to a service, instance, database, and host name.

When a database generates an Oracle RAC event, that event is forwarded to the JVM in which JDBC is running. A daemon thread inside the JVM receives the Oracle RAC event and passes it on to the Connection Cache Manager. The Connection Cache Manager then throws SQL exceptions to the applications affected by the Oracle RAC event.

A typical failover scenario may work like the following:

A database instance fails, leaving several stale connections in the cache.
The Oracle RAC mechanism in the database generates an Oracle RAC event which is sent to the JVM containing JDBC.
The daemon thread inside the JVM finds all the connections affected by the Oracle RAC event, notifies them of the closed connection through SQL exceptions, and rolls back any open transactions.
Each individual connection receives a SQL exception and must retry.

9.2.3 Fast Connection Failover Prerequisites

Fast Connection Failover is available under the following circumstances:

The Universal Connection Pool is enabled.

Fast Connection Failover works in conjunction with the JDBC connection caching mechanism. This helps applications manage connections to ensure high availability.
The application uses service names to connect to the database.

The application cannot use service identifiers.
The underlying database has Oracle Database 12c Release 1 (12.1) Real Application Clusters (Oracle RAC) capability or Oracle Data Guard configured with either single instance Databases or Oracle RAC.

If failover events are not propagated, then connection failover cannot occur.
Oracle Notification Service (ONS) is configured and available on the node where JDBC is running.

JDBC depends on ONS to propagate database events and notify JDBC of them.
The Java Virtual Machine (JVM) in which your JDBC instance is running must have oracle.ons.oraclehome set to point to your ORACLE_HOME.

9.2.4 Example of Fast Connection Failover Configuration

The following example demonstrates a connection pool that uses the FCF feature. FCF is configured through a pool-enabled data source. The example includes enabling FCF, configuring the Oracle Notification Service (ONS) and configuring a connection URL. These topics are discussed after the example.

The isValid method of the oracle.ucp.jdbc.ValidConnection interface is typically used in conjunction with the FCF feature and is used to check if a borrowed connection is still usable after an SQL exception has been thrown due to an Oracle RAC down event. For example:

try {  conn = pds.getConnection;  ...}catch (SQLException sqlexc)
{
   if (conn == null || !((ValidConnection) conn).isValid())
        
   // take the appropriate action
   
...
conn.close();
}

Example 9-1 Fast Connection Failover Configuration Example

PoolDataSource  pds = PoolDataSourceFactory.getPoolDataSource();

pds.setConnectionPoolName("FCFSamplePool");
pds.setFastConnectionFailoverEnabled(true);
pds.setONSConfiguration("nodes=racnode1:4200,racnode2:4200\nwalletfile=
/oracle11/onswalletfile");
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setURL("jdbc:oracle:thin@(DESCRIPTION= "+
   "(LOAD_BALANCE=on)"+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=racnode1) (PORT=1521))"+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=racnode2) (PORT=1521))"+
   "(CONNECT_DATA=(SERVICE_NAME=service_name)))");
...

Related Topics

Checking If a Connection Is Valid

9.2.5 Enabling Fast Connection Failover

The FCF pool property is used to enable and disable FCF. FCF is disabled by default. The following example demonstrates enabling FCF as shown in Example 9-1.

pds.setFastConnectionFailoverEnabled(true);

Note:

Starting from Oracle Database 12c Release 1 (12.1.0.2), UCP supports the oracle.ucp.PlannedDrainingPeriod system property. It specifies the grace time period (in integer seconds) over which the pool smoothly drains the borrowed connections affected by a planned shut down. Draining starts when the same Database service becomes available on another instance different from the one that is going down.

When this property is not set, or set to 0, then the pool closes any affected borrowed connection immediately when it is returned to the pool.

Querying Fast Connection Failover Status

An application determines if Fast Connection Failover is enabled by calling OracleDataSource.getFastConnectionFailoverEnabled, which returns true if failover is enabled, false otherwise.

Note:

FCF must also be enabled to use run-time connection load balancing and connection affinity. These features are discussed later in this chapter.

9.2.6 What is ONS?

FCF relies on the Oracle Notification Service (ONS) to propagate database events between the connection pool and the Oracle RAC database. At run time, the connection pool must be able to setup an ONS environment. ONS (ons.jar) is included as part of the Oracle Client software. ONS can be configured using either remote configuration or client-side ONS daemon configuration. Remote configuration is the preferred configuration for standalone client applications. This section discusses the following topics:

9.2.6.1 Overview of ONS Configuration File

ONS configuration is controlled by the ONS configuration file, ORACLE_HOME/opmn/conf/ons.config. This file tells the ONS daemon how it should behave. Configuration information within ons.config is defined in simple name and value pairs.

Some parameters in the ons.config file are required and some are optional. Table 9-2 lists the required ONS configuration parameters and Table 9-3 lists the optional ONS configuration parameters. ONS must be refreshed after updating the ons.config file.

Table 9-2 Required ONS Configuration Parameters

Parameter Explanation

Parameter	Explanation
`localport`	Specifies the port that ONS binds to on the local host interface to talk to local clients. For example, `localport=4100`
`remoteport`	Specifies the port that ONS binds to on all interfaces for talking to other ONS daemons. For example, `remoteport=4200`
`nodes`	Specifies a list of other ONS daemons to talk to. Node values are given as a comma-delimited list of either host names or IP addresses plus ports. The port value that is given is the remote port that each ONS instance is listening on. In order to maintain an identical file on all nodes, the `host:port` of the current ONS node can also be listed in the nodes list. It will be ignored when reading the list. For example, `nodes=myhost.example.com:4200,123.123.123.123:4200` The nodes listed in the nodes line correspond to the individual nodes in the Oracle RAC instance. Listing the nodes ensures that the middle-tier node can communicate with the Oracle RAC nodes. At least one middle-tier node and one node in the Oracle RAC instance must be configured to see one another. As long as one node on each side is aware of the other, all nodes are visible. You need not list every single cluster and middle-tier node in the ONS configuration file of each Oracle RAC node. In particular, if one ONS configuration file cluster node is aware of the middle tier, then all nodes in the cluster are aware of it.

localport

Specifies the port that ONS binds to on the local host interface to talk to local clients.

For example, localport=4100

remoteport

Specifies the port that ONS binds to on all interfaces for talking to other ONS daemons.

For example, remoteport=4200

nodes

Specifies a list of other ONS daemons to talk to. Node values are given as a comma-delimited list of either host names or IP addresses plus ports. The port value that is given is the remote port that each ONS instance is listening on. In order to maintain an identical file on all nodes, the host:port of the current ONS node can also be listed in the nodes list. It will be ignored when reading the list.

For example, nodes=myhost.example.com:4200,123.123.123.123:4200

The nodes listed in the nodes line correspond to the individual nodes in the Oracle RAC instance. Listing the nodes ensures that the middle-tier node can communicate with the Oracle RAC nodes. At least one middle-tier node and one node in the Oracle RAC instance must be configured to see one another. As long as one node on each side is aware of the other, all nodes are visible. You need not list every single cluster and middle-tier node in the ONS configuration file of each Oracle RAC node. In particular, if one ONS configuration file cluster node is aware of the middle tier, then all nodes in the cluster are aware of it.

Table 9-3 Optional ONS Configuration Parameters

Parameter	Description
`logcomp`	Specifies the ONS components to log. The format is as follows: <component>[<subcomponent>,...];<component>[<subcomponent>,...];... If no subcomponents need to be specified, then do not include the brackets ([]) after the component name. To exclude messages from a subcomponent, precede the subcomponent name with an exclamation mark (!). For example, to exclude messages from the `topology` subcomponent, you use the following format: [all,!topology] Note that before specifying a subcomponent from which you want to exclude messages, you must first ensure that the subcomponent includes the messages. Following are the valid values for components: `internal` `ons` If you specify the component as `internal`, then there are no valid values for subcomponent. If you specify the component as `ons`, then you can specify the following values for subcomponent: `all`: Specifies all messages `ons`: ONS local information `listener`: ONS listener information `discover`: ONS discover (server or multicast) information `servers`: ONS remote servers currently up and connected to the cluster `topology`: ONS current cluster wide server connection topology `server`: ONS remote server connection information `client`: ONS client connection information `connect`: ONS generic connection information `subscribe`: ONS client subscription information `message`: ONS notification receiving and processing information `deliver`: ONS notification delivery information `special`: ONS special notification processing `internal`: ONS internal resource information `secure`: ONS SSL operation information `workers`: ONS worker threads The following example shows that you want to log messages for all the subcomponents under `ons`, except the `secure` subcomponent: logcomp=ons[all,!secure]
`logfile`	Specifies a log file that ONS should use for logging messages. The default value for log file is `$ORACLE_HOME/opmn/logs/ons.log`. For example, `logfile=/private/oraclehome/opmn/logs/myons.log`
`walletfile`	Specifies the wallet file used by the Oracle Secure Sockets Layer (SSL) to store SSL certificates. If a wallet file is specified to ONS, then it uses SSL when communicating with other ONS instances and require SSL certificate authentication from all ONS instances that try to connect to it. This means that if you want to turn on SSL for one ONS instance, then you must turn it on for all instances that are connected. This value should point to the directory where your `ewallet.p12` file is located. For example, `walletfile=/private/oraclehome/opmn/conf/ssl.wlt/default`
`useocr`	Specifies the value, reserved for use on the server-side, to indicate ONS whether it should store all Oracle RAC nodes and port numbers in Oracle Cluster Registry (OCR) instead of the ONS configuration file or not. A value of `useocr=on` is used to store all Oracle RAC nodes and port numbers in Oracle Cluster Registry (OCR). Do not use this option on the client-side.
`allowgroup`	Specifies the ONS setting to indicate the user group connecting to the `localport`. When set to `true`, ONS allows users within the same OS group to connect to its local port. When set to `false`, ONS only allows the same user running the ONS daemon to access its local port. The default value of this parameter is `false`. When using remote ONS configuration, there is no need to set this parameter.

The ons.config file allows blank lines and comments on lines that begin with the number sign (#).

9.2.6.2 Remote Configuration of ONS

UCP supports remote configuration of ONS through the ONSConfiguration pool property. The ONSConfiguration pool property value is a string that closely resembles the content of the ons.config file. The string contains a list of name=value pairs separated by a new line character (\n). You can set this pool property in the following two ways:

The name can be one of the following: nodes, walletfile, or walletpassword. The parameter string should at least specify the ONS configuration nodes attribute as a list of host:port pairs separated by a comma. SSL is used when the walletfile attribute is specified as an Oracle wallet file.

The following example demonstrates an ONS configuration string as shown in Example 9-1:
```
...
pds.setONSConfiguration("nodes=racnode1:4200,racnode2:4200\nwalletfile=/oracle11/onswalletfile");
...
```
The name can be only propertiesfile. The value is the location of an ONS-specific Java properties file. This file must contain the oracle.ons.nodes property, and one or both of the following ONS Java properties:
- oracle.ons.walletfile
- oracle.ons.walletpassword
  
  The following example illustrates such an ONSConfiguration string:
```
pds.setONSConfiguration("propertiesfile=/usr/ons/ons.properties");
```
  The following is an example of the content of the Java properties ons.properties file:
```
oracle.ons.nodes=racnode1:4200,racnode2:4200
oracle.ons.walletfile=/oracle11/onswalletfile
```

Note:

The parameters in the configuration string must match those for the Oracle RAC Database. In addition, if you are using Oracle Application Server, then you must configure ONS using procedures that are applicable to the server.

For standalone Java applications, you must configure ONS using the setONSConfiguration method. However, if your application meets the following requirements, then you no longer need to call the setONSConfiguration method for enabling FCF:

Your application is using Oracle Database 12c Release 1 (12.1) UCP and Oracle RAC Database 12c Release 1 (12.1)
Your application does not require ONS wallet or keystore

9.2.6.3 Configuration of Client-Side ONS Daemon

Client-side ONS daemon configuration is typical of applications that run on a middle-tier server such as the Oracle Application Server. Clients in this scenario directly configure ONS by updating the ons.config file. The location of the file may be different depending on the platform. Example 9-2 demonstrates an ons.config file for Example 9-1:

Note:

For client-side ONS daemon configuration, if the operating system (OS) user that starts the connection pool and the OS user that starts the client-side daemon are different, then they both must belong to the same OS group. Also, the value of the allowgroup parameter must be set to true in the ons.config file.

After configuring ONS, you start the ONS daemon with the onsctl command. You must make sure that an ONS daemon is running at all times.

Using the onsctl Command

After configuring, use ORACLE_HOME/opmn/bin/onsctl to start, stop, reconfigure, and monitor the ONS daemon. Table 9-4 is a summary of the commands that onsctl supports.

Table 9-4 onsctl Commands

Command	Effect	Output
`start`	Starts the ONS daemon	`onsctl: ons started`
`stop`	Stops the ONS daemon	`onsctl: shutting down ons daemon...`
`ping`	Verifies whether or not the ONS daemon is running	`ons is running ...`
`reconfig`	Triggers a reload of the ONS configuration without shutting down the ONS daemon
`help`	Prints a help summary message for onsctl
`detailed`	Prints a detailed help message for onsctl

9.2.7 Configuring the Connection URL

The connection URL of a connection factory must use the service name syntax when using FCF. The service name is used to map the connection pool to the service. In addition, the factory class must be an Oracle factory class. The following example demonstrates configuring the connection URL as shown in Example 9-1:

...
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setURL("jdbc:oracle:thin@//host:port/service_name");
...

Note:

An exception is thrown if a service identifier (SID) is specified for the connection URL when FCF is enabled.

The following examples demonstrate valid connection URL syntax when connecting to an Oracle RAC database. Examples for both the Oracle JDBC thin and Oracle OCI driver are included. Notice that the URL can be used to explicitly enable load balancing among Oracle RAC nodes:

Valid Connection URL Usage

pds.setURL("jdbc:oracle:thin@//host:port/service_name"); 

pds.setURL("jdbc:oracle:thin@//cluster-alias:port/service_name"); 

pds.setURL("jdbc:oracle:thin:@(DESCRIPTION= "+
   "(LOAD_BALANCE=on)"+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=host1)(PORT=1521))"+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521))"+
   "(CONNECT_DATA=(SERVICE_NAME=service_name)))");

pds.setURL("jdbc:oracle:thin:@(DESCRIPTION= "+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=cluster_alias) (PORT=1521)) "+
   "(CONNECT_DATA=(SERVICE_NAME=service_name)))");

pds.setURL("jdbc:oracle:oci:@TNS_ALIAS");

pds.setURL("jdbc:oracle:oci:@(DESCRIPTION= "+
   "(LOAD_BALANCE=on) "+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=host1) (PORT=1521)) "+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521)) "+
   "(CONNECT_DATA=(SERVICE_NAME=service_name)))"); 

pds.setURL("jdbc:oracle:oci:@(DESCRIPTION= "+
   "(ADDRESS=(PROTOCOL=TCP)(HOST=cluster_alias) (PORT=1521)) "+
   "(CONNECT_DATA=(SERVICE_NAME=service_name)))");