Configuring and Managing WebLogic JDBC

Using WebLogic Server with Oracle RAC

More and more customers are looking for solutions to make their back-end systems more scalable and more available. In response to these requests, BEA supports Oracle Real Application Clusters (RAC) for use with WebLogic Server.

The following sections describe the requirements and configuration tasks for using Oracle Real Application Clusters with WebLogic Server:

Both Oracle RAC and WebLogic Server are complex systems. To use them together requires specific configuration on both systems, as well as clustering software and a shared storage solution. This document describes the configuration required at a high level. For more details about configuring Oracle RAC, your clustering software, your operating system, and your storage solution, see the documentation from the respective vendors.

Overview of Oracle Real Application Clusters

Oracle Real Application Clusters (RAC) is a software component you can add to a high-availability solution that enables users on multiple machines to access a single database with increased performance. RAC comprises two or more Oracle database instances running on two or more clustered machines and accessing a shared storage device via cluster technology. To support this architecture, the machines that host the database instances are linked by a high-speed interconnect to form the cluster. The interconnect is a physical network used as a means of communication between the nodes of the cluster. Cluster functionality is provided by the operating system or compatible third party clustering software. Figure B-1 shows an Oracle RAC configuration.

Oracle RAC offers features in the following areas:

Oracle RAC Scalability with WebLogic Server

An Oracle RAC installation appears like a single standard Oracle database and is maintained using the same tools and practices. All the nodes in the cluster execute transactions against the same database and RAC coordinates each node's access to the shared data to maintain consistency and ensure integrity. You can add nodes to the cluster easily and there is no need to partition data when you add them. This means that you can horizontally scale the database tier as usage and demand grows by adding RAC nodes, storage, or both. You can then scale WebLogic Server by adding a data source that maps to the new node.

Oracle RAC Availability with WebLogic Server

Because every RAC node in the cluster has equal access and authority, the loss of a node may impact performance but does not result in downtime. Depending upon your configuration, when a RAC node fails, in-flight transactions are redirected to another node in the cluster either by WebLogic Server or by the Oracle Thin driver. Note that Oracle RAC does not provide failover for database connections; nor does WebLogic Server. But transactions are failed over in the sense that they are driven to completion, based on the time of the failure.

Oracle RAC Load Balancing with WebLogic Server

If your application requires load balancing across RAC nodes, BEA supports this capability through use of JDBC multi data sources with Oracle RAC nodes. The data sources that form a multi data source are accessed using a round-robin scheme. When switching connections, WebLogic Server selects a connection from the next data source in the order listed. For more information about using multi data sources with Oracle RAC, see Using Multi Data Sources with Oracle RAC.

When multi data sources are not an option, load balancing when using XA is not supported. In a configuration without a multi data source, WebLogic Server relies on the connect-time failover feature provided by the Oracle Thin driver to work with Oracle RAC. As described in Oracle's Technical Note 235118.1, the Oracle Thin driver cannot guarantee that a transaction is initiated and concluded on the same Oracle RAC instance when the driver is configured for load balancing. As Oracle RAC requires that all database operations inside a global transaction be routed to the same Oracle instance, this known limitation means that you cannot use driver-level load balancing when using XA with Oracle RAC and therefore you cannot use a primary/primary RAC configuration.

Oracle RAC Failover with WebLogic Server

Although Oracle RAC offers JDBC connect-time failover features, for most configurations, BEA recommends using WebLogic JDBC multi data sources to handle failover instead. While connect-time failover does not provide the ability to pre-create connections to alternate RAC nodes, multi data sources have multiple connections available at all times to handle failover. For more information see Using Multi Data Sources with Oracle RAC.

Environment

When using WebLogic Server with Oracle RAC, consider the following requirements:

Hardware Requirements

A typical WebLogic Server/Oracle RAC system includes a WebLogic Server cluster, an Oracle RAC cluster, and hardware for shared storage.

WebLogic Server Cluster

The WebLogic Server cluster can be configured in many ways and with various hardware options. See Using WebLogic Server Clusters for more details about configuring a WebLogic Server cluster.

Oracle RAC Cluster

For the latest hardware requirements for Oracle RAC, see the Oracle RAC documentation. However, to use Oracle RAC with WebLogic Server, you must run Oracle RAC instances on robust, production-quality hardware. The Oracle RAC configuration must deliver database processing performance appropriate for reasonably-anticipated application load requirements. Unusual database response delays can lead to unexpected behavior during database failover scenarios.

Shared Storage

In an Oracle RAC configuration, all data files, control files, and parameter files are shared for use by all RAC instances. An HA storage solution that uses one of the following architectures is recommended:

For a complete list of supported storage solutions, see your Oracle documentation.

Software Requirements

To use WebLogic Server with Oracle RAC, you must install the following software on each RAC node:

Configuration Considerations for Oracle

Once you have installed and configured Oracle RAC, you must configure the listener process for each RAC instance as described in the sections that follow. For information about installing and configuring your operating system and the Oracle software for Oracle RAC nodes see the Oracle documentation.

Configuring the Listener Process for Each Oracle RAC Instance

For Oracle RAC, the listener process establishes a communication pathway between a user process and an Oracle instance. When you use Oracle RAC with WebLogic Server, the user process is typically a JDBC data source.

When a data source is created, it attempts to create a pool of database connections for applications to borrow. If a pooled database connection becomes inoperative or if the data source is configured to grow in capacity, the data source attempts to create additional database connections up to the maximum specified in the configuration file. In all of these instances, the Oracle listener process handles the connection request on the Oracle RAC instance.

Figure B-2 shows the Oracle listener process functionality.

To enable this functionality, you must configure the listener process for each RAC instance in the Oracle cluster. BEA requires that you configure a local listener on each RAC instance. Each database instance should be configured to register with its local listener only.

Oracle instances can be configured to register with the listener statically in the listener.ora file or registered dynamically using the instance initialization parameter local_listener, or both. BEA recommends using dynamic registration.

A listener can start either a shared dispatcher process or a dedicated process. When using with WebLogic Server, BEA recommends using dedicated processes.

Disabling Remote Listeners

Although Oracle RAC allows you to configure remote listeners to handle connection failover, remote listeners are typically too slow and BEA does not support their use. To disable remote listeners, delete any listed remote listeners in spfile.ora on each RAC node. For example:

   *.remote_listener=''

BEA recommends using JDBC multi data sources to handle failover instead.

For more information see Configuration Considerations for Failover.

Configuration Options in WebLogic Server with Oracle RAC

When using WebLogic Server with Oracle 9i RAC or Oracle 10g RAC, you must configure your WebLogic Domain so that it can interact with RAC instances and so that it performs as expected. The following sections describe configuration options and requirements:

Choosing a WebLogic Server Configuration for Use with Oracle RAC

BEA supports several configuration options for using Oracle RAC with WebLogic Server:

The following table may help you as you try to determine which configuration is right for your particular application:

Required JDBC Drivers

To use WebLogic Server with Oracle RAC, your WebLogic JDBC data sources must use the Oracle JDBC Thin driver 10g to create database connections.

Configuration Considerations for Failover

Consider the following information when configuring for failover.

Multi Data Source-Managed Failover

Multi data sources offer failover for global transactions without the limitations and known issues associated with a data source configuration with connect-time failover. For a description of multi data source failover features, see “ Multi Data Source Failover Enhancements” in Programming WebLogic JDBC.

With this configuration, pictured in Figure B-3, you get:

The multi data source handles failover for database connections when a RAC node becomes unavailable. When WebLogic Server tests a connection and the connection fails, it attempts to recreate the connection. If that attempt fails, the server disables the data source and routes connection requests to other data sources (which correspond to other RAC nodes) in the multi data source. WebLogic Server periodically tries to recreate the database connections in the disabled data source. When WebLogic Server is successful in recreating the connections, it next re-enables the data source and begins routing connection requests to the data source again. Because of the connection request routing and automatic health checking features, there is minimal delay in satisfying connection requests after a failure compared to when relying on the Oracle Thin driver connect-time failover configuration.

Connect-Time Failover

When multi data sources are not an option, WebLogic Server relies on the connect-time failover feature of the Oracle Thin driver to handle connection failover when a RAC instance becomes unavailable and a primary/primary configuration is not an option. When WebLogic Server tests a connection and the connection fails, the server replaces it by getting a new connection, and the driver again determines which RAC instance to use based on instance availability.When a connection fails a connection test, WebLogic Server automatically closes all connections in the data source. WebLogic Server replaces the connection with a new one, relying on the driver to determine to which node it should connect. In this case, the primary RAC node has failed, so the new connection is to the secondary RAC node. WebLogic Server tests the new connection before satisfying the request.

Delays During Failover

Occasionally, when one RAC node fails over to another, there may be a delay before the data associated with a transaction branch in progress on the now failed node is available throughout the cluster. This prevents incomplete transactions from being properly completed, which could further result in data locking in the database. To protect against the potential consequences of such a delay, WebLogic Server provides two configuration attributes that enable XA call retry for Oracle RAC: XARetryDurationSeconds and XARetryIntervalSeconds.

XARetryDurationSeconds controls the period of time during which WebLogic Server will repeatedly retry XA operations such as recover, commit and rollback for pending transactions. XARetryIntervalSeconds controls the frequency of the retry attempts within the established time period.

To enable XA call retries, add a value for XARetryDurationSeconds to all JDBC data sources in your WebLogic domain that connect to an Oracle RAC instance. For example:

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>oracleRACXAPool</name> 
  ...
  <jdbc-xa-params>
    ...
    <xa-retry-duration-seconds>300</xa-retry-duration-seconds> 
  </jdbc-xa-params>
</jdbc-data-source>

Use the following formula to determine the value for XARetryDurationSeconds:

XARetryDurationSeconds = (longest transaction timeout for transactions that use connections from the data source) + (delay before XIDs are available on all RAC nodes, typically less than 5 minutes)

For example, if your application sets the longest transaction timeout as 180 seconds, you should set XARetryDurationSeconds to 180 seconds + 300 seconds, for a total of 480 seconds.

You can also optionally set a value for XARetryIntervalSeconds. This value determines the time between XA retry calls. By default, the value is 60 seconds. Decreasing the value will decrease the amount of time between XA retry attempts. The default value should suffice in most cases.

To enable XARetryDurationSeconds and XARetryIntervalSeconds from the Administration Console, use the following steps:

Optionally, you can use the weblogic.Admin command line utility, the WebLogic Scripting Tool (WLST), or a JMX program.

Failure Handling Walkthrough for Global Transactions

What happens to inflight transactions to a database node if that node fails? When the primary Oracle RAC node fails? Does WebLogic Server support transparent failover? To answer these and other questions about how WebLogic Server handles failures, let’s walk through the transaction processing steps and describe how a failure would be handled at each stage along the way.

The first stage at which a failure may occur is before the application calls for the transaction to be committed. If a database or RAC node fails at this stage, the application receives an exception and must get a new connection and make a new attempt at processing the transaction. WebLogic Server does not support transparent failover.

If a failure occurs after the application has called for the transaction to be committed, the handling of any in-flight transaction depends upon whether the PREPARE operation is complete. If the PREPARE operation is not complete, the transaction manager rolls back the transaction and sends the application an exception for the failed transaction. If the PREPARE operation is complete, the transaction manager attempts to drive the in-flight transaction to completion using another node.

If a failure occurs during the COMMIT operation, the transaction manager attempts to retry the COMMIT operation several times. Note that the connection is blocked during these attempts. If the COMMIT operation is not successful during the first set of retry attempts, the application receives an exception. The transaction manager then continues to retry the COMMIT operation periodically until it is successful; if the transaction cannot be completed successfully within the abandon time period, the transaction is driven to completion heuristically.

Using Multi Data Sources with Oracle RAC

To connect WebLogic Server to multiple Oracle RAC nodes using multi data sources, first configure a JDBC data source for each RAC instance in your RAC cluster with the Oracle Thin driver. Then configure a multi data source, using either the algorithm for load balancing or the algorithm for failover, and add the data sources to it.

Figure B-3 shows a typical multi data source configuration.

You can use the Administration Console or any other means that you prefer to configure your domain, such as the weblogic.Admin command line utility, the WebLogic Scripting Tool (WLST), or a JMX program. For information about configuring a WebLogic JDBC multi data source see Configuring JDBC Multi Data Sources.

To use a database connection in this configuration, your applications look up one multi data source on the JNDI tree and then request a connection. The multi data source determines which data source to use to satisfy the connection request based on the algorithm type specified in the configuration (that is, failover or load balancing).

Attributes of a Multi Data Source

The multi data source may have the following attributes, depending on the role of RAC in your system—load balancing or failover:

Using Multi Data Sources with Global Transactions

In this configuration, a multi data source “pins” a transaction to one and only one Oracle RAC instance and failover is handled at the multi data source level when a RAC instance becomes unavailable. If there is a failure on a RAC instance before PREPARE, the operation is retried until the retry duration has expired. If there is a failure after PREPARE the transaction is failed over to another instance.

Rules for Data Sources within a Multi Data Source Using Global Transactions

The following rules apply to the XA data sources within a multi data source:

Required Attributes of Data Sources within a Multi Data Source Using Global Transactions

Each data source within the multi data source should have the following attributes:

Sample Configuration Code

Sample configuration code for a multi data source and two associated data sources is shown below.

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>oracleRACXAPool</name> 
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@lcqsol24:1521:SNRAC1</url> 
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name> 
    <properties>
      <property>
        <name>user</name> 
        <value>wlsqa</value> 
      </property>
    </properties>
    <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted> 
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name> 
    <profile-type>0</profile-type> 
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>oracleRACXAJndiName</jndi-name>
    <global-transactions-protocol>TwoPhaseCommit
        </global-transactions-protocol>
  </jdbc-data-source-params>
  <jdbc-xa-params>
    <keep-xa-conn-till-tx-complete>true</keep-xa-conn-till-tx-complete> 
    <xa-end-only-once>true</xa-end-only-once> 
    <xa-set-transaction-timeout>true</xa-set-transaction-timeout> 
    <xa-transaction-timeout>120</xa-transaction-timeout> 
    <xa-retry-duration-seconds>300</xa-retry-duration-seconds> 
  </jdbc-xa-params>
</jdbc-data-source>

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>oracleRACXAPool2</name> 
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@lcqsol25:1521:SNRAC2</url> 
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name> 
    <properties>
      <property>
        <name>user</name> 
        <value>wlsqa</value> 
      </property>
    </properties>
    <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted> 
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name> 
    <profile-type>0</profile-type> 
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>oracleRACXAJndiName2</jndi-name> 
    <global-transactions-protocol>TwoPhaseCommit
        </global-transactions-protocol> 
  </jdbc-data-source-params>
  <jdbc-xa-params>
    <keep-xa-conn-till-tx-complete>true</keep-xa-conn-till-tx-complete> 
    <xa-end-only-once>true</xa-end-only-once> 
    <xa-set-transaction-timeout>true</xa-set-transaction-timeout> 
    <xa-transaction-timeout>120</xa-transaction-timeout> 
    <xa-retry-duration-seconds>300</xa-retry-duration-seconds> 
  </jdbc-xa-params>
</jdbc-data-source>

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>oracleRACXAMDS</name> 
  <jdbc-data-source-params>
    <jndi-name>oracleRACMDSJndiName</jndi-name> 
    <algorithm-type>Load-Balancing</algorithm-type> 
    <data-source-list>oracleRACXAPool,oracleRACXAPool2</data-source-list> 
  </jdbc-data-source-params>
</jdbc-data-source>

Using Multi Data Sources without Global Transactions

The following sections describe a configuration that uses Oracle RAC with multi data sources in an application that does not require global transactions.

Attributes of Data Sources within a Multi Data Source Not Using Global Transactions

Data sources must have the following attributes:

Sample Configuration Code

Sample configuration code for a WebLogic JDBC multi data source and associated data sources is shown below.

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>jdbcPool</name> 
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@lcqsol24:1521:snrac1</url> 
    <driver-name>oracle.jdbc.OracleDriver</driver-name> 
    <properties>
      <property>
        <name>user</name> 
        <value>wlsqa</value> 
      </property>
    </properties>
    <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted> 
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <test-connections-on-reserve>true</test-connections-on-reserve> 
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name> 
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbcDataSource</jndi-name> 
  </jdbc-data-source-params>
</jdbc-data-source>

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>jdbcPool2</name> 
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@lcqsol25:1521:SNRAC2</url> 
    <driver-name>oracle.jdbc.OracleDriver</driver-name> 
    <properties>
      <property>
        <name>user</name> 
        <value>wlsqa</value> 
      </property>
    </properties>
    <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted> 
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <test-connections-on-reserve>true</test-connections-on-reserve> 
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name> 
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbcDataSource2</jndi-name> 
    <global-transactions-protocol>OnePhaseCommit
         </global-transactions-protocol> 
  </jdbc-data-source-params>
</jdbc-data-source>

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>jdbcNonXAMultiPool</name> 
  <jdbc-data-source-params>
    <jndi-name>jdbcDataSource</jndi-name> 
    <algorithm-type>Failover</algorithm-type> 
    <data-source-list>jdbcPool,jdbcPool2</data-source-list> 
    <failover-request-if-busy>true</failover-request-if-busy> 
  </jdbc-data-source-params>
</jdbc-data-source>

Using Connect-Time Failover with Oracle RAC

When multi data sources are not an option in your application, you can configure your data sources to use connect-time failover and load balancing. To connect WebLogic Server to multiple Oracle RAC nodes using data sources configured for connect-time failover and load balancing, configure a JDBC data source for each RAC instance in your RAC cluster with the Oracle Thin driver, as described in the sections that follow. Figure B-4 shows an overview of the system.

You can use the Administration Console or any other means that you prefer to configure your domain, such as the weblogic.Admin command line utility, the Weblogic Scripting Tool (WLST), or a JMX program.

When connections are created in the data source, the Oracle Thin driver determines which Oracle RAC instance to use. When an application gets a connection, it looks up a data source on the JNDI tree and requests a connection from the data source. The data source delivers one of the available connections from the pool of connections in the data source.

The following sections describe configuration options and requirements:

Using Connect-Time Failover without Global Transactions

The following sections describe a configuration which uses Oracle RAC’s connect-time failover features to handle connection failures. With this configuration, in some failure cases, the failover time is as long as the TCP timeout, which can be several minutes, depending on your environment.

Attributes of a Connect-Time Failover Configuration without Global Transactions

To use this configuration, create JDBC data sources in your WebLogic domain with the following attributes.

Sample Configuration Code

Sample configuration code is shown below.

<jdbc-data-source xmlns="http://www.bea.com/ns/weblogic/91"
  xmlns:sec="http://www.bea.com/ns/weblogic/91/security"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:wls="http://www.bea.com/ns/weblogic/91/security/wls"
  xsi:schemaLocation="http://www.bea.com/ns/weblogic/91/domain.xsd">
  <name>oracleRACNonXAPool</name> 
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@(DESCRIPTION=
         (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)
         HOST=lcqsol24)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)
         (HOST=lcqsol25)(PORT=152))(FAILOVER=on)
         (LOAD_BALANCE=off))(CONNECT_DATA=(SERVER=DEDICATED)
         (SERVICE_NAME=snrac)))</url> 
    <driver-name>oracle.jdbc.OracleDriver</driver-name> 
    <properties>
      <property>
        <name>user</name> 
        <value>wlsqa</value> 
      </property>
    </properties>
    <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted> 
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <test-connections-on-reserve>true</test-connections-on-reserve> 
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name> 
    <profile-type>4</profile-type> 
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>oracleRACJndiName</jndi-name> 
    <global-transactions-protocol>OnePhaseCommit
         </global-transactions-protocol> 
  </jdbc-data-source-params>
</jdbc-data-source> 

Using Fast Connection Failover

WebLogic Server supports Fast Connection Failover, a Oracle feature which provides an application independent method to implement RAC event notifications, such a detection and cleanup of invalid connections, load balancing of available connections, and work redistribution on active RAC instances.

For more information, see Fast Connection Failover in the Oracle® Database JDBC Developer's Guide and Reference.

Required JDBC Driver Configuration for use with Oracle Fast Connection Failover

To enable Fast Connection Failover on an data source, set the following connection pool properties:

XA Considerations and Limitations with Oracle RAC

When using XA (global transactions) with Oracle RAC, consider the following requirements and limitations.

Required JDBC Driver Configuration for Use with XA

In this configuration, you must use the Oracle Thin driver connect-time failover to create database connections as described in Using Connect-Time Failover without Global Transactions.

Oracle 9i RAC XA Requirements

Oracle 9i RAC has the following requirements when using XA.

A Global Transaction Must Be Initiated, Prepared, and Concluded in the Same Instance of the RAC Cluster

Global transactions must be initiated, prepared, and concluded in the same instance of the RAC cluster. WebLogic Server data sources manage this for you when you set KeepXAConnTillTxComplete="true" in the JDBC data source configuration.

Transaction IDs Must Be Unique Within the RAC Cluster

When using global transactions, transaction IDs (XIDs) must be unique within the RAC cluster. However, neither the Oracle Thin driver nor an Oracle RAC instance can determine if an XID is unique within the RAC cluster. Transactions with the same XID can execute SQL code on different instances of the RAC cluster without any exception.

The WebLogic Server Transaction Manager generates unique transaction IDs. However, in some failover scenarios, a transaction can continue on a RAC instance other than the originating instance, which can cause data inconsistencies. See Potential for Inconsistent Transaction Completion (Data Loss) in Some Failure Conditions.

Known Limitations When Using Oracle 9i RAC with WebLogic Server

The following sections describe known issues and limitations when using XA and WebLogic Server with Oracle RAC:

Potential for Inconsistent Transaction Completion (Data Loss) in Some Failure Conditions

In some failure conditions, when multi data sources are not being used, transaction processing (data changes) that occurred on a RAC instance other than the instance on which a transaction was initiated will be lost without any notification or exception.

For example, consider the following WebLogic Server configuration:

In the following scenario, some data changes will be lost:

Result: Data changes on RAC1 are committed. Data changes on RAC 2 are ignored. The WebLogic Server transaction manager calls prepare and commit on the resource. In this case, because the data sources have the same name, they are considered to be the same resource, so the calls are made on only one instance of the data source. Because the data sources contain connections to different RAC instances, the data changes are committed on one RAC instance, but the changes on the other RAC instance are lost.

Workaround: Provide redundant network hardware between the WebLogic Server instance and the Oracle RAC instance to avoid the network failure.

Potential for Data Deadlocks in Some Failure Scenarios

There is a window of time in which transaction IDs are not available across the RAC cluster. Because of this known Oracle limitation, after some failure conditions, some incomplete transactions cannot be properly completed, which can result in deadlocks in the database. To prevent these failure conditions from arising, WebLogic Server provides two configuration attributes that enable XA call retry for Oracle RAC: XARetryDurationSeconds and XARetryIntervalSeconds. For more information about these configuration options, see Delays During Failover.

Potential for Transactions Completed Out of Sequence

When using the Oracle DataBase Control, the order of transaction processing is not guaranteed. For example, if you implement a web service that uses DataBase Control do the following transaction sequence:

If the primary node goes down momentarily after the table is created, it is possible that transactions submitted to the database are performed out of sequence.

Known Issue Occurring After Database Server Crash

If, while a transaction is being processed, the database server instance crashes after the PREPARE operation is complete but before the results of that operation have been written to the transaction log, a COMMIT call from a client for that transaction may hang for several minutes and possibly until the TCP timeout period has expired. The window of time in which this might occur is small and the problem occurs rarely. There is no workaround for the issue at this time.

JDBC Store Recovery with Oracle RAC

If you are using a JDBC Store with Oracle RAC, there are features and limitations to consider that concern Oracle RAC node failover. See the following sections:

Configuring a JDBC Store for Use with Oracle RAC

The way that a JDBC Store works limits the options you have for configuring one for use with Oracle RAC. You cannot configure a JDBC store to use a JDBC data source that is configured to support global transactions. The JDBC store must use a JDBC data source that uses a non-XA JDBC driver.

A JDBC Store holds on to a connection until that connection fails, at which point it moves on to the next connection and repeats the process. Therefore you cannot implement load balancing with a JDBC Store, including using a load balancing multi data source. You should configure a multi data source for a JDBC store to use the Failover algorithm. For more information about this configuration option, see Using Multi Data Sources without Global Transactions.

Automatic Retry

JMS has a limited connection retry mechanism which enables it to silently react to the failure of the RAC node that hosts its database connection. If the database has experienced either a minor network 'hiccup' or a RAC database has failed over to another node, the second connection attempt (the retry) will succeed to the next RAC node.

The time within which this retry is attempted and the number of retries attempted are limited to minimize the negative effects that an extended connection retry time could cause. If the database connection remains unavailable for a long period of time, the delay can impede the ability of JMS to properly continue its processing (for example, to maintain proper message ordering). Also, the transaction manager could declare the JMS resource of a transaction to be dead if there is not enough processing progress made within this time period, or out-of memory conditions could arise. There are system-level tuning guidelines that can help minimize the RAC failover time frame which is critical to the success of the automatic retry.

The tight loop on the automatic retry is particularly important when JMS processing occurs with transactions. If an I/O failure occurs in the JDBC Store, the store record is in an unknown state which will put the message itself in an unknown state. To prevent the message from being committed in this unknown state, JMS will mark the transaction associated with the message as a “failedTransaction.” Any future attempts by the transaction manager to finishing committing the message will cause JMS to throw a javax.transaction.xa.XAException with an errorCode set to XAException.XAER_RMERR. This exception is an indication to the transaction manager that a transient error has occurred in the resource manager (JMS) and that the transaction manager should retry commit processing. The retry logic provides a second attempt to establish the connection before JMS communicates any failure to the upper layer which would translate into an RMERR. If the RMERR is generated, then the only way to recover the message and complete the transaction is to restart WebLogic Server.

The automatic retry logic is currently governed by an option on WebLogic Server as follows:

-Dweblogic.store.jdbc.IORetryDelayMillis=X

Where X is the number of milliseconds that should elapse before the connection to the database is retried. The default value is 1000 milliseconds. This value is restricted to the range 0 to 15000 milliseconds, and the retry will only be attempted once. If a failure occurs on the second attempt, an exception will be propagated up the call stack and a manual restart will be required to recover the messages associated with the failed transaction.