Skip navigation.

eDocs Home > BEA WebLogic Server and WebLogic Express 8.1 Documentation > Programming WebLogic JDBC > Using WebLogic Server with Oracle RAC

Programming WebLogic JDBC

  Previous Next vertical dots separating previous/next from contents/index/pdf Contents View as PDF   Get Adobe Reader

Using WebLogic Server with Oracle RAC

More and more customers are looking for solutions to make their back-end systems more scalable and more available. In response to these requests, BEA supports Oracle Real Application Clusters (RAC) for use with WebLogic Server.

The following sections describe the requirements and configuration tasks for using Oracle Real Application Clusters with WebLogic Server:

Both Oracle RAC and WebLogic Server are complex systems. To use them together requires specific configuration on both systems, as well as clustering software and a shared storage solution. This document describes the configuration required at a high level. For more details about configuring Oracle RAC, your clustering software, your operating system, and your storage solution, see the documentation from the respective vendors.

 

Overview of Oracle Real Application Clusters

Oracle Real Application Clusters (RAC) is a software component you can add to a high-availability solution that enables users on multiple machines to access a single database with increased performance. RAC comprises two or more Oracle database instances running on two or more clustered machines and accessing a shared storage device via cluster technology. To support this architecture, the machines that host the database instances are linked by a high-speed interconnect to form the cluster. The interconnect is a physical network used as a means of communication between the nodes of the cluster. Cluster functionality is provided by the operating system or compatible third party clustering software. Figure A-1 shows an Oracle RAC configuration.

Figure A-1 Oracle Real Application Clusters Configuration

Oracle Real Application Clusters Configuration


 

Oracle RAC offers features in the following areas:

Oracle RAC Scalability with WebLogic Server

An Oracle RAC installation appears like a single standard Oracle database and is maintained using the same tools and practices. All the nodes in the cluster execute transactions against the same database and RAC coordinates each node's access to the shared data to maintain consistency and ensure integrity. You can add nodes to the cluster easily and there is no need to partition data when you add them. This means that you can horizontally scale the database tier as usage and demand grows by adding RAC nodes, storage, or both. You can then scale WebLogic Server by adding a connection pool that maps to the new node.

Oracle RAC Availability with WebLogic Server

Because every RAC node in the cluster has equal access and authority, the loss of a node may impact performance but does not result in downtime. Depending upon your configuration, when a RAC node fails, in-flight transactions are redirected to another node in the cluster either by WebLogic Server or by the Oracle Thin driver. Note that Oracle RAC does not provide failover for database connections; nor does WebLogic Server. But transactions are failed over in the sense that they are heuristically driven to completion, based on the time of the failure.

Oracle RAC Load Balancing with WebLogic Server

If your application requires load balancing across RAC nodes when using global transactions (XA), BEA supports this capability through use of JDBC MultiPools with Oracle RAC nodes. The connection pools that form a Multipool are accessed using a round-robin scheme. When switching connections, WebLogic Server selects a connection from the next connection pool in the order listed. For information on which version(s) of WebLogic Server support JDBC MultiPools, Weblogic Platform Supported Configurations. For more information about using MultiPools with Oracle RAC, see Using MultiPools with Oracle RAC.

In a configuration without a MultiPool, WebLogic Server relies on the connect-time failover feature provided by the Oracle Thin driver to work with Oracle RAC. As described in Oracle's Technical Note 235118.1, the Oracle Thin driver cannot guarantee that a transaction is initiated and concluded on the same Oracle RAC instance when the driver is configured for load balancing. As Oracle RAC requires that all database operations inside a global transaction be routed to the same Oracle instance, this known limitation means that you cannot use driver-level load balancing when using XA with Oracle RAC and therefore you cannot use a primary/primary RAC configuration.

Oracle RAC Failover with WebLogic Server

Although Oracle RAC offers JDBC connect-time failover features, for most configurations, BEA recommends using WebLogic JDBC MultiPools to handle failover instead.

Note: Transparent Application Failover (TAF) requires the use of the Oracle OCI driver. Because BEA requires the use of the Oracle Thin driver, TAF is not supported.

 

Environment

When using WebLogic Server with Oracle RAC, consider the following requirements:

Note: See the WebLogic Platform Supported Configurations documentation for the latest WebLogic Server hardware platform and operating system support, and for the Oracle RAC versions supported by WebLogic Server versions and service packs. See the Oracle documentation for hardware and software requirements for running the Oracle RAC software.

Hardware Requirements

A typical WebLogic Server/Oracle RAC system includes a WebLogic Server cluster, an Oracle RAC cluster, and hardware for shared storage.

WebLogic Server Cluster

The WebLogic Server cluster can be configured in many ways and with various hardware options. See Using WebLogic Server Clusters for more details about configuring a WebLogic Server cluster.

Oracle RAC Cluster

For the latest hardware requirements for Oracle RAC, see the Oracle RAC documentation. However, to use Oracle RAC with WebLogic Server, you must run Oracle RAC instances on robust, production-quality hardware. The Oracle RAC configuration must deliver database processing performance appropriate for reasonably-anticipated application load requirements. Unusual database response delays can lead to unexpected behavior during database failover scenarios.

Shared Storage

In an Oracle RAC configuration, all data files, control files, and parameter files are shared for use by all RAC instances. An high availability (HA) storage solution that uses one of the following architectures is recommended:

For a complete list of supported storage solutions, see your Oracle documentation.

Software Requirements

To use WebLogic Server with Oracle RAC, you must install the following software on each RAC node:

Note: See the WebLogic Platform Supported Configurations documentation for the latest WebLogic Server hardware platform and operating system support, and for the Oracle RAC versions supported by WebLogic Server versions and service packs. See the Oracle documentation for hardware and software requirements required for running the Oracle RAC software.

 

Configuration Considerations for Oracle

Once you have installed and configured Oracle RAC, you must configure the listener process for each RAC instance as described in the sections that follow. For information about installing and configuring your operating system and the Oracle software for Oracle RAC nodes see the Oracle documentation.

Configuring the Listener Process for Each Oracle RAC Instance

For Oracle RAC, the listener process establishes a communication pathway between a user process and an Oracle instance. When you use Oracle RAC with WebLogic Server, the user process is typically a JDBC connection pool.

When a connection pool is created, it attempts to create a pool of database connections for applications to borrow. If a pooled database connection becomes inoperative or if the connection pool is configured to grow in capacity, the connection pool attempts to create additional database connections up to the maximum specified in the configuration file. In all of these instances, the Oracle listener process handles the connection request on the Oracle RAC instance.

Figure A-2 shows the Oracle listener process functionality.

Figure A-2 Oracle Listener Process Functionality

Oracle Listener Process Functionality


 

To enable this functionality, you must configure the listener process for each RAC instance in the Oracle cluster. BEA requires that you configure a local listener on each RAC instance. Each database instance should be configured to register with its local listener only.

Oracle instances can be configured to register with the listener statically in the listener.ora file or registered dynamically using the instance initialization parameter local_listener, or both. BEA recommends using dynamic registration.

A listener can start either a shared dispatcher process or a dedicated process. When using with WebLogic Server, BEA recommends using dedicated processes.

Disabling Remote Listeners

Although Oracle RAC allows you to configure remote listeners to handle connection failover, remote listeners are typically too slow and BEA does not support their use. To disable remote listeners, delete any listed remote listeners in spfile.ora on each RAC node. For example:

*.remote_listener=''

BEA recommends using one of the following methods of handling failover instead.

 

Configuration Options in WebLogic Server with Oracle RAC

When using WebLogic Server with Oracle 9i RAC or Oracle 10g RAC, you must configure your WebLogic Domain so that it can interact with RAC instances and so that it performs as expected. The following sections describe configuration options and requirements:

Choosing a WebLogic Server Configuration for Use with Oracle RAC

BEA supports several configuration options for using Oracle RAC with WebLogic Server:

The following table may help you as you try to determine which configuration is right for your particular application:

Does Your Application Require

Load Balancing?

Failover?

Global Transactions (XA)?

JMS JDBC Store?

Y

Y

Y

N

See Using MultiPools with Global Transactions.

Y

Y

N

N

See Using MultiPools without Global Transactions.

N

Y

N

Y

See Using Connect-Time Failover without Global Transactions

N

Y

Y

N

See Using Connect-Time Failover with Global Transactions

Required JDBC Drivers

To use WebLogic Server with Oracle RAC, your WebLogic JDBC connection pools must use the Oracle JDBC Thin driver 10g to create database connections.

Configuration Considerations for Failover

Consider the following information when configuring for failover.

MultiPool-Managed Failover

MultiPools offer failover for global transactions without the limitations and known issues associated with a connection pool configuration with connect-time failover. For a description of MultiPool failover features, see Configuring and Using MultiPools.

With this configuration, pictured in Figure A-3, MultiPool Configuration, on page A-13, you get:

The MultiPool handles failover for database connections when a RAC node becomes unavailable. When WebLogic Server tests a connection and the connection fails, it attempts to recreate the connection. If that attempt fails, the server disables the connection pool and routes connection requests to other connection pools (which correspond to other RAC nodes) in the MultiPool. WebLogic Server periodically tries to recreate the database connections in the disabled connection pool. When WebLogic Server is successful in recreating the connections, it next re-enables the connection pool and begins routing connection requests to the connection pool again. Because of the connection request routing and automatic health checking features, there is minimal delay in satisfying connection requests after a failure compared to when relying on the Oracle Thin driver connect-time failover configuration.

Connect-Time Failover

When MultiPools are not an option, WebLogic Server relies on the connect-time failover feature of the Oracle Thin driver to handle connection failover when a RAC instance becomes unavailable and a primary/primary configuration is not an option. When WebLogic Server tests a connection and the connection fails, the server replaces it by getting a new connection, and the driver again determines which RAC instance to use based on instance availability. The CountOfTestFailuresTillFlush="1" attribute helps to minimize the delay in delivering a connection to applications caused by the testing task. With this attribute setting, when a connection fails a connection test the first time, WebLogic Server automatically closes all connections in the connection pool. WebLogic Server replaces the connection with a new one, relying on the driver to determine to which node it should connect. In this case, the primary RAC node has failed, so the new connection is to the secondary RAC node. WebLogic Server tests the new connection before satisfying the request.

Delays During Failover

Occasionally, when one RAC node fails over to another, there may be a delay before the data associated with a transaction branch in progress on the now failed node is available throughout the cluster. This prevents incomplete transactions from being properly completed, which could further result in data locking in the database. To protect against the potential consequences of such a delay, WebLogic Server provides two configuration attributes that enable XA call retry for Oracle RAC: XARetryDurationSeconds and XARetryIntervalSeconds.

XARetryDurationSeconds controls the period of time during which WebLogic Server will repeatedly retry XA operations such as recover, commit and rollback for pending transactions. XARetryIntervalSeconds controls the frequency of the retry attempts within the established time period.

To enable XA call retries, add a value for XARetryDurationSeconds to all JDBC connection pools in your WebLogic domain that connect to an Oracle RAC instance. For example:

<JDBCConnectionPool 
        Name="oracleRACPool"
        DriverName="oracle.jdbc.xa.client.OracleXADataSource"
        ...
        XARetryDurationSeconds=480
        />

Note: XARetryDurationSeconds is available in the Administration Console. To enable this feature, you must manually edit your config.xml file or change the configuration using the weblogic.Admin command line utility or a JMX program.

Use the following formula to determine the value for XARetryDurationSeconds:

XARetryDurationSeconds = (longest transaction timeout for transactions that use connections from the connection pool) + (delay before XIDs are available on all RAC nodes, typically less than 5 minutes)

For example, if your application sets the longest transaction timeout as 180 seconds, you should set XARetryDurationSeconds to 180 seconds + 300 seconds, for a total of 480 seconds.

Note: It is generally better to set XARetryDurationSeconds higher than minimally necessary to make sure that all transactions are completed properly. Setting the value higher than minimally required should not affect application performance during normal operations. The additional processing only affects transactions that have been prepared but have failed to complete.

You can also optionally set a value for XARetryIntervalSeconds. This value determines the time between XA retry calls. By default, the value is 60 seconds. Decreasing the value will decrease the amount of time between XA retry attempts. The default value should suffice in most cases.

Failure Handling Walkthrough for Global Transactions

What happens to inflight transactions to a database node if that node fails? When the primary Oracle RAC node fails? Does WebLogic Server support transparent failover? To answer these and other questions about how WebLogic Server handles failures, let's walk through the transaction processing steps and describe how a failure would be handled at each stage along the way.

The first stage at which a failure may occur is before the application calls for the transaction to be committed. If a database or RAC node fails at this stage, the application receives an exception and must get a new connection and make a new attempt at processing the transaction. WebLogic Server does not support transparent failover.

If a failure occurs after the application has called for the transaction to be committed, the handling of any in-flight transaction depends upon whether the PREPARE operation is complete. If the PREPARE operation is not complete, the transaction manager rolls back the transaction and sends the application an exception for the failed transaction. If the PREPARE operation is complete, the transaction manager attempts to drive the in-flight transaction to completion using another node.

If a failure occurs during the COMMIT operation, the transaction manager attempts to retry the COMMIT operation several times. Note that the connection is blocked during these attempts. If the COMMIT operation is not successful during the first set of retry attempts, the application receives an exception. The transaction manager then continues to retry the COMMIT operation periodically until it is successful; if the transaction cannot be completed successfully within the abandon time period, the transaction is driven to completion heuristically.

Using MultiPools with Oracle RAC

To connect WebLogic Server to multiple Oracle RAC nodes using MultiPools, first configure a JDBC connection pool for each RAC instance in your RAC cluster with the Oracle Thin driver. Then configure a MultiPool, using either the algorithm for load balancing or the algorithm for high availability, and add the connection pools to it.

Note: WebLogic Server does not support the use of MultiPools with JMS JDBC Stores. If your application makes use of JMS JDBC Stores, you must configure your JMS JDBC Store to use Oracle RAC with connect-time failover. For more information see Using Connect-Time Failover with Oracle RAC.

Figure A-3 shows a typical MultiPool configuration.

Figure A-3 MultiPool Configuration

MultiPool Configuration


 

You can use the Administration Console or any other means that you prefer to configure your domain, such as the weblogic.Admin command line utility, the WebLogic Scripting Tool (WLST), or a JMX program. For information about configuring a WebLogic JDBC MultiPool see Configuring Multipools in the Administration Console Online Help.

To use a database connection in this configuration, your applications look up a data source on the JNDI tree and then request a connection from the data source. The data source communicates with the MultiPool and the MultiPool determines which connection pool to use to satisfy the connection request based on the algorithm type specified in the configuration (that is, high availability or load balancing).

Attributes of a MultiPool

The MultiPool may have the following attributes, depending on the role of RAC in your system—load balancing or failover:

Using MultiPools with Global Transactions

In this configuration, a MultiPool "pins" a transaction to one and only one Oracle RAC instance and failover is handled at the MultiPool level when a RAC instance becomes unavailable. If there is a failure on a RAC instance before PREPARE, the operation is retried until the retry duration has expired. If there is a failure after PREPARE the transaction is failed over to another instance.

Rules for Connection Pools within a MultiPool Using Global Transactions

The following rules apply to the XA connection pools within a MultiPool:

Notes:

Required Attributes of Connection Pools within a MultiPool Using Global Transactions

Each connection pool within the MultiPool should have the following attributes:

Sample config.xml Code

An example of the connection pools, a WebLogic JDBC MultiPool, and an associated data source in the config.xml file would be:

<JDBCConnectionPool 
	CapacityIncrement="1"
	ConnLeakProfilingEnabled="true" 
	CountOfTestFailuresTillFlush="1"
	DriverName="oracle.jdbc.xa.client.OracleXADataSource"
	InitialCapacity="5" 
	KeepXAConnTillTxComplete="true" 
	MaxCapacity="100"
	Name="jdbcXAPool1" 
	PasswordEncrypted="{3DES}lBifoTsg8fc="
	Properties="user=wlsqa"
	RefreshMinutes="1" SupportsLocalTransaction="true"
	Targets="WLSCluster" 
	TestConnectionsOnReserve="true"
	TestTableName="dual"
	URL="jdbc:oracle:thin:@db_server1:1521:SNRAC1"
	XAEndOnlyOnce="true" 
	XARetryDurationSeconds="300" 
	XASetTransactionTimeout="true"/>
	XATransactionTimeout="302"/>
<JDBCConnectionPool 
	CapacityIncrement="1"
	ConnLeakProfilingEnabled="true" 
	CountOfTestFailuresTillFlush="1"
	DriverName="oracle.jdbc.xa.client.OracleXADataSource"
	InitialCapacity="5" 
	KeepXAConnTillTxComplete="true" 
	MaxCapacity="100"
	Name="jdbcXAPool2" 
	PasswordEncrypted="{3DES}lBifoTsg8fc="
	Properties="user=wlsqa"
	RefreshMinutes="1" 
	SupportsLocalTransaction="true"
	Targets="WLSCluster" 
	TestConnectionsOnReserve="true"
	TestTableName="dual"
	URL="jdbc:oracle:thin:@db_server2:1521:SNRAC2"
	XAEndOnlyOnce="true" 
	XARetryDurationSeconds="300" 
	XASetTransactionTimeout="true"/> 
	XATransactionTimeout="302"/>
<JDBCMultiPool 
	Name="jdbcXAMultiPool1"
	PoolList="jdbcXAPool1,jdbcXAPool2" 
	Targets="WLSCluster"
	AlgorithmType="Load-Balancing"/>
<JDBCTxDataSource 
	JNDIName="jdbcXADataSource1" 
	Name="jdbcXADataSource1"
	PoolName="jdbcXAMultiPool1" 
	Targets="WLSCluster"/>

Note: Line breaks added for readability.

Using MultiPools without Global Transactions

The following sections describe a configuration that uses Oracle RAC with MultiPools in an application that does not require global transactions.

Attributes of Connection Pools within a MultiPool Not Using Global Transactions

Connection pools must have the following attributes:

Sample config.xml Code

An example of the connection pools, a WebLogic JDBC MultiPool, and an associated data source in the config.xml file would be:

<JDBCConnectionPool 
        Name="oracleRACPool_1"
        DriverName="oracle.jdbc.OracleDriver"
        InitialCapacity="5"
        MaxCapacity="100"
        Password="{3DES}I5fj3vh4+nI=" 
        Properties="user=SCOTT"
        RefreshMinutes="5"
        TestConnectionsOnReserve="true"
        TestTableName="dual"
        Targets="myWebLogicCluster"
        URL="jdbc:oracle:thin:@dbhost1:1521:dbservice"
        />
<JDBCConnectionPool 
        Name="oracleRACPool_2"
        DriverName="oracle.jdbc.OracleDriver"
        InitialCapacity="5"
        LoginDelaySeconds="1" 
        MaxCapacity="5"
        Password="{3DES}I5fj3vh4+nI=" 
        Properties="user=SCOTT"
        RefreshMinutes="5"
        TestConnectionsOnReserve="true"
        TestTableName="dual"
        PreparedStatementCacheSize="15"
        Targets="myWebLogicCluster"
        URL="jdbc:oracle:thin:@dbhost2:1521:dbservice"
        />
<JDBCMultiPool 
        AlgorithmType="Load-Balancing"
        Name="MyJDBCMultiPool" 
        HealthCheckFrequencySeconds="300"
        PoolList="oracleRACPool_1,oracleRACPool_2" 
        Targets="myWebLogicCluster"
        />
<JDBCDataSource 
        JNDIName="oracleRACDataSource"
        Name="oracleRACDataSource" 
        PoolName="MyJDBCMultiPool" 
        Targets="myWebLogicCluster"
        />

Note: Line breaks added for readability.

Using Connect-Time Failover with Oracle RAC

When MultiPools are not an option in your application (as when you are using a JMS JDBC Store, which does not support the use of MultiPools) and if load balancing is not required, you can configure your connection pools to use connect-time failover.

To connect WebLogic Server to multiple Oracle RAC nodes using connection pools configured for connect-time failover, configure a JDBC connection pool for each RAC instance in your RAC cluster with the Oracle Thin driver. Configure each connection pool to use connect-time failover, as described in the sections that follow. Figure A-4 shows an overview of the system.

Figure A-4 Connection Pool Configuration with Oracle Thin Driver Connect-Time Failover

Connection Pool Configuration with Oracle Thin Driver Connect-Time Failover


 

You can use the Administration Console or any other means that you prefer to configure your domain, such as the weblogic.Admin command line utility, the Weblogic Scripting Tool (WLST), or a JMX program.

When connections are created in the connection pool, the Oracle Thin driver determines which Oracle RAC instance to use. When an application gets a connection, it looks up a data source on the JNDI tree and requests a connection from the data source. The underlying connection pool delivers one of the available connections from the pool.

The following sections describe configuration options and requirements:

Using Connect-Time Failover without Global Transactions

The following sections describe a configuration which uses Oracle RAC's connect-time failover features to handle connection failures. With this configuration, in some failure cases, the failover time is as long as the TCP timeout, which can be several minutes, depending on your environment.

Attributes of a Connect-Time Failover Configuration without Global Transactions

To use this configuration, create JDBC connection pools in your WebLogic domain with the following attributes.

You can also optionally set a time-out value using the ConnectionReserveTimeoutSeconds attribute. This value determines the maximum time an application will wait for a connection to be made available. For example the following statement will set the time-out value to 120 seconds: 

ConnectionReserveTimeoutSeconds="120"

Sample config.xml Code

<JDBCConnectionPool Name="jdbcPool2"
	Targets="JdbcRacCluster"
	DriverName="oracle.jdbc.OracleDriver"
	URL="jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST = (ADDRESS = 
	(PROTOCOL = TCP)(HOST=dbhost1)(PORT=1521))
	(ADDRESS=(PROTOCOL=TCP)(HOST=dbhost2)(PORT=1521))
	(FAILOVER=on) (LOAD_BALANCE=off))
	(CONNECT_DATA = (SERVER=DEDICATED) (SERVICE_NAME =dbservice)))"
	InitialCapacity="10" 
	MaxCapacity="100" 
	CapacityIncrement="1" 
	Password="tiger" 
	Properties="user=scott" 
	PreparedStatementCacheSize="15" 
	ConnLeakProfilingEnabled="true" 
	TestTableName="dual" 
	TestConnectionsOnReserve="true" 
	CountOfTestFailuresTillFlush="1" /> 
<JDBCDataSource Name="jdbcDataSource2" 
	Targets="JdbcRacCluster" `
	JNDIName="jdbcDataSource2" 
	PoolName="jdbcPool2" />

Note: Line breaks added for readability.

Using Connect-Time Failover with Global Transactions

To use XA with a connect-time failover configuration, you must be using Oracle 9i RAC. When connections are created in the connection pool, the Oracle Thin driver determines which Oracle 9i RAC instance to use. When an application gets a connection, it looks up a data source on the JNDI tree and requests a connection from the data source. The underlying connection pool delivers one of the available connections from the pool.

With this configuration, in some failure cases, the failover time is as long as the TCP timeout, which can be several minutes, depending on your environment.

Attributes of a Connect-Time Failover Configuration with Global Transactions

To use this configuration, create JDBC connection pools in your WebLogic domain with the following attributes.

You can also optionally set a time-out value using the ConnectionReserveTimeoutSeconds attribute. This value determines the maximum time an application will wait for a connection to be made available. For example the following statement will set the time-out value to 120 seconds: 

ConnectionReserveTimeoutSeconds="120"

Sample config.xml Code

An example of the connection pool and associated data source in the config.xml file would be:

<JDBCConnectionPool 
        Name="oracleRACPool"
        DriverName="oracle.jdbc.xa.client.OracleXADataSource"
        InitialCapacity="5"
        LoginDelaySeconds="1" 
        MaxCapacity="5"
        Password="{3DES}I5fj3vh4+nI=" 
        Properties="user=SCOTT"
        CountOfTestFailuresTillFlush="1"
        KeepXAConnTillTxComplete="true"
        XARetryDurationSeconds="300"
        RefreshMinutes="5"
        TestConnectionsOnReserve="true"
        TestTableName="dual"
        XASetTransactionTimeout="true"
        StatementCacheSize="15"
        Targets="myWebLogicCluster"
        URL="jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=
        (ADDRESS=(PROTOCOL=TCP)(HOST=dbhost1)(PORT=1521))
        (ADDRESS=(PROTOCOL=TCP)(HOST=dbhost2)(PORT=1521))
        (FAILOVER=on)(LOAD_BALANCE=off)(CONNECT_DATA=(SERVER=DEDICATED)
        (SERVICE_NAME=dbservice)))"
        />
<JDBCTxDataSource 
        JNDIName="oracleRACDataSource"
        Name="oracleRACDataSource" 
        PoolName="oracleRACPool" 
        Targets="myWebLogicCluster"
        />

Note: Line breaks added for readability.

 

XA Considerations and Limitations with Oracle 9i RAC

When using XA (global transactions) with Oracle 9i RAC, consider the following requirements and limitations:

Required JDBC Driver Configuration for Use with XA

In this configuration, you must use the Oracle Thin driver connect-time failover to create database connections as described in Using Connect-Time Failover without Global Transactions.

Oracle 9i RAC XA Requirements

Oracle 9i RAC has the following requirements when using XA.

A Global Transaction Must Be Initiated, Prepared, and Concluded in the Same Instance of the RAC Cluster

Global transactions must be initiated, prepared, and concluded in the same instance of the RAC cluster. WebLogic Server connection pools manage this for you when you set KeepXAConnTillTxComplete="true" in the JDBC connection pool configuration.

Note: WebLogic Server relies on the connect-time failover feature in the Oracle Thin driver to work with Oracle RAC. As described in Oracle's Technical Note 235118.1, the Oracle Thin driver cannot guarantee that a transaction is initiated and concluded on the same RAC instance when the driver is configured for load balancing. As Oracle RAC requires that all database operations inside a global transaction be routed to the same Oracle instance, this known limitation means that you cannot use connect-time load balancing when using XA with Oracle RAC and, therefore, you cannot use a primary/primary RAC configuration.

Transaction IDs Must Be Unique Within the RAC Cluster

When using global transactions, transaction IDs (XIDs) must be unique within the RAC cluster. However, neither the Oracle Thin driver nor an Oracle RAC instance can determine if an XID is unique within the RAC cluster. Transactions with the same XID can execute SQL code on different instances of the RAC cluster without any exception.

The WebLogic Server Transaction Manager generates unique transaction IDs. However, in some failover scenarios, a transaction can continue on a RAC instance other than the originating instance, which can cause data inconsistencies. See Potential for Inconsistent Transaction Completion (Data Loss) in Some Failure Conditions.

Known Limitations When Using Oracle RAC with WebLogic Server

The following sections describe known issues and limitations when using XA and WebLogic Server with Oracle RAC:

Note: Some of these limitations are also described in Oracle's bug numbers 3428146 and 395790. Contact Oracle for more information about these issues.

Potential for Inconsistent Transaction Completion (Data Loss) in Some Failure Conditions

In some failure conditions, when MultiPools are not being used, transaction processing (data changes) that occurred on a RAC instance other than the instance on which a transaction was initiated will be lost without any notification or exception.

For example, consider the following WebLogic Server configuration:

In the following scenario, some data changes will be lost:

  1. Network connectivity between server2 and RAC1 is lost, which causes database connections in cp1 on server2 to fail over to RAC2. The same connection pool on server1 still has connections to RAC1.
  2. On server1, an application starts a transaction and uses a database connection from cp1 (a connection to RAC1) to make data changes.
  3. The application invokes an EJB on server2, which uses a database connection from cp1 on server2 (a connection to RAC2) to make data changes.
  4. The application completes the transaction on server1.

Result: Data changes on RAC1 are committed. Data changes on RAC 2 are ignored. The WebLogic Server transaction manager calls prepare and commit on the resource. In this case, because the connection pools have the same name, they are considered to be the same resource, so the calls are made on only one instance of the connection pool. Because the connection pools contain connections to different RAC instances, the data changes are committed on one RAC instance, but the changes on the other RAC instance are lost.

Workaround: Provide redundant network hardware between the WebLogic Server instance and the Oracle RAC instance to avoid the network failure.

Potential for Data Deadlocks in Some Failure Scenarios

There is a window of time in which transaction IDs are not available across the RAC cluster. Because of this known Oracle limitation, after some failure conditions, some incomplete transactions cannot be properly completed, which can result in deadlocks in the database. BEA has provided a patch to work around this known issue in 8.1 SP4, and recommends that customers using WebLogic Server and XA with Oracle RAC migrate to WebLogic Server 8.1 SP4 and use this patch.

Potential for Transactions Completed Out of Sequence

When using the Oracle DataBase Control, the order of transaction processing is not guaranteed. For example, if you implement a web service that uses DataBase Control do the following transaction sequence:

  1. Create a table
  2. Insert record 1
  3. Insert record 2
  4. Insert record 3
  5. Select records

If the primary node goes down momentarily after the table is created, it is possible that transactions submitted to the database are performed out of sequence.

Known Issue Occurring After Database Server Crash

If, while a transaction is being processed, the database server instance crashes after the PREPARE operation is complete but before the results of that operation have been written to the transaction log, a COMMIT call from a client for that transaction may hang for several minutes and possibly until the TCP timeout period has expired. The window of time in which this might occur is small and the problem occurs rarely. There is no workaround for the issue at this time.

 

JMS Store Recovery with Oracle RAC

If you are using a JMS JDBC Store with Oracle RAC, there are features and limitations to consider that concern Oracle RAC node failover. See the following sections:

Configuring a JMS JDBC Store for Use with Oracle RAC

The way that a JMS JDBC Store works limits the options you have for configuring one for use with Oracle RAC. A JMS JDBC Store holds on to a connection until that connection fails, at which point it grabs the next connection and repeats the process. Therefore you cannot implement load balancing with a JMS JDBC Store. Also, JMS JDBC Stores do not support MultiPools. The only remaining configuration option is to use connect-time failover without global transactions. For more information about this configuration option, see Using Connect-Time Failover without Global Transactions.

Automatic Retry

JMS has a limited connection retry mechanism which enables it to silently react to the failure of the RAC node that hosts its database connection. If the database has experienced either a minor network 'hiccup' or a RAC database has failed over to another node, the second connection attempt (the retry) will succeed to the next RAC node.

The time within which this retry is attempted and the number of retries attempted are limited to minimize the negative effects that an extended connection retry time could cause. If the database connection remains unavailable for a long period of time, the delay can impede the ability of JMS to properly continue its processing (for example, to maintain proper message ordering). Also, the transaction manager could declare the JMS resource of a transaction to be dead if there is not enough processing progress made within this time period, or out-of memory conditions could arise. There are system-level tuning guidelines that can help minimize the RAC failover time frame which is critical to the success of the automatic retry.

The tight loop on the automatic retry is particularly important when JMS processing occurs with transactions. If an I/O failure occurs in the JMS JDBC Store, the store record is in an unknown state which will put the message itself in an unknown state. To prevent the message from being committed in this unknown state, JMS will mark the transaction associated with the message as a "failedTransaction". Any future attempts by the transaction manager to finishing committing the message will cause JMS to throw a javax.transaction.xa.XAException with an errorCode set to XAException.XAER_RMERR. This exception is an indication to the transaction manager that a transient error has occurred in the resource manager (JMS) and that the transaction manager should retry commit processing. The retry logic provides a second attempt to establish the connection before JMS communicates any failure to the upper layer which would translate into an RMERR. If the RMERR is generated, then the only way to recover the message and complete the transaction is to restart the JMS Server manually. See Manual Retry.

Note: In certain scenarios, it is not possible to restart JMS server during runtime so, you need to restart WebLogic Server. If restarting WebLogic Server is not acceptable, you can choose File Store instead of JDBCStore. For more information, see "JMS Store Tasks" in JMS: Configuring in Administration Console Online Help.

The automatic retry logic is currently governed by an option on WebLogic Server as follows:

-Dweblogic.jms.store.JMSJDBCIORetryDelay=X

Where X is the number of seconds that should elapse before the connection to the database is retried. The default is one second. This value is restricted to the range 0 to 15, and the retry will only be attempted once. If a failure occurs on the second attempt, an exception will be propagated up the call stack and a manual restart will be required to recover the messages associated with the failed transaction.

Note: If you continue to encounter XAException.XAER_RMERR, try setting JMSJDBCIORetryDelay to a higher number.

Manual Retry

In certain test scenarios, empirical data has shown that it can take an extended period of time for everything to be completely transferred and operational (including transactional data) on a new RAC node once a particular node fails. If this exceeds the maximum elapsed time for the JMS automatic retry logic, then the current design expects that some type of manual intervention will occur to restart the JMS server so that it will begin processing again and deliver/recover messages in accordance with its specification and configuration. Below are two methods in which a JMS server can be manually restarted.

Alternative: JMS File Store

Generic file system anomalies can affect the availability of the file store (e.g., a disk crash, disk full, etc.). To mitigate these issues, use a data redundancy scheme such as dual-ported SCSI disks, a RAID array, or perhaps a SAN.

For more information, see the following documents:

 

Back to Top Previous Next