5 Configuring JDBC Multi Data Sources

This chapter provides information on how to configure and use a multi data source to provides load balancing or failover processing at the time of connection requests, between the data sources associated with the multi data source.

A multi data source is an abstraction around a group of data sources that is bound to the JNDI tree or local application context just like data sources are bound to the JNDI tree. Applications lookup a multi data source on the JNDI tree or in the local application context (java:comp/env) just as they do for data sources, and then request a database connection. The multi data source determines which data source to use to satisfy the request depending on the algorithm selected in the multi data source configuration: load balancing or failover.

This section includes the following information:

Multi Data Source Features

A multi data source can be thought of as a pool of data sources. Multi data sources are best used for failover or load balancing between nodes of a highly available database system, such as redundant databases or Oracle Real Application Clusters (Oracle RAC).

The data source member list for a Multi data source supports dynamic updates. This allows environments, such as those using Oracle RAC, to add and remove database nodes and corresponding data sources without redeployment and provide the ability to:

See Using Multi Data Sources with Oracle RAC.

Note:

Multi data sources do not provide any synchronization between databases. It is assumed that database synchronization is handled properly outside of WebLogic Server so that data integrity is maintained.

Removing a Database Node

You can remove a database node and corresponding data sources without redeployment. This capability provides you the ability to shutdown a node for maintenance or shrink a cluster. Use the following high-level steps to shutdown a database node:

Note:

Failure to follow these step may cause transaction roll-backs.
  1. Remove the data source from the multi data source. See "Add or remove data sources in a JDBC multi data source" in Oracle WebLogic Server Administration Console Help

  2. When all transactions have completed, suspend the data source. See "Suspend JDBC data sources" in Oracle WebLogic Server Administration Console Help

  3. When all transactions have completed, shut down the data source. See "Shut down JDBC data sources" in Oracle WebLogic Server Administration Console Help

  4. Shut down the database node.

Adding a Database Node

You can add a database node and corresponding data sources without redeployment. This capability provides you the ability to start a node after maintenance or grow a cluster. Use the following high-level steps to add a database node:

  1. Restart the database node.

  2. Restart the data source. See "Start JDBC data sources" in Oracle WebLogic Server Administration Console Help

  3. Add the data source back to the multi data source. See "Add or remove data sources in a JDBC multi data source" in Oracle WebLogic Server Administration Console Help

Creating and Configuring Multi Data Sources

You create a multi data source by first creating data sources, then creating the multi data source using the Administration Console or the WebLogic Scripting Tool and then assigning the data sources to the multi data source.

For instructions to create a multi data source, see "Configure JDBC multi data sources" in the Oracle WebLogic Server Administration Console Help.

For information about the configuration files created when configuring a multi data source, see Understanding JDBC Resources in WebLogic Server. Also see Creating a JDBC Multi Data Source Module.

Choosing the Multi Data Source Algorithm

Before you set up a multi data source, you need to determine the primary purpose of the multi data source—failover or load balancing. You can choose the algorithm that corresponds with your requirements.

Failover

The Failover algorithm provides an ordered list of data sources to use to satisfy connection requests. Normally, every connection request to this kind of multi data source is served by the first data source in the list. If a database connection test fails and the connection cannot be replaced, or if the data source is suspended, a connection is sought sequentially from the next data source on the list.

Note:

This algorithm requires that Test Reserved Connections (TestConnectionsOnReserve) on the data source is enabled. If enabled, a connection in the first data source is tested to verify if the data source is healthy. If the connection fails the test, the multi data source uses a connection from the next data source listed in the multi data source. See Connection Testing Options for a Data Source for information about configuring TestConnectionsOnReserve.

JDBC is a highly stateful client-DBMS protocol, in which the DBMS connection and transactional state are tied directly to the socket between the DBMS process and the client (driver). For this reason, failover of a connection while it is in use is not supported.

Load Balancing

Connection requests to a load-balancing multi data source are served from any data source in the list. The multi data source selects data sources to use to satisfy connection requests using a round-robin scheme. When the multi data source provides a connection, it selects a connection from the data source listed just after the last data source that was used to provide a connection. Multi data sources that use the Load Balancing algorithm also fail over to the next data source in the list if a database connection test fails and the connection cannot be replaced, or if the data source is suspended.

Multi Data Source Fail-Over Limitations and Requirements

WebLogic Server provides the Failover algorithm for multi data sources so that if a data source fails (for example, if the database management system crashes), your system can continue to operate. However, you must consider the following limitations and requirements when configuring your system.

Test Connections on Reserve to Enable Fail-Over

Data sources rely on the Test Reserved Connections (TestConnectionsOnReserve) feature on the data source to know when database connectivity is lost. Testing reserved connections must be enabled for the data sources within the multi data source. WebLogic Server will test each connection before giving it to an application. With the Failover algorithm, the multi data source uses the results from connection test to determine when to fail over to the next data source in the multi data source. After a test failure, the data source attempts to recreate the connection. If that attempt fails, the multi data source fails over to the next data source.

No Fail-Over for In-Use Connections

It is possible for a connection to fail after being reserved, in which case your application must handle the failure. WebLogic Server cannot provide fail-over for connections that fail while being used by an application. Any failure while using a connection requires that the application code close the failed connection, and the transaction must be restarted from the beginning with a new connection.

Multi Data Source Failover Enhancements

The following enhancements improve failover processing for multi data sources:

Connection Request Routing Enhancements When a Data Source Fails

To improve performance when a data source within a multi data source fails, WebLogic Server automatically disables the data source when a pooled connection fails a connection test. After a data source is disabled, WebLogic Server does not route connection requests from applications to the data source. Instead, it routes connection requests to the next available data source listed in the multi data source.

This feature requires that data source testing options are configured for all data sources in a multi data source, specifically Test Table Name and Test Reserved Connections. See Connection Testing Options for a Data Source.

If a callback handler is registered for the multi data source, WebLogic Server calls the callback handler before failing over to the next data source in the list. See Controlling Multi Data Source Failover with a Callback for more details.

Automatic Re-enablement on Recovery of a Failed Data Source within a Multi Data Source

After a data source is automatically disabled because a connection failed a connection test, the multi data source periodically tests a connection from the disabled data source to determine when the data source (or underlying database) is available again. When the data source becomes available, the multi data source automatically re-enables the data source and resumes routing connection requests to the data source, depending on the multi data source algorithm and the position of the data source in the list of included data sources. Frequency of these tests is controlled by the Test Frequency Seconds attribute of the multi data source. The default value for Test Frequency is 120 seconds, so if you do not specifically set a value for the option, the multi data source will test disabled data sources every 120 seconds. See "JDBC Multi Data Source: Configuration: General" in the Oracle WebLogic Server Administration Console Help.

WebLogic Server does not test and automatically re-enable data sources that you manually disable. It only tests data sources that are automatically disabled.

If a callback handler is registered for the multi data source, WebLogic Server calls the callback handler before re-enabling the data source. See Controlling Multi Data Source Failback with a Callback for more details.

Enabling Failover for Busy Data Sources in a Multi Data Source

By default, for multi data sources with the Failover algorithm, when the number of requests for a database connection exceeds the number of available connections in the current data source in the multi data source, subsequent connection requests fail.

To enable the multi data source to failover when all connections in the current data source are in use, you can enable the Failover Request if Busy option on the JDBC Multi Data Source: Configuration: General page in the Administration Console. (Also available as the FailoverRequestIfBusy attribute in the JDBCDataSourceParamsBean). If enabled (set to true), when all connections in the current data source are in use, application requests for connections will be routed to the next available data source within the multi data source. When disabled (set to false, the default), connection requests do not failover.

If a ConnectionPoolFailoverCallbackHandler is included in the multi data source configuration, WebLogic Server calls the callback handler before failing over. See Controlling Multi Data Source Failover with a Callback for more details.

Controlling Multi Data Source Failover with a Callback

You can register a callback handler with WebLogic Server that controls when a multi data source with the Failover algorithm fails over connection requests from one JDBC data source in the multi data source to the next data source in the list.

You can use callback handlers to control if or when the failover occurs so that you can make any other system preparations before the failover, such as priming a database or communicating with a high-availability framework.

Callback handlers are registered via the Failover Callback Handler attribute of the multi data source and are registered per multi data source. You must register the callback handler for each multi data source to which you want the callback handler to apply. And you can register different callback handlers for each multi data source in your domain.

Callback Handler Requirements

A callback handler used to control the failover and failback within a multi data source must include an implementation of the weblogic.jdbc.extensions.ConnectionPoolFailoverCallback interface. When the multi data source needs to failover to the next data source in the list or when a previously disabled data source becomes available, WebLogic Server calls the allowPoolFailover() method in the ConnectionPoolFailoverCallback interface, and passes a value for the three parameters, currPool, nextPool, and opcode, as defined below. WebLogic Server then waits for the return from the callback handler before completing the task.

Your application must return OK, RETRY_CURRENT, or DONOT_FAILOVER as defined below. The application should handle failover and failback cases.

See the Javadoc for the weblogic.jdbc.extensions.ConnectionPoolFailoverCallback interface for more details.

Note:

Failover callback handlers are optional.If no callback handler is specified in the multi data source configuration, WebLogic Server proceeds with the operation (failing over or re-enabling the disabled data source).

Callback Handler Configuration

There are two multi data source configuration attributes associated with the failover and failback functionality:

  • Failover Callback Handler (ConnectionPoolFailoverCallbackHandler)—To register a failover callback handler for a multi data source, you add a value for this attribute to the multi data source configuration. The value must be an absolute name, such as com.bea.samples.wls.jdbc.MultiDataSourceFailoverCallbackApplication. You can set the Failover Callback Handler using the Administration Console (see "Register a failover callback handler" in theOracle WebLogic Server Administration Console Help) or on the JDBCDataSourceParamsBean for the multi data source using WLST.

  • Test Frequency (TestFrequencySeconds)—To control how often the multi data source checks disabled (dead) data sources to see if they are now available. See Automatic Re-enablement on Recovery of a Failed Data Source within a Multi Data Source for more details.

How It Works—Failover

WebLogic Server attempts to failover connection requests to the next data source in the list when the current data source fails a connection test or, if you enabled FailoverRequestIfBusy, when all connections in the current data source are busy.

To enable the callback feature, you register the callback handler with Weblogic Server using Failover Callback Handler in the multi data source configuration.

With the Failover algorithm, connection requests are served from the first data source in the list. If a connection from that data source fails a connection test, WebLogic Server marks the data source as dead and disables it. If a callback handler is registered, WebLogic Server calls the callback handler, passing the following information, and waits for a return:

  • currPool—For failover, this is the name of data source currently being used to supply database connections. This is the "failover from" data source.

  • nextPool—The name of next available data source listed in the multi data source. For failover, this is the "failover to" data source.

  • opcode—A code that indicates the reason for the call:

    • OPCODE_CURR_POOL_DEAD—WebLogic Server determined that the current data source is dead and has disabled it.

    • OPCODE_CURR_POOL_BUSY—All database connections in the data source are in use. (Requires FailoverIfBusy=true in the multi data source configuration. See Enabling Failover for Busy Data Sources in a Multi Data Source.)

Failover is synchronous with the connection request: Failover occurs only when WebLogic Server is attempting to satisfy a connection request.

The return from the callback handler can indicate one of three options:

  • OK—proceed with the operation. In this case, that means to failover to the next data source in the list.

  • RETRY_CURRENT—Retry the connection request with the current data source.

  • DONOT_FAILOVER—Do not retry the current connection request and do not failover. WebLogic Server will throw a weblogic.jdbc.extensions.PoolUnavailableSQLException.

WebLogic Server acts according to the value returned by the callback handler.

If the secondary data sources fails, WebLogic Server calls the callback handler again, as in the previous failover, in an attempt to failover to the next available data source in the multi data source, if there is one.

Note:

WebLogic Server does not call the callback handler when you manually disable a data source.

For multi data sources with the Load-Balancing algorithm, WebLogic Server does not call the callback handler when a data source is disabled. However, it does call the callback handler when attempting to re-enable a disabled data source. See the following section for more details.

Controlling Multi Data Source Failback with a Callback

If you register a failover callback handler for a multi data source, WebLogic Server calls the same callback handler when re-enabling a data source that was automatically disabled. You can use the callback to control if or when the disabled data source is re-enabled so that you can make any other system preparations before the data source is re-enabled, such as priming a database or communicating with a high-availability framework.

See the following sections for more details about the callback handler:

How It Works—Failback

WebLogic Server periodically checks the status of data sources in a multi data source that were automatically disabled. (See Automatic Re-enablement on Recovery of a Failed Data Source within a Multi Data Source.) If a disabled data source becomes available and if a failover callback handler is registered, WebLogic Server calls the callback handler with the following information and waits for a return:

  • currPool—For failback, this is the name of the data source that was previously disabled and is now available to be re-enabled.

  • nextPool—For failback, this is null.

  • opcode—A code that indicates the reason for the call. For failback, the code is always OPCODE_REENABLE_CURR_POOL, which indicates that the data source named in currPool is now available.

Failback, or automatically re-enabling a disabled data source, differs from failover in that failover is synchronous with the connection request, but failback is asynchronous with the connection request.

The callback handler can return one of the following values:

  • OK—proceed with the operation. In this case, that means to re-enable the indicated data source. WebLogic Server resumes routing connection requests to the data source, depending on the multi data source algorithm and the position of the data source in the list of included data sources.

  • DONOT_FAILOVER—Do not re-enable the currPool data source. Continue to serve connection requests from the data source(s) in use.

WebLogic Server acts according to the value returned by the callback handler.

If the callback handler returns DONOT_FAILOVER, WebLogic Server will attempt to re-enable the data source during the next testing cycle as determined by the TestFrequencySeconds attribute in the multi data source configuration, and will call the callback handler as part of that process.

The order in which data sources are listed in a multi data source is very important. A multi data source with the Failover algorithm will always attempt to serve connection requests from the first available data source in the list of data sources in the multi data source. Consider the following scenario:

  1. MultiDataSource_1 uses the Failover algorithm, has a registered ConnectionPoolFailoverCallbackHandler, and includes three data sources: DS1, DS2, and DS3, listed in that order.

  2. DS1 becomes disabled, so MultiDataSource_1 fails over connection requests to DS2.

  3. DS2 then becomes disabled, so MultiDataSource_1 fails over connection requests to DS3.

  4. After some time, DS1 becomes available again and the callback handler allows WebLogic Server to re-enable the data source. Future connection requests will be served by DS1 because DS1 is the first data source listed in the multi data source.

  5. If DS2 subsequently becomes available and the callback handler allows WebLogic Server to re-enable the data source, connection requests will continue to be served by DS1 because DS1 is listed before DS2 in the list of data sources.

Deploying JDBC Multi Data Sources on Servers and Clusters

All data sources used by a multi data source to satisfy connection requests must be deployed on the same servers and clusters as the multi data source. A multi data source always uses a data source deployed on the same server to satisfy connection requests. Multi data sources do not route connection requests to other servers in a cluster or in a domain.

To deploy a multi data source to a cluster or server, you select the server or cluster as a deployment target. When a multi data source is deployed on a server, WebLogic Server creates an instance of the multi data source on the server. When you deploy a multi data source to a cluster, WebLogic Server creates an instance of the multi data source on each server in the cluster.

For instructions, see "Target and deploy JDBC multi data sources" in the Oracle WebLogic Server Administration Console Help.