Understanding WebLogic Integration High Availability

A clustered WebLogic Integration application provides scalability and high availability. A highly available deployment has recovery provisions in the event of hardware or network failures, and provides for the transfer of control to a backup component when a failure occurs.

The following sections describe clustering and high availability for a WebLogic Integration deployment:

For additional recommendations and database-specific requirements for configuring high availability WebLogic Integration applications, see Maintaining Availability in the Deployment section of the WebLogic Integration documentation set located at the following URL:

About WebLogic Integration High Availability

For a cluster to provide high availability, it must be able to recover from service failures. WebLogic Server supports failover for replicated HTTP session states, clustered objects, and services pinned to servers in a clustered environment. For information about how WebLogic Server handles such failover scenarios, see Communications in a Cluster in Using WebLogic Server Clusters.

Recommended Hardware and Software

The basic components of a highly available WebLogic Integration environment include the following:

An administration server.
A set of managed servers in a cluster.
An HTTP load balancer (router).
Physically shared, highly-available disk subsystems for transaction recovery—Transaction logs from a failed server must be available to a managed server in order for migration to occur. A typical and recommended way to do this is by using a multi-ported disk subsystem or SAN, and allowing two or more servers to mount file systems within the disk subsystem. It is not necessary for the file system to be simultaneously shared; it is only necessary for one server to mount a file system at any one time.
An IBM DB2, Microsoft SQL Server, Oracle, or Sybase database—You should take advantage of any high availability or failover solutions offered by your database vendor. (For database-specific information, see your database vendor's documentation.)

Note:

For information about availability and performance considerations associated with the various types of JDBC drivers, see in Configure JDBC Data sources in Administration Console Online Help.

A full discussion of how to plan the network topology of your clustered system is beyond the scope of this section. For information about how to fully utilize load balancing and failover features for your Web application by organizing one or more WebLogic Server clusters in relation to load balancers, firewalls, and Web servers, see Cluster Architectures in Using WebLogic Server Clusters.

For a simplified view of a cluster, showing the http load balancer, highly available database and multi-ported file system, see the following figure.

Regarding JMS File Stores

The default WebLogic Integration domain configuration uses a JDBC store for JMS servers. A file store can be used for JMS persistence in cases where a highly available multi-ported disk can be shared between managed servers, as described in the configuration shown in the preceding graphic. This will typically be more performant than a JDBC store.

For information about configuring JMS file stores, see "JMS Store Tasks" in JMS in Administration Console Online Help.

What Happens When a Server Fails

A server can fail due to either software or hardware problems. The following sections describe the processes that occur automatically in each case and the manual steps that must be taken in these situations.

Software Faults

Hardware Faults

If a hardware fault occurs, the physical machine may need to be repaired and could be out of operation for an extended period. In this case, the following events occur:

The http load balancer will detect the failed server and will redirect to other managed servers. (The actual algorithm for doing this will depend on the vendor for the http load balancer.)
All new internal requests, either RMI or JMS, will be redirected to other managed servers (JMS, if using distributed destinations).
All in-flight transactions on the failed server are terminated.
Another managed server can access process state, since it is held in the highly available database server. (A process will be in the state of the last successful transaction commit.)
JMS messages that are already enqueued are not automatically migrated, but must be manually migrated. For more information, see Server Migration.

Server Migration

In the case of a failure of extended duration, it may be necessary to migrate to another, operational managed server. When manually migrating a failed server to another managed server:

The transaction logs from the failed server must be made available to the new migrated server. If you are using a shared disk subsystem, you should mount the file system from the failed server containing the transaction logs on the migrated server.
The server must be manually migrated using the WebLogic Server Administration Console or, alternatively, through the command line utility.
When JTA is migrated, it will read the TLOGs from the failed server and recover any in-doubt transactions. We recommend that you migrate JTA before migrating JMS.
When JMS is migrated, it will allow access to the messages enqueued on the failed server.
Any "singleton" Message Driven Beans (message driven beans that were tied to physical queues rather than distributed destination) will be activated, if the migrated JMS server contains the physical queues needed by the message driven beans.

For detailed information regarding WebLogic Server migration, see the following topics in the WebLogic Server documentation set:

Failover and Replication in a Cluster in Using WebLogic Server Clusters
Recovering Failed Servers in Configuring and Managing WebLogic Server

WebLogic Integration Failure and Recovery

In addition to the high availability features of WebLogic Server, WebLogic Integration has failure and recovery characteristics that are based on the implementation and configuration of your WebLogic Integration solution. The following sections discuss failure and recovery topics for specific WebLogic Integration functional areas:

For additional information about WebLogic Integration failure and recovery topics, see WebLogic Integration Application Recovery in the WebLogic Integration Solutions Best Practices FAQ.

Trading Partner Integration

RosettaNet and ebXML handle failure and recovery differently because of differences in the business protocols. However, both protocols send messages that fail to be delivered after the configured number of retries to wli.b2b.failedmessage.queue. If you require additional processing of failed messages, you can implement custom message listeners for this queue.

RosettaNet

When message delivery fails in the case of RosettaNet messages, the WebLogic Integration protocol layer does not retry messages. It returns HttpStatus code to the workflow layer, instead. RosettaNet workflows are usually designed to handle retries.

The WebLogic Integration Administration Console enables you to specify retry intervals, retry counts, and process timeouts for various trading partners based on the PIP(s) being used. For example, RosettaNet typically supports three retries at two-hour intervals with an overall 24-hour limit on the life of the actual PIP exchange. For information about changing these settings, see "Viewing and Changing Bindings" in Trading Partner Management in Using the WebLogic Integration Administration Console.

If one instance of WebLogic Integration sends a message to another instance and the destination instance has failed, you may see one or more error messages followed by a stack trace in the server console.

ebXML

You can specify ebXML message retries using the WebLogic Integration Administration Console, Trading Partner Management Bulk Loader, or third-party Trading Partner Management message beans. If you set ebXML Delivery Semantics to OnceAndOnlyOnce or AtLeastOnce, messages will be retried according to the values you specify for Retry Count and Retry Interval. For information about using the WebLogic Integration Administration Console to set ebXML message retries, see "Defining Protocol Bindings" in Trading Partner Management in Using the WebLogic Integration Administration Console.

For ebXML processes, set the action mode value to non-default to guarantee recovery and high availability. For information about setting the action mode, see "ebXML Business Processes" in Introducing ebXML Solutions in Introducing Trading Partner Integration.

Application Integration

WebLogic Integration provides you with great flexibility in managing application integration resources for high availability. The following sections describe how application integration resources behave in the case of software or hardware failures, and actions you can take to recover when failover is not automatic:

Retargeting Services

In most cases, service invocations will continue uninterrupted because the application view containing the service is deployed to more than one managed server in the cluster. If this is not the case, use the WebLogic Server Administration Console to target the application view EJB and the adapter for the application view to a live managed server.

For information about using the WebLogic Server Administration Console to retarget services, see the following topics in the Administration Console Online Help:

"Setting an EJB Module's Target Server and/or Cluster" in Tasks.
"Changing the Target Servers for a Deployment" in Tasks.

Retargeting Events

In the case of a single managed server failure, delivery of events targeted to other managed servers in the cluster continues. Uninterrupted delivery of events targeted to the failed managed server will continue if both of the following conditions exist:

The event connection delivering the events has been deployed to more than one managed server in the cluster and at least one of those managed servers is still operational.
The adapter for the EIS supports multiple event connections within a cluster for a single application view or event type.

If the event connection was targeted to the failed managed server only, use the WebLogic Integration Administration Console to target the event connection to a live managed server on which the application view is deployed. Retargeting the event connection will cause event delivery to resume on the targeted managed server.

For information about using WebLogic Integration Administration Console to retarget event connections, see "Changing Event Generation Targets" in Application Integration in Using The WebLogic Integration Administration Console.

EIS Instance Failover

When an EIS instance fails, all service invocations and event deliveries cease. Asynchronous service requests to the failed EIS instance will fail until the affected application views and adapter instances are placed in the Suspended state. If there is an operational instance of the EIS available, you can edit the affected application views and adapter instances to make use of the operational instance. (You use the WebLogic Integration Administration Console to perform these edits.) Otherwise, service invocations and event deliveries continue when you take application views and adapter instances out of the Suspended state.

The following sections describe suspending application views and adapters, resuming operation of application views and adapter instances, and retargeting application views and adapter instances to a different EIS instance:

Suspending Application Views and Adapter Instances

In the case of an EIS failure, service requests and attempts at event delivery generate errors until the affected application views and adapter instances are suspended.

Application views and adapter instances can be suspended automatically or manually:

If you have enabled the AutoSuspend option for application views and adapter instances (and the adapter supports auto-suspend functionality), the application views and adapter instances automatically go into the Suspended state in the case of an EIS instance failure.

For information about using the WebLogic Integration Administration Console to enable the AutoSuspend option for application views, see "Viewing and Changing Application View Auto Suspend Settings" in Application Integration in Managing WebLogic Integration Solutions.

For information about enabling the AutoSuspend option for adapter instances, see the JavaDoc for the AdapterDeploymentMBean.

Note:

The WebLogic Integration Administration Console allows you to enable the auto suspend option for all adapters, whether or not they provide auto-suspend functionality. Contact your adapter vendor to verify that your adapter supports auto suspend. (The DBMS sample adapter supports auto suspend.)

If you have not enabled the AutoSuspend option or the adapter does not support auto-suspend functionality, you must first detect the EIS failure and then manually suspend the affected Application Views and adapter instances using the WebLogic Integration Administration Console.

For information about using the WebLogic Integration Administration Console to suspend Application Views or adapter instances, see "Suspending or Resuming an Application View or Adapter Instance" in Application Integration in Managing WebLogic Integration Solutions.

You can detect an EIS instance failure using a monitoring tool for the EIS or by monitoring the error counts for the application view or adapter instance in the WebLogic Integration Administration Console. For information about monitoring errors using the WebLogic Integration Administration Console, see Application Integration in Using The WebLogic Integration Administration Console.

Resuming Operation

Once an EIS instance is again operational, you must remove the affected application views and adapter instances from their Suspended state in order for service requests and event delivery to resume.

For information about using the WebLogic Integration Administration Console to return application views or adapter instances to the Deployed state, see "Suspending or Resuming an Application View or Adapter Instance" in Application Integration in Using The WebLogic Integration Administration Console.

Retargeting to an Operational EIS Instance

If you expect that an EIS instance failure will have an extended duration, you can point application views and adapter instances at an alternate, operational EIS instance. If an adapter instance already exists that points to an operational EIS instance, you can edit the Event Connection and Service Connection properties of any application view pointing to the failed EIS instance so that they are set to the adapter instance pointing to the operational EIS instance.

For information about using the WebLogic Integration Administration Console to change the adapter for an application view, see "Changing Event Connections for an Application View" and "Changing Service Connections for an Application View" in Application Integration in Using The WebLogic Integration Administration Console.

You can also edit the Event Connection and Service Connection properties that point to the old EIS instance, and give the Event and Service Connections new property values to point them to the new, operational EIS instance.

For more information on changing Event and Service Connection properties, see "Viewing and Changing Event Connection Properties" and "Viewing and Changing Service Connection Properties" in Application Integration in Using The WebLogic Integration Administration Console.

Deploying WebLogic Integration Solutions