14 Recovering from a Server Failure

Understand how WebLogic JMS client applications reconnect or recover from a server/network failure and learn how to migrate JMS data after a server failure.

• Automatic JMS Client Failover

• Manually Migrating JMS Data to a New Server

Automatic JMS Client Failover

With the automatic JMS client reconnect feature, if a server or network failure occurs, some JMS client objects will transparently failover to use another server instance, as long as one is available.

Note:

The WebLogic JMS automatic reconnect feature is deprecated. The JMS connection factory configuration, javax.jms.extension.WLConnection API, and javax.jms.extension.JMSContext API for this feature will be removed or ignored in a future release. They do not handle all possible failures and so are not an effective substitute for standard resiliency best practices. Oracle recommends that client applications handle connection exceptions as described in Client Resiliency Best Practices in Administering JMS Resources for Oracle WebLogic Server.

With the automatic JMS client reconnect feature, if a fatal server failure occurs, then JMS clients automatically attempt to reconnect to the server when it becomes available.

A network connection failure could be due to transient reasons (a temporary interruption in the network connection) or non-transient reasons (a server bounce or network failure). In such cases, some JMS client objects will try to automatically operate with another server instance in a cluster, or possibly with the host server.

By default, JMS producer session objects automatically try to reconnect to an available server instance without any manual configuration or modifications to the existing client code. If you do not want your JMS producers to be automatically reconnected, then you must explicitly disable this feature either programmatically or administratively.

In addition, JMS consumer session objects can also be configured to automatically attempt to reconnect to an available server, but due to their potentially asynchronous nature, you must explicitly enable this capability using the WebLogic Server Administration Console or public WebLogic JMS APIs.

Related Topics

• Automatic Reconnect Limitations

• Automatic Failover for JMS Producers

• Configuring Automatic Failover for JMS Consumers

• Explicitly Disabling Automatic Failover on JMS Clients

• Best Practices for JMS Clients Using Automatic Failover

Automatic Reconnect Limitations

Automatic reconnect logic can provide a seamless failover for clients in many failure scenarios. However, there are some connection failure scenarios where the result of a message operation is undetermined and WebLogic Server throws an exception. Your application must deal with the exception appropriately. For instance:

• If the message send operation is idempotent, resend the message.

• Otherwise, your application may need to take some action. For instance, you may need to check if the message is already available on the queue before resending to avoid duplicates.

Note:

If the destination or distributed destination member is unavailable, you will not be able to determine if the message send operation was successful until that member becomes available.

Implicit failover of the following JMS objects is not supported before WebLogic Server 9.2:

• Queue browsers: javax.jms.QueueBrowser

• The WebLogic JMS thin client (wljmsclient.jar) does not automatically reconnect.

• Client statistics are reset on each reconnect, which results in the loss historical data for the client.

• Under some circumstances, automatic reconnect is not possible. If it is not possible, an exception is reported.

• Temporary destinations (javax.jms.TemporaryQueue and javax.jms.TemporaryTopic).

  Note:

  Temporary destinations may still be accessible after a sever/network failure. This is because temporary destinations are not always on the same server instance as the local connection factory due to server load balancing. Therefore, if a temporary destination survives a server/network failure and a producer continues sending messages to it, an auto-reconnected consumer may or may not be able consume messages from the same temporary destination it was connected to before the failure occurred.

Automatic Failover for JMS Producers

In most cases, JMS producer applications will transparently failover to another server instance if one is available. The following WebLogic JMS producer-oriented objects will attempt to automatically reconnect to an available sever instance without any manual configuration or modification to the existing client code:

• Connection

• Session

• MessageProducer

If you do not want your JMS clients to be automatically reconnected, then you must explicitly disable this feature either programatically or administratively, as described in Explicitly Disabling Automatic Failover on JMS Clients.

Sample Producer Code

In the event of a network failure, the WebLogic JMS client code for message production will try to reconnect to an available server during Steps 3-8 shown in Example 14-1.

Example 14-1 Sample JMS Client Code for Message Production

  //set exception listener
1. public void onException(javax.jms.JMSException jsme) {
         connection.setExceptionListener
         // handle the exception, which may require checking for duplicates
         // or sending the message again 
         }

2.  Context ctx = create WebLogic JNDI context with credentials etc.
3.   ConnectionFactory cf = ctx.lookup(JNDI name of connection factory)
4.   Destination dest     = ctx.lookup(JNDI name of destination)
     // the following operations recover from network failures
5.   Connection  con      = cf.createConnection()
6.   Session     sess     = con.createSession(no transactions, ack mode)
7.   MessageProducer prod = sess.createProducer(dest)

8.   Loop over:
9.      Message msg = sess.createMessage() 
       // try block to handle destination availablitiy scenarios
10.     try {  
              prod.send(msg)}
           catch (Some Destination Availability Exception e) {
              //handle the exception, in most cases, the destination or member
              //is not yet available, so the code should try to resend
           }
      //end loop

     // done sending messages
11.   con.close(); ctx.close(); 

The JMS producer will transparently fail-over to another server instance, if one is available. This keeps the client code as simple as listed in Example 14-1and eliminates the need for client code for retrying across network failures.

The WebLogic JMS does not reconnect MessageConsumers by default. For this to automatically occur programmatically, your client application code must call the WebLogic WLConnection extension, with the setReconnectPolicy set to "all", as explained in Configuring Automatic Failover for JMS Consumers.

Re usable ConnectionFactory Objects

A ConnectionFactory object looked up using JNDI (see Step 1 in Example 14-1 and Example 14-2) is re usable after a server or network failure without requiring a re-lookup. A network failure could be between the JMS client JVM and the remote WebLogic Server instance it is connected to as part of the JNDI lookup, or between the JMS client JVM and any remote WebLogic Server instance in the same cluster where the JMS client subsequently connects.

Re usable Destination Objects

A destination object (queue or topic) looked up using JNDI (see Step 2 in Example 14-1 and Example 14-2) is re usable after a server or network failure without requiring another lookup. The same principle applies to producers that send to a distributed destinations, because the client looks up the distributed destination in JNDI, and not the unavailable distributed member.

A network failure could be between the client JVM and the WebLogic Server instance it is connected to, or between that WebLogic Server instance and the WebLogic Server instance that actually hosts the destination. The Destination object will also be robust after restarting the WebLogic Server instance hosting the destination.

Note:

For information on how consumers of distributed destinations behave with automatic JMS client reconnect, see Consumers of Distributed Destinations.

Reconnected Connection Objects

The JMS connection object is used to map one-to-one to a physical network connection between the client JVM and a remote WebLogic Server instance. With the JMS client reconnect feature, the JMS Connection object that the client gets from the ConnectionFactory.createConnection() method (see Step 3 in Example 14-1 and Example 14-2) maps in a one-to-one-at-a-time fashion to the physical network connection. One consequence is that while the JMS client continues to use the same Connection object, it could be actually communicating with a different WebLogic Server instance after an implicit failover.

If there is a network disconnection and a subsequent implicit refresh of the connection, then all objects derived from the connection (such as javax.jms.Session and javax.jms.MessageProducer objects) are also implicitly refreshed. During the refresh, any synchronous operation on the connection or its derived objects that go to the server (such as producer.send() or connection.createSession()), may block for a period of time before giving up on the connection refresh. This time is configured using the WebLogic Server Administration Console or the setReconnectBlockingMillis(long) API in the weblogic.jms.extension.WLConnection interface.

The reconnect feature keeps trying to reconnect to the WebLogic Server instance's ConnectionFactory object in the background until the application calls connection.close(). The ReconnectBlockingMillis parameter is the time-out for a synchronous caller trying to use the connection when the connection in being retried in the background.

If a synchronous call times out without seeing a refreshed connection, then it then behaves in exactly the same way (that is, throws the same Exceptions) as without the implicit reconnect (that is, it will behave as if it was called on a stale connection without the reconnect feature).

The caller can then decide to retry the synchronous call (with a potentially lower quality of service, like duplicate messages), or decide to call connection.close()method , which will terminate the background retries for that connection.

Special Cases for Reconnected Connections

There are special cases that can occur when producer connections are refreshed:

• Connections with a ClientID for Durable Subscribers – If your Reconnect Policy field is set to None or Producer, and a JMS Connection has a Client ID specified at the time of a network/server failure, then the Connection will not be automatically refreshed. The reason for this restriction is backward compatibility, which avoids breaking existing JMS applications that try to re-create a JMS Connection with the same connection name after a failure. If implicit failover also occurs on a network failure, then the application's creation of the connection will fail due to a duplicate ClientID.

  Note:

  For information on how a consumer connection with a ClientID behaves, see Consumer Connections with a ClientID for Durable Subscriptions.

• Closed Objects Are Not Refreshed – When the application calls javax.jms.Connection.close(), javax.jms.Session.close(), etc., that object and it descendents are not refreshed. Similarly, when the JMS client is told its Connection has been administratively destroyed, it is not refreshed.

• Connection with Registered Exception Listener – If the JMS Connection has an application ExceptionListener registered on it, that ExceptionListener's onException() callback will be invoked even if the connection is implicitly refreshed. This notifies the application code of the network disconnect event. The JMS client application code might normally call connection.close() in onException; however, if it wants to take advantage of the reconnect feature, it may choose not to call connection.close(). The registered ExceptionListener is also migrated transparently to the internally refreshed connection to listen for exceptions on the refreshed connection.

• Multiple Connections – If there are multiple JMS Connections created off the same ConnectionFactory object, each connection will behave independently of the other connections as far as the reconnect feature is concerned. Each connection will have its own connection status, its own connection retry machinery, etc.

Reconnected Session Objects

As described in Reconnected Connection Objects, JMS Session objects are refreshed when their associated JMS connection gets refreshed (see Step 4 in Example 14-1 and Example 14-2). Session states, such as acknowledge mode and transaction mode, are preserved across each refresh occurrence. The same session object can be used for calls, like createMessageProducer(), after a refresh.

Special Cases for Reconnected Sessions

These sections discuss special cases that can occur when Sessions are reconnected.

• Transacted Sessions With Pending Commits or Rollbacks – Operations similar to non-transacted JMS Sessions, transacted JMS sessions are automatically refreshed. However, if there were send or receive operations on a session pending a commit or rollback at the time of the network disconnect, then the first commit call after the Session refresh will fail throwing a javax.jms.TransactionRolledBackException. When a JMS session transaction spans a network refresh, the commit for that transaction cannot vouch for the operations done before the refresh as part of that transaction (from an application code perspective).

  After a session refresh, operations like send() or receive() will not throw an exception; it is only the first commit after a refresh that will throw an exception. However, the first commit after a session refresh will not throw an exception if there were no pending transactional operations in that JMS session at the time of the network disconneciont. In case of Session.commit() throwing the exception, the client application code can simply retry all the operations in the transaction again with the same (implicitly refreshed) JMS objects. The stale operations before a refresh will not be committed and will not be duplicated.

• Pending Unacknowledged Messages – If a session had unacknowledged messages prior to the session refresh, then the first WLSession.acknowledge() call after a refresh throws a weblogic.jms.common.LostServerException. This indicates that the acknowledge() call may not have removed messages from the server. As a result, the refreshed session may receive duplicate messages that were also delivered before the disconnect.

Reconnected MessageProducer Objects

As described in Reconnected Connection Objects, JMS MessageProducer objects are refreshed when their associated JMS connection gets refreshed (see Step 5 in Example 14-1). If producers are non-anonymous, that is, they are specific to a destination object (standalone or distributed destination), then the producer's destination is also implicitly refreshed, as described in Re usable Destination Objects. If a producer is anonymous, that is not specific to a destination object, then the possibly stale destination object specified on the producer's send() operation is implicitly refreshed.

Special Case for Distributed Destinations

It is possible that a producer can send a message at the same time that a distributed destination member becomes unavailable. If WebLogic JMS can determine that the distributed destination member is not available, or was not available when the message was sent, the system will retry sending the message to another distributed member. If there is no way to determine if the message made it through the connection all the way to the distributed member before it went down, the system will not attempt to resend the message because doing so may create a duplicate message. In that case, WebLogic JMS will throw an exception. It is up to the application to catch that exception and decide whether or not to resend the message.

Configuring Automatic Failover for JMS Consumers

JMS MessageConsumer objects that are part of a JMS Connection (through a JMS Session) can be refreshed during a JMS connection refresh (see Step 5 in Example 14-2). However, due to the stateful nature of JMS consumers, as well as their potential asynchronous nature, you must explicitly enable this capability using either the weblogic.jms.extension.WLConnection API or the WebLogic Server Administration Console.

Explicitly enabling automatic refresh of consumers also refreshes connections with a configured client ID for a durable subscriber, as described in Consumer Connections with a ClientID for Durable Subscriptions. However, refreshed consumers does not include QueueBrowser clients, which are never refreshed, as described in Automatic Reconnect Limitations.

Sample Consumer Client Code

When Message Consumer refresh is explicitly activated, in the event of a network failure, the WebLogic JMS client code for message consumption will attempt to reconnect during Steps 3-8 in Example 14-2.

Example 14-2 Sample JMS Client Code for Message Consumption

0.  Context ctx = create WebLogic JNDI context with credentials etc.
1.  ConnectionFactory cf = ctx.lookup(JNDI name of connection factory)
2.  Destination dest     = ctx.lookup(JNDI name of destination)
     // the following operations recover from network failures
3.  Connection  con      = cf.createConnection()
    (weblogic.jms.extensions.WLConnection)con).setReconnectPolicy("all")
4.  Session     sess     = con.createSession(no transactions, auto ack)
5.  MessageConsumer cons = sess.createConsumer(dest, message selector)
    - also for async consumers : cons.setMessageListener(onMessage impl)
6.  con.start()
7.  Loop over:
      for sync consumers: Message msg = consumer.receive() 
      for async consumers (in different thread): onMessage() invoked
8.  con.close(), ctx.close() 

Note that the connection factory does not refresh MessageConsumer objects by default. For this to occur programmatically, your client application code must call the WebLogic WLConnection extension, with the setReconnectPolicy set to "all", as shown in Step 3 in Example 14-2.

Configuring Automatic Client Refresh Options

The JMS client reconnect API includes the following configuration parameters, which enables you to make some choices that affect the behavior of the reconnect feature for consumers.

Table 14-1 Automatic JMS Client Reconnect Options

Console Label/MBean Attribute	Value	Description
Reconnect Policy ReconnectPolicy	None Producer (default) All	Determines which JMS client objects are implicitly refreshed when a network disconnect or server reboot. It only affects the implicit refresh of connections, sessions, producers, and consumers derived from this connection factory. This attribute does not affect Destination or ConnectionFactory objects in the JMS client, since those objects are always refreshed implicitly. Nor does it affect the QueueBrowser object in the JMS client, since that object is never refreshed.
Reconnect Blocking Time ReconnectBlockingTimeMillis	6000	Determines how long any synchronous JMS calls, such as producer.send(), consumer.receive(), and session.createBrowser() will block the calling thread before giving up on a JMS client reconnect in progress.
TotalReconnectPeriodMillis	-1	Determines how long JMS clients should keep retrying to connect after either the initial network disconnection or the last synchronous JMS call attempt (whichever occurs most recently), before giving up retrying.

For instructions about configuring client parameters on a connection factory using the WebLogic Server Administration Console, see Configure connection factory client parameters in the Oracle WebLogic Server Administration Console Online Help. For more information about these parameters, see ClientParamsBean in the MBean Reference for Oracle WebLogic Server.

Common Cases for Reconnected Consumers

This section describes the common scenarios when refreshing synchronous and asynchronous consumers.

Synchronous Consumers

Synchronous consumers use MessageConsumer.receive(), MessageConsumer.receive(timeout), and MessageConsume.receiveNoWait() methods to consume messages. The first two methods are already expected to potentially block the application code, while the third method is not expected to block the application code. To retain these semantics, the following rules describe interaction of the reconnect feature with the synchronous consumer calls:

• MessageConsumer.receive()– If there is a network disconnection during this call, this method can block for up to Reconnect Blocking Time property (described in the configuration section) for a reconnect to go through before throwing an Exception.

• MessageConsumer.receive(timeout) – This call will block for the at-most timeout in milliseconds specified by the caller. If the Reconnect Blocking Time property is less than the timeout, then the receive will still block up to the Reconnect Blocking Time setting; if the Reconnect Blocking Time value is more than the timeout, the receive will only block up to timeout.

• MessageConsumer.receiveNoWait() – This call will not block if the JMS Connection is in the process of reconnecting. The Reconnect Blocking Time value has no effect on this call.

If these methods eventually reach their respective timeout/wait periods, they all will throw the same Exceptions. as they would reconnect. If a reconnect succeeds while these methods are blocked/called, then these methods will continue returning messages, but with a potentially lowered quality-of-service and with generally similar semantics of receiving messages (like Redelivered messages), as after a recover. The application is notified of this possibility by a Connection ExceptionListener callback with theLostServerException. In addition, for non-AUTO_ACK acknowledge modes, the first acknowledge call after a refresh will throw a LostServerException to notify the application of this possibility.

Asynchronous Consumers

In the context of a reconnect, the behavior for asynchronous consumers will be governed by the setting on the Total Reconnect Period property. The JMS consumer's registered message listener's onMessage()method will continue to be invoked if the reconnect framework is able to successfully re-establish a connection within the Total Reconnect Period setting after a connection failure. If the user explicitly calls a close() on the JMS Connection (or on the JMS Session corresponding to the asynchronous Consumer), then the reconnect framework will not invoke any further onMessages for that Consumer. The onMessage() should expect post recover behavior (like redelivered messages) if the Connection ExceptionListener's onException is invoked with a LostServerException.

Special Cases for Reconnected Consumers

These sections discuss special cases that can occur when consumers are refreshed.

Consumers of Distributed Destinations

Before to WebLogic Server 9.2, consumers of distributed destinations (DDs) were pinned to a particular destination member of the DD for the life of the pinned consumer. This applies to queue consumers of distributed queues, and non-durable subscribers of distributed topics (durable subscribers are not supported distributed topics).

With MessageConsumer reconnect, DD consumers are also refreshed; however, the refreshed consumer is almost never on the same destination member as the stale consumer. Therefore, even though the application is using the same DD consumer across a refresh, it is effectively not pinned to the same destination member across a refresh.

Message-Driven EJBs

Message-driven EJBs (MDBs) are a special sub case of asynchronous consumers that have their own behavior requirements and their own refresh framework. As such, MDBs are not expected to participate in MessageConsumer refreshes, and are not expected to be affected in any other way by the JMS client reconnect framework.

Consumer Connections with a ClientID for Durable Subscriptions

Durable subscriptions on standalone topics will not notice any difference due to the client reconnect feature if the topic is still available across a disconnect. The JMS client reconnect framework implicitly refreshes the durable subscriber on that topic and continue from where it was interrupted. Note that if your Reconnect Policy is set to All, JMS Connections with a ClientID will also refresh automatically, thus allowing durable subscriptions (which are scoped by ClientID) to refresh automatically. Connections with a ClientID set will not reconnect for any other Reconnect Policy setting.

Note:

If a JMS Connection has a ClientID specified at the time of a network/server failure, then reconnecting that client make take significantly longer than your other clients. For example, in a cluster the JMS server must wait for the WebLogic Server "heartbeat" notification that is broadcast from other members of the cluster, as explained in Failover and Replication in a Cluster in Administering Clusters for Oracle WebLogic Server.

WebLogic JMS does not support durable subscriptions on distributed topics, so there is no issue of failover to another distributed topic member during a refresh.

Non Durable Subscriptions and Possible Missed Messages

For consumers that are non-durable subscribers of topics, though the consumption apparently continues successfully across a refresh from an application perspective, it is possible for messages to be published to the topic and dropped (e.g., for lack of consumers) while the reconnect was happening. Missed messages can occur with either synchronous or asynchronous non durable subscribers.

Duplicate Messages

Due to the nature of the consumer refresh feature, there is a possibility of redelivered messages without the client application code calling recover explicitly because a consumer refresh effectively does an implicit equivalent of a recover upon a refresh. This is the main reason why implicit Consumer refresh is not on by default. The semantics of never redelivering a successfully acknowledged message still hold true.

There is also an unlikely case when non-durable subscribers of distributed topics can receive duplicate messages that are not marked redelivered (e.g., when failover happens faster than messages are discarded in topics). This is a consequence of a non-durable subscriber refresh for the distributed topic not being pinned to a topic member across a refresh.

Variations Due to Acknowledge Modes

There will be no difference in the reconnect behaviors of Consumers due to different acknowledge modes. However, the first acknowledge call after a refresh for non-AUTO_ACK modes will throw a LostServerException as described earlier to notify user of potential lowered quality of service.

Reconnecting with Migrated JMS Destinations In a Cluster

Consumers will not always reconnect after a JMS server (and its destinations) is migrated to another server in a cluster. If consumers do not get migrated with the destinations, then either an exception is thrown or onException will occur to inform the application that the consumer is no longer valid. As a workaround, an application can refresh the consumer either in the exception handler or through onException.

Explicitly Disabling Automatic Failover on JMS Clients

If you do not want your JMS clients to be automatically reconnected, then you must explicitly disable this feature either programatically or administratively.

Programmatically

If you do not want your JMS clients to be automatically reconnected, then your applications should call the following code:

   ConnectionFactory cf = (javax.jms.ConnectionFactory)ctx.lookup
                          (JNDI name of connection factory)
   javax.jms.Connection con = cf.createConnection();
   ((weblogic.jms.extensions.WLConnection)con).setReconnectPolicy("none")

For more information about the setReconnectPolicy method, see the weblogic.jms.extension.WLConnection API.

Administratively

Administrators that do not want JMS clients to automatically reconnect should use the following steps to disable the Reconnect Policy on the JMS connection factory:

1. Follow the directions for getting to the JMS Connection Factory: Configuration: Client pages, see Configure connection factory client parameters in the Oracle WebLogic Server Administration Console Online Help.
2. In the Reconnect Policy field, select None to disable the JMS client reconnect feature on this connection factory.

   For more information about the Reconnect Policy field, see JMS Connection Factory: Configuration: Client in the Oracle WebLogic Server Administration Console Online Help.

3. Click Save.

For more information about the other JMS connection factory client parameters, see ClientParamsBean in the MBean Reference for Oracle WebLogic Server.

Best Practices for JMS Clients Using Automatic Failover

Oracle recommends the following best practices for JMS clients when using the Automatic JMS Client Reconnect feature:

• Always Catch exceptions,

• Use Transactions to Group Message Work,

• JMS Clients Should Always Call the close() Method,

Always Catch exceptions

There are some connection failure scenarios where the result of a message operation is undetermined and WebLogic Server throws an exception. Your application must deal with the exception appropriately. See the following:

• Automatic Reconnect Limitations

• Special Cases for Reconnected Sessions

• Special Case for Distributed Destinations

Use Transactions to Group Message Work

Use transacted sessions (JMS) or user transactions (JTA) to group related or dependent work, including messaging work, so that either all of the work is completed or none of it is. If a server instance goes down and a message is lost in the middle of a transaction, the entire transaction is rolled back and the application does not need to make a decision for each message after a failure.

Note:

Be aware of transaction commit failures after a server reconnect, which may occur if the transaction subsystem cannot reach all the participants involved in the transaction.

JMS Clients Should Always Call the close() Method

As a best practice, your applications should not rely on the JVM's garbage collection to clean up JMS connections because the JMS automatic reconnect feature keeps a reference to the JMS connection. Therefore, always use theconnection.close() to clean up your connections. Also consider using a Finally block to ensure that your connection resources are cleaned up. Otherwise, WebLogic Server allocates system resources to keep the connection available.

For more information about closing JMS client connections, see Best Practice: Always Close Failed JMS ClientIDs.

Manually Migrating JMS Data to a New Server

WebLogic JMS uses the migration framework to allow WebLogic JMS to respond properly to migration requests and bring a WebLogic JMS server online and offline in an orderly fashion. This includes both scheduled migrations as well as migrations in response to a WebLogic Server failure.

After a JMS server is properly configured, a JMS server and all of its destinations can migrate to another WebLogic Server within a cluster.

You can manually recover JMS data from a failed WebLogic Server by starting a new server and doing one or more of the tasks in Table 14-2.

Note:

There are special considerations when you migrate a service from a server instance that has crashed or is unavailable to the Administration Server. If the Administration Server cannot reach the previously active host of the service at the time you perform the migration, see Migrating a Service From an Unavailable Server in Administering Clusters for Oracle WebLogic Server.

Table 14-2 Migration Task Guide

If Your JMS Application Uses	Perform the Following Task
Persistent messaging—JDBC Store	If the JDBC database store physically exists on the failed server, then migrate the database to a new server and ensure that the JDBC connection pool URL attribute reflects the appropriate location reference. If the JDBC database does not physically exist on the failed server, access to the database has not been affected , and no changes are required.
Persistent messaging—File Store	If you are using a shared file system, ensure that your file store directories are explicitly configured to reference the shared location (do not depend on the default), otherwise you will need to copy the files to the new server and ensure they have the same directory path as the original server before restarting the migrated file stores. See Using Custom File Stores and File Locations in Administering the WebLogic Persistent Store.
Transactions	To facilitate recovery after a failure , WebLogic Server provides the Transaction Recovery Service, which automatically tries to recover transactions when the system startup. The Transaction Recovery Service owns the transaction log for a server. For detailed instructions about recovering transactions from a failed server, see Transaction Recovery After a Server Fails in Developing JTA Applications for Oracle WebLogic Server.

Note:

JMS persistent stores can increase the amount of memory required during initialization of WebLogic Server as the number of stored messages increases. When rebooting WebLogic Server, if initialization fails due to insufficient memory, then increase the heap size of the Java Virtual Machine (JVM) proportionally to the number of messages that are currently stored in the JMS persistent store and try the reboot again.

See Starting and Stopping Servers: Quick Reference. For information about recovering a failed server, refer to Avoiding and Recovering From Server Failure in Administering Server Startup and Shutdown for Oracle WebLogic Server.

For more information about defining migratable services, see Service Migration in Administering Clusters for Oracle WebLogic Server.