Recovering Transactions

There are some situations where the commit or rollback operations might be interrupted, typically because the server crashed or a resource manager crashed. Crash situations can leave some transactions stranded between steps. GlassFish Server is designed to recover from these failures. If the failed transaction spans multiple servers, the server that started the transaction can contact the other servers to get the outcome of the transaction. If the other servers are unreachable, the transaction uses heuristic decision information to determine the outcome.

The following topics are addressed here:

• Automatic Transaction Recovery

• To Manually Recover Transactions

• Distributed Transaction Recovery

• Recovery Workarounds and Limitations

Automatic Transaction Recovery

GlassFish Server can perform automatic recovery in these ways:

• Pending transactions are completed upon server startup if automatic-recovery is set to true.

• Periodic automatic recovery is performed by a background thread if the pending-txn-cleanup-interval property is set to a positive value.

Changing these settings requires a server restart. For more information about how to change these settings, see Configuring the Transaction Service.

If commit fails during recovery, a message is written to the server log.

To Manually Recover Transactions

Use the recover-transactions subcommand in remote mode to manually recover transactions that were pending when a resource or a server instance failed.

For a standalone server, do not use manual transaction recovery to recover transactions after a server failure. For a standalone server, manual transaction recovery can recover transactions only when a resource fails, but the server is still running. If a standalone server fails, only the full startup recovery process can recover transactions that were pending when the server failed.

For an installation of multiple server instances, you can use manual transaction recovery from a surviving server instance to recover transactions after a server failure. For manual transaction recovery to work properly, transaction logs must be stored on a shared file system that is accessible to all server instances. See Transaction Logging.

When you execute recover-transactions in non-delegated mode, you can recover transactions that didn't complete two-phase commit because of a resource crash. To use manual transaction recovery in this way, the following conditions must be met:

• The recover-transactions command should be executed after the resource is restarted.

• Connection validation should be enabled so the connection pool is refreshed when the resource is accessed after the recovery. For more information, see Connection Validation Settings in Oracle GlassFish Server 3.1 Performance Tuning Guide.

If commit fails during recovery, a message is written to the server log.

1. Ensure that the server is running.

   Remote subcommands require a running server.

2. Manually recover transactions by using the recover-transactions(1) subcommand.

Example 19-4 Manually Recovering Transactions

This example performs manual recovery of transactions on instance1, saving them to instance2.

asadmin recover-transactions --target instance2 instance1
Transaction recovered.

See Also

You can also view the full syntax and options of the subcommand by typing asadmin help recover-transactions at the command line.

Distributed Transaction Recovery

To enable cluster-wide automatic recovery, you must first facilitate storing of transaction logs in a shared file system. See Transaction Logging.

Next, you must set the transaction service's delegated-recovery property to true (the default is false). For information about setting tx-log-dir and delegated-recovery, see Configuring the Transaction Service.

Recovery Workarounds and Limitations

The GlassFish Server provides workarounds for some known issues with transaction recovery implementations.

General Recovery Limitations

The following general limitations apply to transaction recovery:

• Recovery succeeds if there are no exceptions during the process. This is independent of the number of transactions that need to be recovered.

• Only transactions that did not complete the two-phase commit can be recovered (one of the XA resources failed or GlassFish Server crashed after resources were prepared).

• Manual transaction recovery cannot recover transactions after a server crash on a standalone server instance. Manual operations are intended for cases when a resource dies unexpectedly while the server is running. In case of a server crash, only startup recovery can recover in-doubt transactions.

• It is not possible to list transaction IDs for in-doubt transactions.

• Delegated transaction recovery (by a different server instance in a cluster) is not possible if the failed instance used an EMBEDDED Message Queue broker, or if it used a LOCAL or REMOTE Message Queue broker and the broker also failed. In this case, only automatic recovery on server instance restart is possible. This is because for conventional Message Queue clustering, state information in a failed broker is not available until the broker restarts.

Oracle Setup for Transaction Recovery

You must configure the following grant statements in your Oracle database to set up transaction recovery:

grant select on SYS.DBA_PENDING_TRANSACTIONS to user; 
grant execute on SYS.DBMS_SYSTEM to user; 
grant select on SYS.PENDING_TRANS$ to user; 
grant select on SYS.DBA_2PC_NEIGHBORS to user; 
grant execute on SYS.DBMS_XA to user;
grant select on SYS.DBA_2PC_PENDING to user;

The user is the database administrator. On some versions of the Oracle driver the last grant execute fails. You can ignore this.

Oracle Thin Driver

In the Oracle thin driver, the XAResource.recover method repeatedly returns the same set of in-doubt Xids regardless of the input flag. According to the XA specifications, the Transaction Manager initially calls this method with TMSTARTSCAN and then with TMNOFLAGS repeatedly until no Xids are returned. The XAResource.commit method also has some issues.

To disable the GlassFish Server workaround, set the oracle-xa-recovery-workaround property value to false. For details about how to set this property, see Configuring the Transaction Service. This workaround is used unless explicitly disabled.

Delegated Recovery After Server Crash Doesn't Work on MySQL

The MySQL database supports XA transaction recovery only when the database crashes. When a GlassFish Server instance crashes, MySQL rolls back prepared transactions.

Call to XATeminator.recover() During ResourceAdapter.start() Hangs If Automatic Recovery Is Enabled

Calls to XATerminator.recover() from the ResourceAdapter.start() method never return because GlassFish Server deadlocks. This only occurs when automatic recovery is enabled.

It is not advisable to do transactional activities, such as starting a transaction or calling XATerminator.recover(), during ResourceAdapter.start(). For more information, see http://markmail.org/message/ogc7qndhaywfkdrp#query:+page:1+mid:kyyzpcexusbnv7ri+state:results.