1. Replication Guide
  2. Managing Database Failover and Recovery
  3. Recovering a Failed Database

Recovering a Failed Database

If the databases are configured in a bidirectional replication scheme, a failed master database is automatically brought up to date from the subscriber. Automatic catch-up also applies to recovery of master databases in active standby pairs.

See Automatic Catch-Up of a Failed Master Database.

If a restarted database cannot be recovered from its master's transaction log so that it is consistent with the other databases in the replicated system, you must re-create the database from one of its replication peers. Use command line utilities or the TimesTen Utility C functions. See Recovering a Failed Database From the Command Line and Recovering a Failed Database From a C Program.

Note:

It is not necessary to re-create the DSN for the failed database.

In the event of a subscriber failure, if any tables are configured with a return service, commits on those tables in the master database are blocked until the return service timeout period expires. To avoid this, you can establish a return service failure and recovery policy in your replication scheme, as described in Establishing Return Service Failure and Recovery Policies. If you are using the RETURN RECEIPT service, an alternative is to use ALTER REPLICATION and set the NO RETURN attribute to disable return receipt until the subscriber is restored and caught up. Then you can submit another ALTER REPLICATION statement to reestablish RETURN RECEIPT.

Recovering a Failed Database From the Command Line

If the databases are fully replicated, you can use the ttDestroy utility to remove the failed database from memory and ttRepAdmin -duplicate to re-create it from a surviving database.

If the database contains any cache groups, you must also use the -keepCG option of ttRepAdmin. See Duplicating a Database.

To recover a failed database, subscriberds, from a master named masterds on host system1, enter: 

ttDestroy /tmp/subscriberds

ttRepAdmin -dsn subscriberds -duplicate -from masterds -host "system1" -uid ttuser

You are prompted for the password of ttuser.

Note:

ttRepAdmin -duplicate is supported only between identical and patch TimesTen releases. The major and minor release numbers must be the same.

After re-creating the database with ttRepAdmin -duplicate, the first connection to the database reloads it into memory. To improve performance when duplicating large databases, you can avoid the reload step by using the ttRepAdmin -ramload option to keep the database in memory after the duplicate operation.

To recover a failed database, subscriberds, from a master named masterds on host system1, and to keep the database in memory and restart replication after the duplicate operation, enter: 

ttDestroy /tmp/subscriberds

ttRepAdmin -dsn subscriberds -duplicate -ramload -from masterds -host "system1" -uid ttuser

You are prompted for the password of ttuser.

Note:

After duplicating a database with the ttRepAdmin -duplicate -ramLoad options, the RAM Policy for the database is manual until explicitly reset by ttAdmin -ramPolicy or the ttRamPolicy function.

Recovering a Failed Database From a C Program

You can use the C functions provided in the TimesTen utility library to recover a failed database programmatically.

If the databases are fully replicated, you can use ttDestroyDataStore function to remove the failed database and the ttRepDuplicateEx function to re-create it from a surviving database.

To recover and start a failed database, named subscriberds on host system2, from a master, named masterds on host system1, enter: 

int          rc;
ttutilhandle utilhandle;
ttrepduplicateexarg arg;
memset( &arg, 0, sizeof( arg ) );
arg.size = sizeof( ttrepduplicateexarg );
arg.flags = tt_repdup_repstart | tt_repdup_ramload;
arg.uid=ttuser;
arg.pwd=ttuser;
arg.localhost = "system2";
rc = ttdestroydatastore( utilhandle, "subscriberds", 30 );
rc = ttrepduplicateex( utilhandle, "dsn=subscriberds",
                      "masterds", "system1", &arg );

In this example, the timeout for the ttDestroyDataStore operation is 30 seconds. The last parameter of the ttRepDuplicateEx function is an argument structure containing two flags:

  • TT_REPDUP_RESTART to set the subscriberds database to the start state after the duplicate operation is completed

  • TT_REPDUP_RAMLOAD to set the RAM policy to manual and keep the database in memory

Note:

When the TT_REPDUP_RAMLOAD flag is used with ttRepDuplicateEx, the RAM policy for the duplicate database is manual until explicitly reset by the ttRamPolicy function or ttAdmin -ramPolicy.

See TimesTen Utility API in Oracle TimesTen In-Memory Database C Developer's Guide for the complete list of the functions provided in the TimesTen C language utility library.