Backing Up and Restoring a Database

The TimesTen Scaleout backup and restore functionality is essential in order to protect your data. It is recommended to perform regular backups in order to minimize the risks of potential data loss. When you perform a backup of a database, TimesTen Scaleout performs the backup asynchronously on each replica set and creates a sub-collection for each replica set that is backed up.

When you are considering backing up and restoring a TimesTen Scaleout database, keep in mind that:

The current grid topology must be the same size or larger than the topology from the grid of the database backup. If your current grid topology is not large enough for n replica sets, TimesTen Scaleout displays an error message. That is, if you backup a database with three replica sets and you want to restore into a database that has only two replica sets, this operation will fail. However, you can use the export and import feature of TimesTen Scaleout to import a database from a grid topology with more replica sets into a database of a grid topology with less replica sets. See Exporting and Importing a Database.
You can restore a backup into a grid of the same grid topology, even the same grid from which the backup was created. That is, if you create a backup of a database where there are three replica sets, then you can restore into the same grid or a new grid where there are three replica sets.
You can restore a backup into a grid that has a larger topology than the grid where the backup was created. If you back up a database that has n replica sets, the restore operation creates a database with exactly n replica sets. However, if your current grid topology is larger than the original grid topology, TimesTen Scaleout creates the additional elements, but TimesTen Scaleout does not add these elements to the distribution map of the database and no data is stored on these elements. Instead, the restore only populates the same number of replica sets as the original grid topology. That is, if you create a backup of a grid where there are three replica sets, you can restore a backup into a new grid where there are four replica sets. However, the restore only populates three of those four replica sets. Thus, in order to populate all replica sets, you must redistribute the data across all replica sets after the restore using the ttGridAdmin dbDistribute command. See Redistributing Data in a Database.
There are two type of backups: normal or staged.
- Normal backups can be performed either on a repository mounted using NFS on each host of your grid or on a repository where each host of your grid uses SSH/SCP to connect to it. The time it takes to create a normal backup varies based on the size of your database, but you should expect every backup to take roughly the same time to complete.
- Staged backups can only be performed on a repository where each host of your grid uses SSH/SCP to connect to it. Even though the first staged backup may take a similar time to complete as a normal backup (or even longer based on the performance of your network), all subsequent staged backups should take only a small fraction of that time to complete. Staged backups are ideal when you want to make regular backups on a second site that is independent to your main site.
See Determining the Size of a Backup or Export for information on the file system space each backup operation requires.

Note:

If the database where the data would be restored is from a version of TimesTen Scaleout that is patch incompatible, such as from a different major release, then you cannot backup and restore a database. Instead, you must export and import that database. See Exporting and Importing a Database.

TimesTen Scaleout enables you to perform the following procedures with backups:

Back Up a Database

Regular backups minimize the risks for potential data loss. Before attempting to back up your database, ensure that you have configured a repository for your grid. See Working with Repositories.

This example creates a backup of the database database1 and stores that backup in the repository repo1. By default, TimesTen Scaleout names your backup with the current date and time, Byyyymmddhhss. The prefix of the backup name, B, stands for backup. Ensure that you run the ttGridAdmin dbBackup command on a management instance.

Note:

You can add the -name parameter to specify a backup name. For example, ttGridAdmin dbBackup database1 -repository repo1 -name mybackup creates a backup named mybackup.

% ttGridAdmin dbBackup database1 -repository repo1
dbBackup B20170222145544 started

Depending on the size of your database, the number of replica sets that your database uses, the performance of your secondary storage device, and the performance of your network the backup time varies. The ttGridAdmin dbBackup command only starts the backup process and the output does not indicate that the backup is complete. Use the ttGridAdmin dbBackupStatus command to see the status of your backup. See Check the Status of a Backup.

For more information about the ttGridAdmin dbBackup command, see Back Up a Database (dbBackup) in Oracle TimesTen In-Memory Database Reference.

Back Up a Database into a Remote Repository (WAN-Friendly)

Normal backups to repositories using the SCP method require two copies of the most recent checkpoint and transaction logs files for each replica set. One copy consists of the checkpoint and log files of one element for each replica set, which are temporarily copied to a directory in the instance home. The second copy consists of the same checkpoint and log files per replica set after they are sent and stored at the repository, which construct the backup itself.

TimesTen Scaleout enables you to create staged backups to SCP repositories. This type of backup eliminates the overhead of creating local copies of the checkpoint and log files and reduces the network traffic required to create a remote copy in the repository. To accomplish this, staged backups use symbolic links instead of temporary local files (with the exception of the latest log file) and maintain a staging directory on the repository with the checkpoint and log files per replica set used for the latest backup. The next staged backup will copy the latest log files from each replica set and synchronize the rest of the files in the staging directory over the network. Finally, the repository uses the resulting files in the staging directory to create the backup, which removes the load of that task from the data instances and network.

Note:

The system hosting the repository makes use of the Linux cp and rsync commands and the TimesTen ttTransferAgent utility to perform staged backup operations. The ttTransferAgent utility is copied to the staging directory at the beginning of a staged backup if it is not already available from a previous staged backup.
See Working with Repositories for more information on SCP repositories.

The next topics describe the recommended settings for staged backups and how to create a staged backup:

Prerequisites

Staged backups have these prerequisites:

Passwordless SSH access: Staged backups require that all hosts with instances (data and management) have passwordless SSH access for the instance administrator to the system hosting the repository. See Setting Passwordless SSH.
The rsync command: Staged backups require that the rsync command is available on hosts with data instances and on the system allocating the repository.

SSH Configuration File

Staged backups depend on SSH for data transport and control. On every host with a data instance, consider updating the SSH configuration file for the instance administrator (/home/instance_administrator/.ssh/config) or the global SSH configuration file (/etc/ssh/ssh_config) to improve the reliability of staged backups. These options may prove useful:

HostName: You can use this option to specify multiple aliases for the repository. SSH tries them in order. Provide a list of multiple aliases in a different order to every host.
Port: SSH uses by default port 22. You may need to use a different port number if SSH has to pass through a NAT gateway.
BindAddress or BindInterface: You can use these options to control which Ethernet interface SSH will use to contact the repository.
ConnectionAttempts: By default SSH only makes one connection attempt. You can use this option to set how many connection attempts SSH will make before terminating and returning a failure notification.
ConnectTimeout: By default SSH uses the system TCP timeout. You can use this option to set the timeout (in seconds) to establish a SSH connection. Consider increasing this connection timeout on high-latency WAN links.
ProxyJump: You can use this option to set bastion hosts to serve as proxies to connect to the repository. The hosts with a data instance may be able to access the bastion hosts but not other hosts, like the repository. Likewise, the bastion hosts may be able to access the remote repository. You can configure multiple bastion hosts for high availability.
ServerAliveCountMax: You can use this option to set the maximum number of keep-alive messages sent through the encrypted channel by a host without receiving any message back from the repository. The connection is terminated after reaching this threshold. You must use this option in conjunction with the ServerAliveInterval option.
ServerAliveInterval: You can use this option to set the time (in seconds) between receiving no data from the repository and the host sending a keep-alive message. This serves to detect if the repository has crashed or the network has gone down.

On the system hosting the repository, consider setting this option in the global SSH daemon configuration file (/etc/ssh/sshd_config):

MaxStartups: You can use this option the set the maximum number of concurrent unauthenticated connections to the SSH daemon. Consider setting the start parameter to a value larger than the number of replica sets and the full parameter to ten times the value of the start parameter. For example, if you have ten replica sets, set this option as:
```
MaxStartups 15:30:150
```

BackupFailThreshold Attribute

The BackupFailThreshold first connection attribute determines the number of transactions log files that can accumulate in the LogDir directory since the start of a backup before TimesTen is forced to release the hold on checkpoint operations. If a checkpoint is initiated before the completion of a backup, the backup is invalidated.

Set the BackupFailThreshold attribute to a value that is high enough to ensure the safe completion of your backup. For example, if a backup typically takes n seconds to complete and your database creates m transaction log files per second, then set BackupFailthreshold to a value greater than n * m. The number of log files generated by your database per any given unit of time is directly proportional to your write workload and inversely proportional to the value set for the LogFileSize attribute.

See Modifying the Connection Attributes of a Database for information on how to modify the value of a first connection attribute.

For more information on the BackupFailThreshold, LogDir, and LogFileSize connection attributes, see Connection Attributes in Oracle TimesTen In-Memory Database Reference.

File System Space

To avoid out-of-space failures due to staged backups, ensure that:

The file system used by each data instance has enough space for one transaction log file (LogFileSize) in the instance home plus enough space to store in LogDir all the transaction log files that may be generated while the backup operation is in progress (BackupFailThreshold * LogFileSize).
The file system used by the repository has enough space to store as many backups you wish to retain plus enough space in the staging directory for 1.25 backups for all staged backups of the same database.

See Determining the Size of a Backup or Export for information on the file system space each backup operation requires.

For more information on the LogFileSize, LogDir, and BackupFailThreshold connection attributes, see Connection Attributes in Oracle TimesTen In-Memory Database Reference.

WAN Throughput

The minimum WAN throughput required by a subsequent staged backup depends on the aggregate size of the database, the desired time for the backup to take, and the speed-up factor provided by the staging repository and the defined transfer compression. You will need to perform a series of staged backups to test how much the performance of your network and overall setup (plus the inherent advantages subsequent staged backups over normal backups provide) reduces the backup time for regular staged backups under typical workload conditions. Consider this formula:.

Minimum WAN throughput = file size of a backup/(desired backup time * (first staged backup time/average subsequent staged backups time))

See Determining the Size of a Backup or Export for information on the file system space each backup operation requires.

Create a Staged Backup

This example creates a staged backup named stgbackup1 of the database1 database and stores that backup in the scprepo1 repository. The staged backup is set to use an aggregate network traffic of 62 MB per second and a compression level of 9 for that network traffic.

% ttGridAdmin dbBackup database1 -repository scprepo1 -name stgbackup1 -backupType staged -bwlimit 62 -compression 9
dbBackup stgbackup1 started

Note:

Ensure that you run the ttGridAdmin dbBackup command as the instance administrator on the active management instance.

For more information about the ttGridAdmin dbBackup command, see Back Up a Database (dbBackup) in Oracle TimesTen In-Memory Database Reference.

Check the Status of a Backup

The ttGridAdmin dbBackupStatus command enables you to view the progress of all backup processes for a specific database.

This example displays the status of all backup processes for the database database1.

% ttGridAdmin dbBackupStatus database1
Database  Backup          Repository Host  Instance  Elem State     Started       Finished
--------- --------------- ---------- ----- --------- ---- --------- ------------- --------
database1 B20170222145544 repo1                           Completed 2017-02-22... Y
                                     host3 instance1    1 Complete
                                     host6 instance1    3 Complete

Ensure that the ttGridAdmin dbBackupStatus output shows that the overall state of the backup process is marked as Completed. In case that you see a state value of Failed, perform these tasks:

Use the ttGridAdmin dbStatusdatabase1 -details command to ensure that the host and instance of that element are up and running. If at least one host from each replica set is up, the ttGridAdmin dbBackup command can create a full backup of your database. See Monitor the Status of a Database (dbStatus) in Oracle TimesTen In-Memory Database Reference.
Ensure that the repository where you are attempting to create the backup has enough free file system space to create a backup of your database.

If the backup failed, you may attempt to perform another backup using a different backup name. If a new backup name does not perform a successful backup, diagnose the issue and perform any necessary fixes. After you have resolved the problem, use the ttGridAdmin dbBackupDelete to delete any failed backups. TimesTen Scaleout does not automatically delete a failed backup. Then, use the ttGridAdmin dbBackup command to start a new backup. Depending on your available file system space, you can use these commands in any order. See Delete a Backup and Back Up a Database.

For more information about the ttGridAdmin dbBackupStatus command, see Display the Status of a Database Backup (dbBackupStatus) in Oracle TimesTen In-Memory Database Reference.

Delete a Backup

TimesTen Scaleout does not automatically delete backups. In some cases, you may want to delete backups that have failed or old backups to free up file system space.

Use the ttGridAdmin repositoryList -contents command to view all of your available backups and their respective repositories. See List Repositories and Collections.

This example deletes the backup named B20170222145544 from repository repo1.

% ttGridAdmin dbBackupDelete -repository repo1 -name B20170222145544
Backup B20170222145544 deleted

TimesTen Scaleout deletes the collection and all of the sub-collections that are part of the backup.

For more information about the ttGridAdmin dbBackupDelete command, see Delete a Database Backup (dbBackupDelete) in Oracle TimesTen In-Memory Database Reference.

Restore a Database

Before attempting to restore a database, consider the following:

The database definition name must not be in use by other databases when you attempt to perform a database restore. For example, you cannot name the restored database database1 if another database is using the database1 name.
The database definition of the backed up database does not need to match the database name of the database that you are restoring. For example, you can restore a backup of the payroll database to the new_payroll database definition.
The K-safety value of the database that you backed up does not need to match the K-safety value of the restore database.
The database definition must have at least as many connections as the database definition of the backed up database.

This example restores the database res_db1 from the backup B20170222145544 from repository repo1. Ensure that you run the ttGridAdmin dbRestore command on the management instance.

% ttGridAdmin dbRestore res_db1 -repository repo1 -name B20170222145544
dbRestore B20170222145544 started

Note:

Ensure that the res_db1 database definition exists before attempting to perform a restore. You do not need to create a database from this database definition. See Create a Database Definition.

Depending on the size of your backup, the number of replica sets that your database uses, the performance of your secondary storage device, and the performance of your network the restore time varies. The ttGridAdmin dbRestore command only starts the restore process and the output does not indicate that the restore is complete. The restore process is performed asynchronously on every element. Use the ttGridAdmin dbRestoreStatus command to see the status of your restore. See Check the Status of a Restore.

For more information about the ttGridAdmin dbRestore command, see Restore a Database (dbRestore) in Oracle TimesTen In-Memory Database Reference.

Check the Status of a Restore

The ttGridAdmin dbRestoreStatus command enables you to view the progress of the restore process for a specific database.

This example displays the status of all restore processes for the database res_db1.

% ttGridAdmin dbRestoreStatus res_db1
Database Restore Repository Host  Instance  Elem State                     Started       Finished
-------- ------- ---------- ----- --------- ---- ------------------------- ------------- –-------
res_db1  mybkup  repo1                           Restore_Finale_Complete   2017-03-03... Y
                            host3 instance1      Restore_Instance_Complete
                            host4 instance1      Restore_Instance_Complete
                            host5 instance1      Restore_Instance_Complete
                            host6 instance1      Restore_Instance_Complete
                            host7 instance1      Restore_Instance_Complete
                            host8 instance1      Restore_Finale_Complete

Ensure that the ttGridAdmin dbRestoreStatus output shows that the restore has been completed for every element of your grid. The restore operation is fully completed when the State column of the row with the database name is marked as Completed.

Ensure that the ttGridAdmin dbRestoreStatus output shows that the overall state of the restore process is marked as Completed. In case that you see a state value of Failed or Restore_Instance_Failed for an element or an overall state of Restore_Finale_Failed or Restore_Init_Failed, stop the database with ttGridAdmin dbClose and ttGridAdmin dbUnload commands. Once you have stopped the database, use the ttGridAdmin dbDestroy command to delete the database that did not restore successfully. Then, attempt the restore operation again. See Unloading a Database from Memory and Destroying a Database.

For more information about the ttGridAdmin dbRestoreStatus command, see Display the Status of a Database Restore (dbRestoreStatus) in Oracle TimesTen In-Memory Database Reference.

Set Cache Credentials

After you restore a database backup that has cache groups, you must set the Oracle cache administration user name and password for the database with the ttGridAdmin dbCacheCredentialSet command.

Only after setting the cache credentials can you can redistribute the data to all replica sets with the ttGridAdmin dbDistribute command. If you redistribute the data before setting the cache credentials, then you will be unable to set the cache credentials for your cache groups.

The following examples sets the cache administration user name and password in the restored database. After which, requests a redistribution of data.

% ttGridAdmin dbCacheCredentialSet res_db1
Provide Oracle user id: cacheadmin
Provide Oracle password: orapwd
Configuring cache.....................................................OK

% ttGridAdmin dbDistribute res_db1 -apply

See Register the Cache Administration User Name and Password in the TimesTen Database.