18 Backup and Restore Operations

Backups provide the ability to restore Oracle Key Vault to a previous state in the case of a failure.

18.1 About Backing Up and Restoring Data in Oracle Key Vault

You can use Oracle Key Vault to back up and restore Oracle Key Vault data.

You should back up data periodically to reduce downtime and recover from unexpected data losses and system failures. You can restore a new or existing Oracle Key Vault server from a backup.

You can perform backup and restore operations from the Oracle Key Vault management console or by using the Oracle Key Vault RESTful services commands. You must be a user who has the System Administrator role to back up and restore Oracle Key Vault data. You can schedule backups at periodic intervals to run automatically at designated times. You also can run these operations on-demand to save a current snapshot of the system.

Oracle strongly recommends that you back up Oracle Key Vault data regularly on a schedule. This practice ensures that backups are current and hold the most recent data. You can use this backup to restore a new or existing Oracle Key Vault server and be fully operational with minimum data loss.

Oracle Key Vault encrypts all backed up data. When you use a remote destination, this data is copied using the secure copy protocol (SCP) or the secure file transfer protocol (SFTP). You must therefore ensure that either SCP or SFTP is supported at the remote backup destination.

In an Oracle Key Vault multi-master cluster environment, the replication intrinsically creates copies of data on other nodes in the cluster. However, you can still perform backups and backup-related operations on all individual Oracle Key Vault cluster nodes. Be aware that backups can still only be restored to standalone Oracle Key Vault servers. Therefore, backups in a cluster are taken for disaster recovery in case of a complete cluster failure and should all be on remote destinations.

18.2 Oracle Key Vault Backup Destinations

A backup destination is the location where Oracle Key Vault data will be copied to and stored.

18.2.1 About the Oracle Key Vault Backup Destination

The backup destination enables the backup data to be available on Oracle Key Vault itself or on another server.

This ensures that you have all the relevant data to recover in case of a catastrophic failure with the Oracle Key Vault server or hardware.

The backup destination is usually another server or computer system that you have access to. You can add, delete, and modify a backup destination.

The backup operation copies Oracle Key Vault data to a backup destination of your choice. The backup destination stores the data until it is needed.

Oracle Key Vault provides two types of backup destinations: local and remote. The local backup destination resides on the Oracle Key Vault server itself, the remote one resides externally in a different server or computer system. You can create more than one backup destination for greater availability.

Local and remote backup destinations have the following characteristics:

  • Local backup destination: The local backup destination, LOCAL, is present by default and cannot be removed. Backups to the local backup destination are local backups.

    Backups to LOCAL are useful to save a current state of Oracle Key Vault. Because these backups are stored on disk, they could be lost in the case of hardware or other catastrophic failure. They will also not be available after a failover or switchover in a primary-standby configuration. You cannot restore the backups to a primary-standby without first unpairing the primary from the standby, nor can you restore the backups to a cluster configuration. Therefore, you should back up the data to a remote destination when using these configurations.

    A LOCAL destination can store only the last full backup and the cumulative incremental backups after that full backup. After a new full backup of the periodic backup to LOCAL completes, the previous periodic full or cumulative incremental backups are deleted.

  • Remote backup destinations: Remote backup destinations reside on external servers and can be dispersed geographically for disaster recovery purposes. Backups to remote backup destinations are remote backups.

    Each backup destination on the external server is associated with a backup catalog file called okvbackup.mgr that Oracle Key Vault maintains at the backup destination. The okvbackup.mgr file catalogs the backups performed and is used to restore data.

    Note:

    You cannot use another Oracle Key Vault server as a remote backup destination.

Caution:

  • Oracle Key Vault may not be able to find the backups if you delete or modify the backup catalog file. Therefore, do not delete or modify this file.

  • Do not configure the same remote backup destination directory for different Oracle Key Vault servers as backup destinations, because backups that happen concurrently from different Oracle Key Vault servers will overwrite each other's catalog file, with the result that Oracle Key Vault may not be able to locate the backups correctly.

  • After you restore a backup that contains a remote backup destination, do not continue to use that remote backup destination. Delete any backup jobs that are configured to send backups to that destination. Continuing to use this backup destination could corrupt the backup catalog file. Oracle Key Vault may not be able to locate backups correctly.

  • Configure each node in a multi-master cluster to send their backups to a different backup destination.

18.2.2 Creating a Remote Backup Destination

You can use the Oracle Key Vault management console to create a remote backup destination.

To create a remote backup destination, you must provide a user account, a unique existing directory location on an external server, and an authentication method (password or key-based). Oracle Key Vault needs this information to make a secure connection with the remote server.
  1. Log in to the Oracle Key Vault management console as a user with the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. In the System Backup page, select Manage Backup Destinations.
    The Manage Backup Destinations window appears.Description of 21_manage_backup_dest.png follows
    Description of the illustration 21_manage_backup_dest.png
  5. Click Create.

    The Create Backup Destination window appears.

    Description of 21_create_backup_dest.png follows
    Description of the illustration 21_create_backup_dest.png

  6. Enter the following information for the backup location:
    • Destination Name: Enter a descriptive name to identify the backup destination.

    • Transfer Method: Choose between scp and sftp to copy files to the remote destination.

    • Hostname: Enter the host name or IP address of the remote server for the backup destination. If you enter the host name, then ensure that DNS is configured to translate the host name to its corresponding IP address. Do not include spaces, single quotation marks, or double quotation marks in a host name that is in a remote backup destination.

    • Port: Enter the SCP or SFTP port number on the external server. The default is 22.

    • Destination Path: Enter the path to an existing directory on the external server where the backup file will be copied. You cannot modify this directory location after the backup destination is created. This path must not be the destination for backups from another Oracle Key Vault server. Do not include spaces, single quotation marks, or double quotation marks destination path that is in a remote backup destination.

    • Username Enter the user name of the user account that can be used to establish an SCP or SFTP connection to the remote server. Ensure that this user has write permissions on the directory that is specified in Destination Path. Do not include spaces, single quotation marks, or double quotation marks in a user name that is in a remote backup destination.

    • Authentication Method: Choose one of the following:

      • Key-based Authentication: Copy the public key that appears and paste it in the appropriate configuration file, such as authorized_keys, on the destination server. Be aware that certain events may trigger a change of the public key, which means that Oracle Key Vault cannot use the backup destination until the new public key is re-copied from Oracle Key Vault to the appropriate configuration file. These events include but are not limited to certificate rotation, changing the IP address, and conversion to a cluster node.

      • Password Authentication: The password of the user account entered in the Username field.

  7. Click Save.
Oracle Key Vault validates the input that you supplied to create the backup destination by creating empty test files under both /tmp and the directory that you supplied in the Destination Path field. If the validation fails, then the backup destination is not created. If this happens, then check values for the user account on the remote server (user name and password or key) and ensure that the directory has write permissions for the user. Finally, ensure that the remote server is active.

18.2.3 Changing Settings on a Remote Backup Destination

After you have created the backup destination, you can change the SCP or SFTP port number and details of the user account.

You cannot change any other setting.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Select Manage Backup Destinations.

    The Manage Backup Destinations page appears displaying LOCAL and remote backup destinations.

  5. Click a remote backup destination name to edit the backup destination.

    The Edit Backup Destination page appears.

  6. Modify the following information:
    • Port: Change the SCP or SFTP port number on the external server.

    • Destination Public Key: This field shows the public key information that Oracle Key Vault currently has stored in its known_hosts file for the remote server. If the remote server's public key changes, then Oracle Key Vault cannot copy backups to or from the remote server. In order to store the new public key in the known_hosts file, click the Reset Dest Public Key button, which will retrieve and save the new public key from the remote server. After you click Reset Dest Public Key, verify that the correct public key was saved.

    • Username: Enter the user name of the user account that can be used to establish an SCP or SFTP connection to the remote server. Ensure that the new user has write permissions on the directory that is specified in Destination Path, because this path cannot be changed.

    • Authentication Method: Choose one of the following:

      • Password Authentication: The password of the user account entered in the Username field.

      • Key-based Authentication: If the Oracle Key Vault public key has changed, re-copy the public key that appears in the Public Key field and then paste it in the appropriate configuration file, such as authorized_keys, on the destination server. Be aware that certain events may trigger a change of the public key, which means that Oracle Key Vault cannot use the backup destination until the new public key is re-copied from Oracle Key Vault to the appropriate configuration file. These events include but are not limited to certificate rotation, changing the IP address, and conversion to a cluster node.

  7. Click Save.
Oracle Key Vault validates the input that you supplied to update the backup destination. If the validation fails, then the backup destination is not updated. If this happens, then check values for the user account on the remote server (user name and password) and ensure that the directory has write permissions for the user. Finally, ensure that the remote server is active.

18.2.4 Deleting a Remote Backup Destination

You can delete a remote backup destination (but not the local destination) to stop future backups to that destination server.

Deleting a remote backup destination from Oracle Key Vault does not remove the backups on the destination.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Select Manage Backup Destinations.
  5. In the Manage Backup Destinations page, check the boxes for the backup destinations that you want to delete.
  6. Click Delete.

18.3 Scheduled Backups and States

Oracle Key Vault provides scheduled backup types depending on the backup destination, and different states that indicate the progress of the backup activity.

18.3.1 About Schedule Backup Types and States

You can schedule backups in Oracle Key Vault for specific times and backup destinations.

The backup process starts at the scheduled time and generates a system backup, which is a file that is stored on the backup destination. There is one backup file for each completed backup.

No backup can start if another backup is in progress. You can change the schedule of backups as needs change. You can continue working with Oracle Key Vault while the backup is in progress.

A system restart will terminate any ongoing backup. If you must restart the system during the time a backup is scheduled to occur, then you can pause the backup and resume the backup after the system restarts.

18.3.2 Types of Oracle Key Vault Backups

Oracle Key Vault provides two types of backup jobs that can be scheduled: one-time backups and periodic backups.

  • One-time backup: A one-time backup makes a full backup of the Oracle Key Vault system. You can schedule multiple one-time backup jobs, each with its own start time.

    You should make one-time local backups before making significant configuration changes to Oracle Key Vault, in case you need to recover from configuration failures.

    LOCAL destinations can only store the last one-time backup. When a one-time backup to LOCAL completes, the previous backup is deleted.

  • Periodic backup: After you schedule a periodic backup, Oracle Key Vault takes a full backup at the designated start time and then puts the backup schedule in the ACTIVE state. After the backup period passes, another backup starts. If it has been less than 7 days since the last full backup, then the next backup will be a cumulative incremental backup, which holds changes since the last full backup. If it has been more than 7 days since the last full backup, then the next backup will be a full backup.

    For example, if the backup period is one day, then every seventh one is a full backup. If the backup period is 8 days, then all backups are full backups. If the backup period is 12 hours, then there are 13 cumulative backups before a full backup.

    You should schedule periodic backups with a period of 1 day or less to minimize data loss.

    A LOCAL destination can store only the last full backup and the cumulative incremental backups after that full backup. After the periodic backup schedule takes a new full backup to LOCAL, previous periodic full and cumulative backups in LOCAL are deleted.

    Cumulative incremental backups are faster than full backups. Only one periodic backup can be scheduled at any time.

18.3.3 Scheduled Backup States in Oracle Key Vault

Scheduled backups have four states, which indicate whether the backup is scheduled, in progress, completed, or paused.

  • ACTIVE: The backup is scheduled and will start at the next start time. (The start time is indicated in the Start Time column on the Scheduled Backups page.)
  • PAUSED: All future backups are on hold and will not start even if the start time has passed. They will start when they are explicitly resumed. You can change the state from active to paused and back. Put a scheduled backup in the paused state for these situations:
    • When communication between Oracle Key Vault and the remote destination is broken
    • If the remote destination is unavailable
    • If you want to defer the backup
  • ONGOING: The backup is in progress.
  • DONE: The backup is complete.

18.4 Scheduling and Managing Oracle Key Vault Backups

You can schedule Oracle Key Vault backups to specific backup destinations and times.

You must create the backup destinations that you will use beforehand, and you can modify or delete backup schedules.

18.4.1 Scheduling a Backup for Oracle Key Vault

You can schedule a one-time or a periodic backup to a local or remote backup destination.

You can start a one-time backup to start immediately without setting a time. However, do not schedule backup operations if a certificate rotation or cluster operation is in progress.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. In the Backup page, click Backup.

    The Backup page appears. The following image shows how it appears when PERIODIC is selected.

    Description of 21_backup.png follows
    Description of the illustration 21_backup.png

  5. In the Name field, enter a name for the backup.
  6. If you want to perform a periodic backup, then do the following:
    1. In the Start Time field, use the Calendar icon to specify the start time for the backup. If you want the first backup to perform immediately after you click Schedule, then select Now.
    2. For Type, select PERIODIC.
      The additional fields Days, Hours, and Mins appear.
    3. In the Days, Hours, and Mins fields, specify the interval at which periodic backups will occur.
    4. For Destination, select the backup destination.
    5. Click Schedule to add the scheduled backup to the Scheduled Backup(s) page.
  7. If you want to perform a one-time backup, then do the following:
    1. In the Start Time field, use the Calendar icon to specify the start time for the backup. If you want the backup to perform immediately after you click Schedule, then select Now.
    2. For Type, select ONE-TIME.
    3. For Destination, select the destination backup from the list.
    4. Click Schedule to add the scheduled backup to the Scheduled Backup(s) page.

18.4.2 Changing a Backup Schedule for Oracle Key Vault

You cannot change the schedule of a backup in progress.

To change the backup schedule the state must be active or paused.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Click the name of the scheduled backup in the Scheduled Backup(s) page.
  5. Enter the Start Time or manually enter or click the calender icon to choose the Start Time of the backup schedule.

    You can only change the scheduled start time if it has not already passed. This means that the state cannot be ongoing or done. For a periodic backup you can change the start time if the scheduled start time has not passed.

  6. If you are changing a periodic backup schedule, then in the Days, Hours, and Mins fields, specify the interval at which periodic backups will occur.
  7. Select Save.

18.4.3 Deleting a Backup Schedule from Oracle Key Vault

You can delete a backup schedule from the Oracle Key Vault management console.

  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Check the boxes of scheduled backups listed in the Scheduled Backup(s) page.
  5. Click Delete to delete the selected backup schedules.

18.4.4 How Primary-Standby Affects Oracle Key Vault Backups

In a primary-standby deployment, you must perform backups on the primary server.

Because the standby synchronizes its state with the primary, you do not need to back up the standby.

Be aware of the following behavior for failover or switchover operations in a primary-standby deployment:

  • Any backups in progress will terminate if there is a failover or a primary-standby switchover. Backups to LOCAL are private to the Oracle Key Vault server and therefore the local backup on the primary server is not available after a failover or switchover.

  • Backups scheduled with password authentication start as usual after the failover or switchover.

  • Because backups can only be restored to standalone servers, you must unpair primary-standby deployments before you can perform a backup restore operation on the former primary.

18.4.5 How Using a Cluster Affects Oracle Key Vault Backups

In a multi-master cluster environment, be aware of how the backup process works in individual nodes and the entire cluster.

  • In an Oracle Key Vault multi-master cluster environment, the replication intrinsically creates copies of data on other nodes in the cluster. However, you can still perform backups and backup-related operations on all individual Oracle Key Vault cluster nodes.
  • Backups in a cluster are taken for disaster recovery in case of a complete cluster failure and should all be done on remote destinations.
  • Backups can still only be restored to standalone Oracle Key Vault servers. Because a cluster node cannot be switched back to a standalone server, only remote backups should be taken.

18.4.6 Protecting the Backup Using the Recovery Passphrase

Oracle Key Vault uses the recovery passphrase to control who can restore system backups.

To restore a backup, use the Oracle Key Vault recovery passphrase from the time when the backup was initiated. This is necessary even if the recovery passphrase was changed after the backup completed. Oracle recommends that you make a new backup every time the recovery passphrase is changed to ensure that there is always a copy of the backup that is protected by the most recent recovery passphrase.

18.5 Restoring Oracle Key Vault Data

Oracle Key Vault data from a remote backup destination can be restored onto another Oracle Key Vault server.

This restore operation minimizes downtime and data loss.

18.5.1 About the Oracle Key Vault Restore Process

The restore process replaces the database with the backup data.

You must restore Oracle Key Vault data to a server only after ensuring that all scheduled backups on the server are completed.

Restoring data to an Oracle Key Vault server replaces the data in the server with that of the backup. The data restored is only as current as the backup. The restore operation replaces the Oracle Key Vault server with the backup. This means that some data can be lost. You might need to restore the endpoint database. Any data that is not in the backup that is getting restored will be lost. Backups can only be restored to the same version of Oracle Key Vault at which the backup was taken.

The maximum life of a backup is 1 year.

You must have the recovery passphrase that was in effect at the time of the backup in order to restore data from a backup. If you have not changed the recovery passphrase since installing Oracle Key Vault, then you must use the recovery passphrase that you created during the post-installation process.

Restoring data in Oracle Key Vault entails the following general steps:

  1. Setting up the backup environment, which includes, after installing Oracle Key Vault, configuring backup destinations.

  2. Performing the restore operation by determining the backup to use from a local or remote backup destination, and then providing the recovery passphrase to begin the restore process.

18.5.2 Procedure for Restoring Oracle Key Vault Data

You can restore Oracle Key Vault data to a standalone server using the Oracle Key Vault management console.

Before you restore, ensure that you have the correct recovery passphrase. You will need to enter this passphrase during the restore process. In addition, do not perform a restore operation while a certificate rotation process is in progress.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. In the System Backup page, click Restore.
  5. Select Source from the drop-down list.
    Values are either LOCAL or the names of configured remote destinations.
  6. Select Restore next to the backup you want to restore from.
  7. Click Restore to initiate the restore or recovery process.

    You are prompted for the recovery passphrase.

  8. Enter the recovery passphrase and then click Restore to begin.

    The system will restore from the backup and then restart.

  9. After the restore is complete and the system has restarted, delete any paused periodic backup jobs and then re-create them, using a new backup destination.
    Oracle recommends that you delete such jobs in order to avoid corrupting the backup catalog file.
  10. If your site uses the Commercial National Security Algorithm (CNSA) suite, then re-install these algorithms on the Oracle Key Vault server after the restore operation is complete.
  11. If you had integrated Oracle Audit Vault with Oracle Key Vault before the restoration process, then do the following:
    1. Log in to the Audit Vault Server console as an administrator and then delete the target and agent for Oracle Key Vault.
    2. Log in to the Oracle Key Vault management console and then re-integrate Oracle Audit Vault with Oracle Key Vault.

18.5.3 Multi-Master Cluster and the Restore Operation

In a multi-master cluster deployment, you must consider several factors before you restore data to Oracle Key Vault.

  • You must restore only if all nodes in the cluster are lost.
  • You must restore the backup to a standalone Oracle Key Vault server that has the same IP address as the node from which the backup was taken. Not doing so may affect the ability of endpoints to connect to the restored backup.
  • After the restore operation, you must now use the restored server as the first node of a new cluster.

18.5.4 Primary-Standby and the Restore Operation

In a primary-standby deployment, you must consider several factors before you restore data to Oracle Key Vault.

  • You must perform the restore operation only if both the primary and standby data are lost.
  • You must restore the backup on a standalone Oracle Key Vault server only, even if the backup was taken from the primary.
  • If you restore a backup taken from the primary node, you must use a freshly installed Oracle Key Vault server as the new standby.
  • If the standby server has taken over as primary, and the former primary is lost, then there is no need to restore data from a backup to a new standby server. Just add a new standby server to the primary-standby deployment, which will automatically synchronize with the new primary.

18.5.5 Certificates and the Restore Operation

A third-party certificate installed at the time of a backup will not be copied when you restore another server from this backup.

You must re-install the third-party certificate on the new server in order to use it.

If you perform a certificate rotation in your configuration (so that the endpoints have been updated to using new certificates as well), but then restore the Oracle Key Vault server from a backup that was taken before the certificate rotation was performed, then the endpoints will not be able to connect to the restored Oracle Key Vault system.

Related Topics

18.5.6 Changes Resulting from a System State Restore

Restoring an Oracle Key Vault server brings the system state back to the time when the backup last performed.

Therefore, any changes that were made after the backup was made do not exist on the restored system. For example, if a user's password was changed after the backup operation, the new password will not be available in the restored system. The restored system will have the password that was in effect when the backup was made.

Note:

Restoring also changes the recovery passphrase to the one that was in effect during the backup.

You should change the user passwords, enroll the endpoints created after backup, and make other similar changes, if required. You should confirm that everything is configured correctly after restoring.

If you are not certain that you restored the correct backup, then you can restore a different one, provided that Oracle Key Vault continues to remain a standalone server. To restore another backup, first configure the remote destination of this backup on the restored Oracle Key Vault itself, and then start the restore process. You do not need to reinstall the Oracle Key Vault appliance.

When the Oracle Key Vault server has been restored and is functional, you can continue to back up Oracle Key Vault data to new remote destinations.

Oracle recomments that you change user passwords after a restore operation and backup the Oracle Key Vault.

18.6 Backup and Restore Best Practices

Oracle provides best practices to keep backups current so that you can recover from catastrophic failures with minimum downtime and data loss.

  • Ensure that the recovery passphrase at the time of backup is accessible because you will need it to restore data from a backup.

  • Back up data any time you change the recovery passphrase.

  • Ensure that you create at least one remote backup destination in a primary-standby deployment. Because the local backup resides on the Oracle Key Vault server itself, it will be lost in a failover or switchover situation.

  • Do not edit or delete the backup catalog file that is associated with a remote backup destination, even if you stop using the backup destination. If you ever need to restore from a backup on this server, you will need the backup catalog file.

  • If you use the same remote server for multiple backup destinations, then ensure that the directories are unique so that you have distinct backup catalog files associated with each backup destination. If you fail to do this, then the backup catalog file will be overwritten during subsequent backups and become unusable.

  • When you restore a backup, do so to a standalone Oracle Key Vault server that has the same IP address as the Oracle Key Vault server on which the backup was taken. Failing to do so may result in endpoints not being able to connect to the restored backup.

  • Before you restore data, ensure that all scheduled backups are complete.

  • To create remote backup destinations successfully:

    • Ensure that the servers used as remote backup destinations are enabled and active.
    • Ensure that there is connectivity between Oracle Key Vault and remote server that you plan to use as a backup destination.
    • Ensure that the remote server designated as a backup destination supports the secure copy protocol (SCP) or the SSH file transfer protocol (SFTP).
    • Validate the user account credentials on the remote server before you create the backup destination on Oracle Key Vault.
    • Ensure that the destination directory has write permissions.
    • Create more than one remote backup destination on multiple servers for redundancy.
    • Ensure that the destination directories are unique if you are using the same remote server for multiple backup destinations. You must do this to prevent later backups from overwriting previous ones.
  • Perform a one-time backup once every seven days.

  • Schedule a periodic backup with a period of one day. This ensures that you have a full backup once in seven days.

  • Perform a local one-time backup before system changes. You can use this backup as a restore point.

  • Backup before and after upgrading Oracle Key Vault server software.

  • Backup before and after performing critical operations such as rotating certificates.

  • Change the backup destination after each upgrade. If at all possible do not reuse the backup destination.