19 Backup and Restore Operations

You may configure automatic backups for continuous, reliable, and protected access to security objects with minimum downtime.

19.1 About Backing Up and Restoring Data in Oracle Key Vault

You can use Oracle Key Vault to back up and restore Oracle Key Vault data.

You should back up data periodically to reduce down time and recover from unexpected data losses and system failures. You can restore a new or existing Oracle Key Vault server from a backup.

Backup and restore operations may be performed from the Oracle Key Vault management console. You must be a user who has the System Administrator role to back up and restore Oracle Key Vault data. You can schedule backups at periodic intervals to run automatically at designated times. You also can run these operations on-demand to save a current snapshot of the system.

Oracle strongly recommends that you back up Oracle Key Vault data regularly on a schedule. This practice ensures that backups are current and hold the most recent data. You can use this backup to restore a new or existing Oracle Key Vault server and be fully operational with minimum downtime and data loss.

Oracle Key Vault encrypts all backed up data, which is copied to the backup destination using the secure copy protocol (SCP) or the secure file transfer protocol (SFTP). You must therefore ensure that either SCP or SFTP are supported at the backup destination.

In an Oracle Key Vault multi-master cluster environment, the replication intrinsically creates copies of data on other nodes in the cluster. However, you can still perform backups and backup-related operations on all individual Oracle Key Vault cluster nodes. Be aware that backups can still only be restored to standalone Oracle Key Vault servers. Therefore, backups in a cluster are taken for disaster recovery in case of a complete cluster failure and should all be on remote destinations.

19.2 Oracle Key Vault Backup Destinations

A backup destination is the location where Oracle Key Vault data will be copied to and stored.

19.2.1 About the Oracle Key Vault Backup Destination

The backup destination enables the backup data to be available in a location other than the Oracle Key Vault server itself.

This ensures that you have all the relevant data to recover in case of a catastrophic failure with the Oracle Key Vault server or hardware.

The backup destination is usually another server or computer system that you have access to. You can add, delete, and modify a backup destination.

The backup operation copies Oracle Key Vault data to a backup destination of your choice. The backup destination stores the data until it is needed.

Oracle Key Vault provides two types of backup destinations: local and remote. The local backup destination resides on the Oracle Key Vault server itself, the remote one resides externally in a different server or computer system. You can create more than one backup destination for greater availability.

Local and remote backup destinations have the following characteristics:

  • Local backup destination: The local backup destination, LOCAL, is present by default and cannot be removed. Backups to the local backup destination are local backups.

    Backups to LOCAL are useful to save a current state of Oracle Key Vault. Because these backups are stored on disk, they could be lost in the case of hardware or other catastrophic failure. They will also not be available after a failover or switchover in a primary-standby configuration. They cannot be restored to a primary-standby or cluster configuration. Therefore, you should back up the data to a remote destination when using these configurations.

    A LOCAL destination can store only the last full backup and the cumulative incremental backups after that full backup. After a new full backup of the periodic backup to LOCAL completes, the previous periodic full or cumulative incremental backups are deleted.

  • Remote backup destinations: Remote backup destinations reside on external servers and can be dispersed geographically for disaster recovery purposes. Backups to remote backup destinations are remote backups.

    Each backup destination on the external server is associated with a backup catalog file called okvbackup.mgr that Oracle Key Vault maintains at the backup destination. The okvbackup.mgr file catalogs the backups performed and is used to restore data.

    Note:

    You cannot use another Oracle Key Vault server as a remote backup destination.

Caution:

  • Oracle Key Vault may not be able to find the backups if you delete or modify the backup catalog file. Therefore, do not delete or modify this file.

  • Do not configure the same remote backup destination directory for different Oracle Key Vault servers as backup destinations, because backups that happen concurrently from different Oracle Key Vault servers will overwrite each other's catalog file, with the result that Oracle Key Vault may not be able to locate the backups correctly.

  • After you restore a backup that contains a remote backup destination, do not continue to use that remote backup destination. Delete any backup jobs that are configured to send backups to that destination. Continuing to use this backup destination could corrupt the backup catalog file. Oracle Key Vault may not be able to locate backups correctly.

  • Configure each node in a multi-master cluster to send their backups to a different backup destination.

19.2.2 Creating a Remote Backup Destination

You can use the Oracle Key Vault management console to create a remote backup destination.

To create a remote backup destination, you must provide a user account, a unique existing directory location on an external server, and an authentication method (password or key-based). Oracle Key Vault needs this information to make a secure connection with the remote server.
  1. Log in to the Oracle Key Vault management console as a user with the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. In the System Backup page, select Manage Backup Destinations.
    The Manage Backup Destinations window appears.Description of 21_manage_backup_dest.png follows
    Description of the illustration 21_manage_backup_dest.png
  5. Click Create.

    The Create Backup Destination window appears.

    Description of 21_create_backup_dest.png follows
    Description of the illustration 21_create_backup_dest.png

  6. Enter the following information for the backup location:
    • Destination Name: Enter a descriptive name to identify the backup destination.

    • Transfer Method: Choose between scp and sftp to copy files to the remote destination.

    • Hostname: Enter the host name or IP address of the remote server for the backup. If you enter the host name, then ensure that DNS is configured to translate the host name to its corresponding IP address. Do not include spaces, single quotation marks, or double quotation marks in a host name that is in a remote backup destination.

    • Port: Enter the SCP or SFTP port number on the external server. The default is 22.

    • Destination Path: Enter the path to an existing directory on the external server where the backup file will be copied. You cannot modify this directory location after the backup destination is created. This path must not be the destination for backups from another Oracle Key Vault server. Do not include spaces, single quotation marks, or double quotation marks destination path that is in a remote backup destination.

    • Username Enter the user name of the user account on the remote server. Ensure that write permissions are set on the directory specified in Destination Path for the user identity that establishes the SCP or SFTP connection. Do not include spaces, single quotation marks, or double quotation marks in a user name that is in a remote backup destination.

    • Authentication Method: Choose one of the following:

      • Key-based Authentication: Copy the public key that appears and paste it in the appropriate configuration file, such as authorized_keys, on the destination server. Check that the permissions of the configuration file are set to allow access only to the backup account owner and no other group or user. Be aware that certain events may trigger a change of the public key, which means that Oracle Key Vault cannot use the backup destination until the new public key is re-copied from Oracle Key Vault to the appropriate configuration file. These events include but are not limited to certificate rotation, changing the IP address, and conversion to a cluster node.

      • Password Authentication: The password of the user account entered in the Username field.

  7. Click Save.
Oracle Key Vault validates the input that you supplied to create the backup destination. If the validation fails, then the backup destination is not created. If this happens, then check values for the user account on the remote server (user name and password or key) and ensure that the directory has write permissions for the user. Finally, ensure that the remote server is active.

19.2.3 Changing Settings on a Remote Backup Destination

After you have created the backup destination, you can only change the SCP or SFTP port number and details of the user account.

You cannot change any other setting.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Select Manage Backup Destinations.

    The Manage Backup Destinations page appears displaying LOCAL and remote backup destinations.

  5. Click the backup destination name to edit the backup destination.

    The Edit Backup Destination page appears.

  6. Modify the following information:
    • Port: Change the SCP or SFTP port number on the external server.

    • Destination Public Key: This field shows the public key information that Oracle Key Vault currently has stored in its known_hosts file for the remote server. If the remote server's public key changes, then Oracle Key Vault cannot copy backups to or from the remote server. In order to store the new public key in the known_hosts file, click the Reset Dest Public Key button, which will retrieve and save the new public key from the remote server.

    • Username: Enter the user name of the user account on the remote server. Ensure that the new user has write permissions on the directory that is specified in Destination Path, because this path cannot be changed.

    • Authentication Method: Choose one of the following:

      • Password Authentication: The password of the user account entered in the Username field.

      • Key-based Authentication: If the Oracle Key Vault public key has changed, re-copy the public key that appears in the Public Key field and then paste it in the appropriate configuration file, such as authorized_keys, on the destination server. Be aware that certain events may trigger a change of the public key, which means that Oracle Key Vault cannot use the backup destination until the new public key is re-copied from Oracle Key Vault to the appropriate configuration file. These events include but are not limited to certificate rotation, changing the IP address, and conversion to a cluster node.

  7. Click Save.
Oracle Key Vault validates the input that you supplied to update the backup destination. If the validation fails, then the backup destination is not updated. If this happens, then check values for the user account on the remote server (user name and password) and ensure that the directory has write permissions for the user. Finally, ensure that the remote server is active.

19.2.4 Deleting a Remote Backup Destination

You can delete a remote backup destination to stop future backups to that destination server.

Backups already on the destination server will remain there.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Select Manage Backup Destinations.
  5. In the Manage Backup Destinations page, check the boxes for the backup destinations that you want to delete.
  6. Click Delete.

19.3 Backup Schedules and States

Oracle Key Vault provides backup schedule types depending on the backup destination, and different states that indicate the progress of the backup activity.

19.3.1 About Backup Schedule Types and States

You can schedule backups in Oracle Key Vault for specific times and backup destinations.

The backup process starts at the scheduled time and generates a system backup, which is a file that is stored on the backup destination. There is one backup file for each completed backup.

No backup can start if another backup is in progress. You can change the schedule of backups as needs change. You can continue working with Oracle Key Vault while the backup is in progress.

A system restart will terminate any ongoing backup. If you must restart the system, then you can cancel a backup that is scheduled to happen at the same time, and backup the system after the restart.

19.3.2 Types of Oracle Key Vault Backups

Oracle Key Vault provides two types of backup jobs that can be scheduled: one-time backups and periodic backups.

  • One-time backup: A one-time backup makes a full backup of the Oracle Key Vault system. You can schedule multiple one-time backup jobs, each with its own start time.

    You should make one-time local backups before making significant configuration changes to Oracle Key Vault, in case you need to recover from configuration failures.

    LOCAL destinations can only store the last one-time backup. When a one-time backup to LOCAL completes, the previous backup is deleted.

  • Periodic backup: The periodic backup process first makes a full backup of the Oracle Key Vault system and puts the backup schedule in active state. At the end of the subsequent periodic interval, a cumulative incremental backup starts. This cumulative incremental backup holds changes from the last full backup. Another full backup is made after 7 days have passed since the last full backup.

    For example, if the backup period is once a day, then every seventh one is a full backup. If the backup period is every 8 days, then all backups are full backups. If the backup period is 12 hours, then there are 13 cumulative backups before a full backup.

    You should schedule periodic backups with a period of at least one day to minimize data loss.

    A LOCAL destination can store only the last full backup and the cumulative incremental backups after that full backup. After a new full backup of the periodic backup to LOCAL completes, previous periodic full or cumulative incremental backups are deleted.

    Cumulative incremental backups are faster than full backups. Only one periodic backup can be scheduled at any time.

19.3.3 Scheduled Backup States in Oracle Key Vault

Scheduled backups have four states, which indicate whether the backup is scheduled, in progress, completed, or paused.

  • ACTIVE: The backup is scheduled and will be processed at the specified start time or period.
  • PAUSED: All future backups are on hold and will not start even if the start time has passed. They will start when they are explicitly resumed. You can change the state from active to paused and back. Put a scheduled backup in the paused state for these situations:
    • When communication between Oracle Key Vault and the remote destination is broken
    • If the remote destination is unavailable or inactive
    • If you want to defer the backup

    You can delete the scheduled backups that have not completed.

  • ONGOING: The backup is in progress.
  • DONE: The backup is complete.

19.4 Scheduling and Managing Oracle Key Vault Backups

You can schedule Oracle Key Vault backups to specific backup destinations and times.

You must create the backup destinations that you will use beforehand, and you can modify or delete backup schedules.

19.4.1 Scheduling a Backup for Oracle Key Vault

You can schedule a one-time or a periodic backup to a local or remote backup destination.

You can start a one-time backup to start immediately without setting a time. However, do not schedule backup operations if a certificate rotation or cluster operation is in progress.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. In the Backup page, click Backup.

    The Backup page appears. The following image shows how it appears when PERIODIC is selected.

    Description of 21_backup.png follows
    Description of the illustration 21_backup.png

  5. In the Name field, enter a name for the backup.
  6. If you want to perform a periodic backup, then do the following:
    1. In the Start Time field, use the Calendar icon to specify the start time for the backup.
    2. For Type, select PERIODIC.
      The additional fields Days, Hours, and Mins appear.
    3. In the Days, Hours, and Mins fields, enter the times for the periodic backups to occur.
    4. For Destination, select the destination backup from the list.
    5. Click Schedule to add the scheduled backup to the Scheduled Backup(s) page.
  7. If you want to perform a one-time backup, then do the following:
    1. In the Start Time field, use the Calendar icon to specify the start time for the backup. If you want the backup to perform immediately after you click Schedule, then select Now.
    2. For Type, select ONE-TIME.
    3. For Destination, select the destination backup from the list.
    4. Click Schedule to add the scheduled backup to the Scheduled Backup(s) page.

19.4.2 Changing a Backup Schedule for Oracle Key Vault

You cannot change the schedule of a backup in progress.

To change the backup schedule the state must be active or paused.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Click the name of the scheduled backup in the Scheduled Backup(s) page.
  5. Enter the Start Time or click the calender icon for a one-time backup.

    For a one-time backup, you can only change the start time if the backup has not already started. This means that the state cannot be ongoing or done. For a periodic backup you can change the start time if the scheduled start time has not passed.

  6. Enter the Days, Hours, and Mins for a periodic backup.
  7. Select Save.

19.4.3 Deleting a Backup Schedule from Oracle Key Vault

You can delete a backup schedule from the Oracle Key Vault management console.

  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. Check the boxes of scheduled backups listed in the Scheduled Backup(s) page.
  5. Click Delete to delete the selected backup schedules.

19.4.4 How Primary-Standby Affects Oracle Key Vault Backups

In a primary-standby deployment, you must perform backups on the primary server.

Because the standby synchronizes its state with the primary, you do not need to back up the standby.

Be aware of the following behavior for failover or switchover operations in a primary-standby deployment:

  • Any backups in progress will terminate if there is a failover or a primary-standby switchover. Backups to LOCAL are private to the Oracle Key Vault server and therefore the local backup on the primary server is not available after a failover or switchover.

  • Backups scheduled with password authentication start as usual after the failover or switchover.

  • Remote backups using key-based authentication will need to update the public key on the destination to match the one shown on the new primary system.

  • Because backups can only be restored to standalone servers, you must unpair primary-standby deployments before you can perform a backup restore operation on the former primary.

19.4.5 How Using a Cluster Affects Oracle Key Vault Backups

In a multi-master cluster environment, be aware of how the backup process works in individual nodes and the entire cluster.

  • In an Oracle Key Vault multi-master cluster environment, the replication intrinsically creates copies of data on other nodes in the cluster. However, you can still perform backups and backup-related operations on all individual Oracle Key Vault cluster nodes.
  • Backups in a cluster are taken for disaster recovery in case of a complete cluster failure and should all be done on remote destinations.
  • Backups can still only be restored to standalone Oracle Key Vault servers. Because a cluster node cannot be switched back to a standalone server, only remote backups should be taken.

19.4.6 Protecting the Backup Using the Recovery Passphrase

Oracle Key Vault uses the recovery passphrase to control who can restore user and system data.

To restore a backup, use the Oracle Key Vault recovery passphrase from the time when the backup was initiated. This is necessary even if the recovery passphrase was changed after the backup completed. Oracle recommends that you make a new backup every time the recovery passphrase is changed to ensure that there is always a copy of the backup that is protected by the most recent recovery passphrase.

19.5 Restoring Oracle Key Vault Data

Oracle Key Vault data from a remote backup destination can be restored onto another Oracle Key Vault server.

This restore operation minimizes downtime and data loss.

19.5.1 About the Oracle Key Vault Restore Process

The restore process replaces all data on the new server except the root and support user passwords.

You will not be able to restore data to a server if there is a scheduled backup in process on the server.

Note:

You must restore Oracle Key Vault data to a server only after ensuring that all scheduled backups on the server are completed.

Restoring data to an Oracle Key Vault server replaces the data in the server with that of the backup. Any changes made since the last backup will be lost. Backups can only be restored to the same version of Oracle Key Vault at which the backup was taken.

The maximum life of a backup is 1 year.

Note:

Any backup older than a year cannot be restored.

You must have the recovery passphrase that was in effect at the time of the backup in order to restore data from a backup. If you have not changed the recovery passphrase since installing Oracle Key Vault, then you must use the recovery passphrase that you created during the post-installation process.

Restoring data in Oracle Key Vault entails the following general steps:

  1. Setting up the backup environment, which includes, after install Oracle Key Vault, configuring backup destinations.

  2. Performing the restore operation by determining the backup to use from a local or remote backup destination, and then providing the recovery passphrase to begin the restore process. You create the recovery passphrase as part of the post-installation tasks for Oracle Key Vault.

19.5.2 Procedure for Restoring Oracle Key Vault Data

You can restore Oracle Key Vault data to a standalone server using the Oracle Key Vault management console.

Before you restore, ensure that you have the correct recovery passphrase. You will need to enter this passphrase during the restore process. In addition, do not perform a restore operation while a certificate rotation process is in progress.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then Settings from the left navigation bar.
  3. In the System Configuration area, click Backup and Restore.
  4. In the System Backup page, click Restore.
  5. Select Source from the drop-down list.
    Values are either LOCAL or the names of configured remote destinations.
  6. Select Restore next to the backup you want to restore from.
  7. Click Restore to initiate the restore or recovery process.

    You are prompted for the recovery passphrase.

  8. Enter the recovery passphrase and then click Restore to begin.

    The system will restore from the backup and then restart.

  9. Delete any paused periodic backup jobs and then re-create them, using a new backup destination.
    Oracle recommends that you delete such jobs in order to avoid corrupting the backup catalog file.
  10. If your site uses the Commercial National Security Algorithm (CNSA) suite, then re-install these algorithms on the Oracle Key Vault server after the restore operation is complete.

19.5.3 Multi-Master Cluster and the Restore Operation

In a multi-master cluster deployment, you must consider several factors before you restore data to Oracle Key Vault.

  • You must restore only if all nodes in the cluster are lost.
  • You must restore the backup to a standalone Oracle Key Vault server that has the same IP address as the node from which the backup was taken. Not doing so may affect the ability of endpoints to connect to the restored backup.
  • The data restored is only as current as the backup.
  • After the restore operation, you must now use the restored server as the first node of a new cluster.

19.5.4 Primary-Standby and the Restore Operation

In a primary-standby deployment, you must consider several factors before you restore data to Oracle Key Vault.

  • You must perform the restore operation only if both the primary and standby data are lost.
  • You must restore the backup on a standalone Oracle Key Vault server only, even if the backup was taken from the primary.
  • The restore operation replaces the Oracle Key Vault server with the backup. This means that some data can be lost. You might need to restore the endpoint database.
  • If you restore a backup taken from the primary node, you must use a freshly installed Oracle Key Vault server as the new standby.
  • If the standby server has taken over as primary, and the former primary is lost, then there is no need to restore data from a backup to a new standby server. Just add a new standby server to the primary-standby deployment, which will automatically synchronize with the new primary.
  • If your site uses the Commercial National Security Algorithm (CNSA) suite, then you must re-install these algorithms on the Oracle Key Vault server after the restore operation is complete.

19.5.5 Certificates and the Restore Operation

A third-party certificate installed at the time of a backup will not be copied when you restore another server from this backup.

You must re-install the third-party certificate on the new server in order to use it.

If you perform a certificate rotation in your configuration (so that the endpoints have been updated to using new certificates as well), but then restore the Oracle Key Vault server from a backup that was taken before the certificate rotation was performed, then the endpoints will not be able to connect to the restored Oracle Key Vault system.

19.5.6 Changes Resulting from a System State Restore

Restoring an Oracle Key Vault server brings the system state back to the time when the backup last performed.

Therefore, any changes that were made after the backup was made do not exist on the restored system. For example, if a user's password was changed after the backup operation, the new password will not be available in the restored system. The restored system will have the password that was in effect when the backup was made.

Note:

Restoring also changes the recovery passphrase to the one that was in effect during the backup.

You should change the user passwords, enroll the endpoints created after backup, and make other similar changes, if required. You should confirm that everything is configured correctly after restoring.

If you are not certain that you restored the correct backup, then you can restore a different one, provided that Oracle Key Vault continues to remain a standalone server. To restore another backup, first configure the remote destination of this backup on the restored Oracle Key Vault itself, and then start the restore process. You do not need to reinstall the Oracle Key Vault appliance.

When the Oracle Key Vault server has been restored and is functional, you can continue to back up Oracle Key Vault data to new remote destinations.

Oracle recomments that you change user passwords after a restore operation and backup the Oracle Key Vault.

19.6 Backup and Restore Best Practices

Oracle provides best practices to keep backups current so that you can recover from catastrophic failures with minimum down time and data loss.

  • Ensure that the recovery passphrase at the time of backup is accessible because you will need it to restore data from a backup.

  • Back up data any time you change the recovery passphrase.

  • Ensure that you create at least one remote backup destination in a primary-standby deployment. Because the local backup resides on the Oracle Key Vault server itself, it will be lost in a failover or switchover situation.

  • Do not edit or delete the backup catalog file that is associated with a remote backup destination, even if you stop using the backup destination. If you ever need to restore from a backup on this server, you will need the backup catalog file.

  • If you use the same remote server for multiple backup destinations, then ensure that the directories are unique so that you have distinct backup catalog files associated with each backup destination. If you fail to do this, then the backup catalog file will be overwritten during subsequent backups and become unusable.

  • When you restore a backup, do so to a standalone Oracle Key Vault server that has the same IP address as the Oracle Key Vault server on which the backup was taken. Failing to do so may result in endpoints not being able to connect to the restored backup.

  • Before you restore data, ensure that all scheduled backups are complete.

  • To create remote backup destinations successfully:

    • Ensure that the servers used as remote backup destinations are enabled and active.
    • Ensure that there is connectivity between Oracle Key Vault and remote server that you plan to use as a backup destination.
    • Ensure that the remote server designated as a backup destination supports the secure copy protocol (SCP) or the SSH file transfer protocol (SFTP).
    • Validate the user account credentials on the remote server before you create the backup destination on Oracle Key Vault.
    • Ensure that the destination directory has write permissions.
    • Create more than one remote backup destination on multiple servers for redundancy.
    • Ensure that the destination directories are unique if you are using the same remote server for multiple backup destinations. You must do this to prevent later backups from overwriting previous ones.
  • Perform a one-time backup once every seven days.

  • Schedule a periodic backup with a period of one day. This ensures that you have a full backup once in seven days.

  • Perform a local one-time backup before system changes. You can use this backup as a restore point.

  • Backup before and after upgrading Oracle Key Vault server software.

  • Change the backup destination after each upgrade. If at all possible do not reuse the backup destination.