These release notes list the new features for this release of Oracle Key Vault, how to download the latest product software and documentation, and how to address known issues in Oracle Key Vault.

1.1 Changes in This Release for Oracle Key Vault

This Oracle Key Vault release introduces new features that enhance the use of Oracle Key Vault in a large enterprise.

1.1.1 Dual NIC Network Interface Support

Starting with this release, Oracle Key Vault supports the use of two network interfaces, referred to as dual NIC configuration.

In a dual NIC configuration, Oracle Key Vault combines the two network interfaces into a single logical interface using the Linux NIC bonding mechanism to provide redundancy at the network layer. The dual NIC configuration maintains the network availability of an Oracle Key Vault in case one of the interfaces becomes unavailable. Depending upon the dual NIC configuration mode, load balancing of the network traffic may also be achieved.

This type of configuration is particularly useful in large Oracle Key Vault deployments where need for operational continuity is higher despite physical or software failures. Configuring a dual NIC network interface helps to avoid the scenario where, for example, a network interface associated with an Oracle Key Vault server becomes unavailable, which can result in a loss of communication between the Oracle Key Vault nodes and between endpoints and Oracle Key Vault server.

In previous releases, Oracle Key Vault supported only one network interface. When you install and configure Oracle Key Vault in this release, you have the option of using a single network interface (Classic mode) or using dual NIC mode.

1.1.2 LDAP User Authentication and Authorization in Oracle Key Vault

Starting with this release, you can configure authentication and authorization of Oracle Key Vault users to be centrally managed in a Microsoft Active Directory.

This feature benefits large deployment environments where enterprise users are centrally managed in a Microsoft Active Directory. Centrally managing users, as opposed to creating user accounts in different systems and applications, is not only easier and more efficient for administrators, it improves compliance, control, and security. You enable the Microsoft Active Directory users to authenticate with Oracle Key Vault through the use of their directory credentials. You manage the authorization of the directory users in Oracle Key Vault through mapping definitions between Microsoft Active Directory groups and Oracle Key Vault administrative roles or user groups. When a directory user successfully logs in to Oracle Key Vault the first time, Oracle Key Vault automatically creates an Oracle Key Vault user account for this user.

1.1.3 RESTful Services Utility Command-Line Interface for Appliance Management

In this release, the the RESTful service command-line interface has been expanded and redesigned to provide more functionality.

This redesign includes the following:

  • Structured and simplified command-line interface with the following format:
    okv category resource action configuration-options command-options 
  • Profile support in configuration file to centrally administer multiple Oracle Key Vault endpoints.
  • JSON support for command input and output.
  • New commands to support system management tasks and monitoring of deployments, in addition to the enhancements for the current functionality for endpoints, wallets, and security objects.

In previous releases, the RESTful command-line interface covered only endpoint, wallet, and security object management commands. The addition of system management commands, which include commands for backup operations and server operations for standalone, multi-master, and primary-standby environments, benefits large deployments where the automation of these types of configuration is needed.

The previous RESTful services APIs are still supported.

1.1.4 Development Using the Java SDK

This release introduces a new Java language software development kit that you can use to integrate endpoints with the Oracle Key Vault server.

The Java SDK enables developers to create their own custom endpoint integration solutions for Oracle Key Vault.

1.1.5 Development Using the C SDK

This release introduces a new C language software development kit.

The C SDK allows developers to create their own custom endpoint integration solutions for Oracle Key Vault.

1.2 Downloading the Oracle Key Vault Software and the Documentation

At any time, you can download the latest version of the Oracle Key Vault software and documentation.

1.2.1 Downloading the Oracle Key Vault Installation Software

For a fresh installation, you can download the Oracle Key Vault software from the Software Delivery Cloud. You cannot use this package to upgrade Oracle Key Vault. For an upgrade from an existing Oracle Key Vault deployment, you can download the Oracle Key Vault upgrade software from the My Oracle Support website which includes a readme file with upgrade instructions.

  1. Use a web browser to access the Oracle Software Delivery Cloud portal:
  2. Click Sign In, and if prompted, enter your User ID and Password.
  3. In the All Categories menu, select Release. In the next field, enter Oracle Key Vault and then click Search.
  4. From the list that is displayed, select Oracle Key Vault 21.1.0.0.0 or click the +Add to Cart button next to the Oracle Key Vault 21.1.0.0.0.
    The download is added to your cart. (To check the cart contents, click View Cart in the upper right of the screen.)
  5. Click Checkout.
  6. On the next page, verify the details of the installation package, and then click Continue.
  7. In the Oracle Standard Terms and Restrictions page, after you have read the terms and restrictions and agree with them, select I have reviewed and accept the terms of the Commercial License, Special Programs License, and/or Trial License, and click Continue.

    The download page appears, which lists the following Oracle Key Vault ISO files:

    • Vpart_number.iso (Oracle Key Vault 21.1.0.0.0 - Disc 1)

    • Vpart_number.iso (Oracle Key Vault 21.1.0.0.0 - Disc 2)

    • Vpart_number.iso (Oracle Key Vault 21.1.0.0.0 - Disc 3)

    • Vpart_number.iso (Oracle Key Vault 21.1.0.0.0 - Disc 4)

  8. To the right of the Print button, click View Digest Details.

    The listing for the ISO files expands to display the SHA-1 and SHA-256 checksum reference numbers for each ISO file.

  9. Copy the SHA-256 checksum reference numbers and store them for later reference.
  10. Click Download and select a location to save the ISO files. 
    You can save each file individually by clicking its name and then specifying a location for the download.
  11. Click Save.

    The combined size of the ISO files exceeds 4 GB, and will take time to download, depending on the network speed. The estimated download time and speed are displayed in the File Download dialog box.

  12. After the ISO files are downloaded to the specified location, verify the SHA-256 checksums of the downloaded files:
    1. From a Linux or Unix machine, generate a SHA256 checksum for the first Vpart_number.iso:
      $ sha256sum Vpart_number.iso

      Ensure that the checksum matches the value that you copied from the File Download dialog box in step 9.

    2. Generate a SHA-256 checksum for the second Vpart_number.iso:
      $ sha256sum Vpart_number.iso

      Ensure that the checksum matches the value that you copied from the File Download dialog box in step 9.

  13. Optionally, burn each of the Vpart_number.iso files to a DVD-ROM disc and then label the discs:
    • OKV 21.1 Disc 1

    • OKV 21.1 Disc 2

    • OKV 21.1 Disc 3

    • OKV 21.1 Disc 4

You can now install Oracle Key Vault on a server machine.

1.2.2 Downloading the Oracle Key Vault Documentation

  1. Access the Oracle documentation site.
  2. Select Oracle Database Related Products.
  3. In the Database Security section, search for and download the most current version of the Oracle Key Vault 21.1 documentation, including these release notes.

1.3 Known Issues

At the time of this release, there are issues with Oracle Key Vault that could occur in rare circumstances. For each issue, a workaround is provided.

1.3.1.1 User Gets Locked and Expired with Multiple Failed Logins

Issue: The current password policy locks the user account for a day if the user has incorrectly entered the password more than three consecutive times. Therefore, the user will be able to log in only after the 24-hour lockout period expires.

Workaround: Make a note of the password and keep it accessible and secure.

Bug Number: 23300720

1.3.1.2 Private Keys Are Not Overwritten When a Java Keystore Is Uploaded Using the -o Option of the okvutil Utility

Issue: When you upload a Java keystore (JKS) or Java Cryptography Extension keystore (JCEKS) to the Oracle Key Vault server using the -o option of the okvutil upload command, user-defined keys are not overwritten.

Workaround: Remove the private key from the wallet and then upload the keystore again.

Bug Number: 26887060

1.3.1.3 OKV Alerts Still Show in the List After Fixing the Problem

Issue: User password expiration alerts are still showing even after the user changes their password.

Workaround: In the Oracle Key Vault management console, select Reports and then Configure Reports. Then uncheck the User Password Expiration option. Alternatively, ignore the alert.

Bug Number: 27620622

1.3.1.4 On HP-UX System, SELECT FROM V$ENCRYPTION_KEYS May Return ORA-28407 Occasionally

Issue: On HP-UX operating system, a Transparent Data Encryption (TDE) query such as the following that is executed in a long-running database process or session may occasionally result in an ORA-28407 Hardware Security Module error detected error:

SELECT * FROM V$ENCRYPTION_KEYS;

This is because the system could not create another thread-specific data key because the process had reached or exceeded the system-imposed limit on the total number of keys per process, which is controlled by the PTHREAD_KEYS_MAX setting. PTHREAD_KEYS_MAX is typically set to 128.

Workaround: Switch the database sessions and execute the TDE query again. If it is not convenient to switch the sessions, then set PTHREAD_USER_KEYS_MAX to 16384 before starting the database and the listener.

Bug Number: 28270280

1.3.1.5 Deleting a Default Wallet Causes Associated Endpoint to Not Be Able to Execute OKVUTIL LIST

Issue: When deleting a virtual wallet, if that virtual wallet is the default wallet of an endpoint, the endpoint may not be able to execute okvutil list. This is because when the endpoint searches for the wallet, it is unable to find it and therefore generates an error.

Workaround: Do not delete a virtual wallet without first being sure that no endpoint is currently using it as its default wallet. If the wallet is deleted, be sure to remove the default wallet from the endpoint via the Endpoint Details page and re-enroll the endpoint.

Bug Number: 30699255

1.3.1.6 Oracle Key Vault Boot-Time Warnings When in FIPS Mode

Issue: When an Oracle Key Vault server operating in FIPS mode is booted, warnings such as the below may be seen on console:
Warning : Error inserting
    serpent_avx2(/lib/modules/4.1.12-124.34.1.1.el6uek.x86_64/kernerl/arch/x86/crypto/serpent_avx2):
    No such device

These are informational messages thrown on screen indicating that instruction sets for ciphers that are not available or not supported in FIPS mode are not being loaded. These warnings can be safely ignored.

Workaround: None.

Bug Number: 30844891

1.3.1.7 KMIPD May Be Stopped At The Same Time On Multiple Nodes During Certificate Rotation

Issue: During certificate rotation, the kmip and kmipus daemons are restarted several times, with downtime of a few minutes each. It is possible that this restart can happen on multiple nodes at the same time. Especially in smaller clusters, this means it is possible, although unlikely, that all of the kmip daemons in the cluster are unable to respond to endpoint requests, potentially leading to downtime.

Workaround: Turn on or increase the in-memory cache timeout, persistent cache timeout, and persistent cache refresh window values prior to certificate rotation in order to avoid endpoint downtime.

Bug Number: 31311978

1.3.1.8 Endpoint Configuration File OKVCLIENT.ORA Could Be Corrupted If Write Fails When Endpoint Resources Are Under Stress

Issue: On receiving configuration updates from an Oracle Key Vault server, an endpoint updates the configuration file okvclient.ora in the conf/ directory with new values of the parameters. In case there is a failure on the endpoint while writing the updated parameter values to file, it can potentially leave the configuration file in a corrupted state and affect the operation of the endpoint.

Workaround: If this happens, contact Oracle Support for guidance on manually updating the configuration file.

Bug Number: 31351160

1.3.1.9 Assigning Default Wallet Fails if an Endpoint Already Has Access to Same Wallet

Issue: If an endpoint already has access to a wallet, then making this wallet as a default wallet fails. As a side-effect, current default wallet setting is also removed.

Workaround: Remove the endpoint's access to the wallet before assigning it as a default wallet of the endpoint.

Bug Number: 31416663

1.3.1.10 Modal pages in system settings page are blank on page load and after saving

Issue: The modal pages (pop-ups) in System Settings page will be blank while the page loads. Also on clicking Save, the page is loaded again and the page goes blank while loading the content.

Workaround: The pages usually will show up in about 5 to10 seconds after opening the modal page. Also, the page load time depends on the network speed on the client side.

Bug Number: 32283750

1.3.2.1 Unpair of Upgraded Primary-Standby Oracle Key Vault 18.x Servers May Fail Due to Permission Issues

Issue: After having completed an upgrade to the current release of Oracle Key Vault, attempting to unpair from a primary-standby configuration sometimes fails, with the following messages written out to the /var/log/debug files:
ORA-48141: error creating directory during ADR initialization: [/var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb/metadata_pv]
ORA-48189: OS command to create directory failed
Workaround: Before attempting an unpair in a Primary-Standby configuration that has been upgraded to Oracle Key Vault 18.1, please ensure that the /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb/metadata_pv directory has the right permissions using the steps below:
  1. Log into the primary Oracle Key Vault system as user support through ssh.
    $ ssh support@okv_instance_ip_address
  2. Switch to user root.
    support$ su - root
  3. Check the permissions on directory /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb/metadata_pv.
    root# ls -l /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb
    The output should be similar to this output.
    drwxr-xr-x 2 root   oinstall  4096 Apr 24 22:01 metadata_pv
  4. If the directory is owned by user root, as shown above, execute the following command:
    root# chown oracle:oinstall /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb/metadata_pv
    List the file and verify that the owner is now oracle.
    root# ls -l /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb
    The output should be similar to this output.
    drwxr-xr-x 2 oracle oinstall 4096 Apr 24 22:01 metadata_pv

Bug Number: 29693700

1.3.2.2 OKV SYSTEMS That Were Unpaired Before Being Upgraded Need a DB_UNIQUE_NAME Reset

Issue: Oracle Key Vault systems that were part of an Oracle Key Vault 12.2 high availability (now primary-standby) configuration before being unpaired, and then upgraded, have their DB_UNIQUE_NAME parameters set to 'DBFWDB_HA1' or 'DBFWDB_HA2'. This parameter needs to be reset to 'DBFWDB' before the system is converted to cluster mode, as attempting to add the node to a cluster would otherwise fail.

Workaround: For a system that was the primary server in an Oracle Key Vault 12.2 high availability configuration, and then unpaired before being upgraded to the current release of Oracle Key Vault, the following commands need to be run on the system after successful upgrade and before it is converted to a cluster node:
  1. Log into the primary Oracle Key Vault system as user support through ssh.
    $ ssh support@okv_instance_ip_address
  2. Switch to user root.
    support$ su - root
  3. Check the owner and group on directory /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb/metadata_pv.
    root# ls -l /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb
    The output should be similar to this output.
    drwxr-xr-x 2 root   oinstall  4096 Apr 24 22:01 metadata_pv
  4. If the directory is owned by user root, as shown above, execute the following command:
    root# chown oracle:oinstall /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb/metadata_pv
    List the file and verify that the owner is now oracle.
    root# ls -l /var/lib/oracle/diag/rdbms/dbfwdb/dbfwdb
    The output should be similar to this output.
    drwxr-xr-x 2 oracle oinstall 4096 Apr 24 22:01 metadata_pv
  5. Switch to user oracle.
    root# su oracle
  6. Start SQL*Plus.
    oracle$ sqlplus / as sysdba
  7. Execute the following statement:
    show parameter db_unique_name;
  8. If the DB_UNIQUE_NAME is something other than DBFWDB, then execute the following statements:
    alter system set db_unique_name='DBFWDB' scope=spfile;
    exit
  9. As user root, execute the following commands:
    oracle$ service dbfwdb stop
    oracle$ service dbfwdb start
  10. Verify that the DB_UNIQUE_NAME parameter has changed.
    Start SQL*Plus.
    oracle$ sqlplus / as sysdba
  11. Execute the following statement:
    show parameter db_unique_name
    The output returned should match the output shown here.
    NAME                  TYPE        VALUE
    --------------------- ----------- -----------
    db_unique_name        string      DBFWDB

Bug Number: 29696058

1.3.2.3 Attempt to Enable STIG on Upgraded Oracle Key Vault 21.x Servers Fail

Issue: After having completed an upgrade to the current release of Oracle Key Vault, attempting to enable STIG will fail. Alternatively, if STIG is enabled before upgrade, then the upgrade to the current release of Oracle Key Vault will fail.

Workaround: Make sure that STIG is disabled before upgrade. Do not attempt to enable STIG after the upgrade. In order to disable STIG before upgrade, use the steps below:
  1. Log into the Oracle Key Vault system as user support through ssh.
    $ ssh support@okv_instance_ip_address
  2. Switch to user root.
    support$ su - root
  3. Disable STIG.
    root# /usr/local/dbfw/bin/stig --apply --disable

Bug Number: 32557999

1.3.3.1 SSH Tunnel Status Shows as Disabled on Failover Case in Primary-Standby

Issue: After a failover operation, the new Oracle Key Vault primary server does not show the correct status of the SSH tunnel. It shows the SSH tunnel as disabled when the SSH tunnel is available. The dashboard also shows an alert, warning that the setup of an SSH tunnel failed. This is because after the failover operation, Oracle Key Vault tried to establish two SSH tunnels to the same database as a service endpoint, resulting in the incorrect status and dashboard alert. The second SSH tunnel to the database as a service endpoint does not affect connectivity between the Oracle Key Vault server and the database as a service endpoint. The first SSH tunnel to the database as a service endpoint is functional and available after the failover.

Workaround: After a failover, the new Oracle Key Vault primary server shows the correct SSH status as available and connected to the database as a service endpoints. You also can use the okvutil list on the database as a service endpoint to check the status of the SSH tunnel.

Bug Number: 24679516

1.3.3.2 HA Setup Succeeds with Different Primary & Standby RO Restricted Mode Config

Issue: For a primary-standby configuration, before pairing if read-only restricted mode is enabled on one Oracle Key Vault server and not on the other Oracle Key Vault server, then the configuration succeeds. This mismatch can lead to issues and confusion in a primary-standby deployment.

Workaround: Use the Oracle Key Vault management console to ensure that both servers have the same read-only restricted mode state applied. To do so, select the System tab, then Primary-Standby. Select the Allow Read-Only Restricted Mode option. Only then apply the primary-standby configuration on each server.

Bug Number: 26536033

1.3.3.3 Re-pair After Un-pair from HA 12.2 BP5 to new OKV Server Still Shows Standalone

Issue: When an unpaired Oracle Key Vault primary server running Oracle Key Vault 12.2.0.5.0 or later is paired with a newly installed Oracle Key Vault server, the Current status on the Primary-Standby page shows that the server is in standalone mode. The Standalone status indicates that the primary-standby configuration has failed. The primary-standby setup fails because the SSH configuration on the primary server is not re-enabled.

Workaround: Before pairing an unpaired Oracle Key Vault primary server running Oracle Key Vault, disable and re-enable the SSH configuration. You should disable and then re-enable the SSH configuration after you perform the primary-standby configuration on the primary server after unpairing it with the standby server.

Note:

Before pairing an unpaired Oracle Key Vault primary server running Oracle Key Vault, ensure that you have closed all other browser instances.

Bug Number: 26617880

1.3.3.4 Audit Trail is not Sent To Remote Syslog on Switchover in Primary-Standby Pair

Description: With syslog configured on the primary, the audit logs are also written to the syslog. On switchover, the audit logs may not be written to the syslog. This is because the syslog has not been configured on the standby. Syslog needs to be configured on primary and standby separately.

Workaround: Configure the syslog on standby after switchover to enable write of audit logs to syslog.

Bug Number: 28790364

1.3.3.5 Failover Issues When Primary OKV Experiences a Controlled Shutdown

Issue: Periodically, the primary Oracle Key Vault node in a primary-standby pair may have a controlled shutdown. For example, a user performs the shutdown by pressing a power off button in the management console or executes the shutdown command from the terminal. When this happens, there will be no failover operation and the standby Oracle Key Vault node will not take over as the primary server. This can be predicted by the existence of the file /var/lock/subsys/dbfwdb on the primary Oracle Key Vault node. If the file exists on the primary at the time of the controlled shutdown, there will not be a failover. If it does not exist, then a failover should occur.

Note that failover still does occur in other situations such as power loss on the primary or database failure, regardless of the file's existence.

Workaround: If performing a controlled shutdown in an attempt to cause the standby node to take over as the new primary node, instead perform a switchover.

Bug Number: 29666606

1.3.4.1 System Settings Changed on an OKV Node After Conversion to a Candidate Node Do Not Reflect On The Controller Node

Issue: If system settings are changed on an Oracle Key Vault node after it has been converted to a candidate node, and after the controller node's initial attempt to verify the candidate node's settings has failed, the updated settings do not reflect on the controller node. The pairing process must be aborted on both the controller and candidate nodes.

Workaround: None. Abort the pairing process on both the controller and candidate nodes. Change the system settings on the candidate node as desired, then re-attempt the pairing process.

Bug Number: 29430349

1.3.4.2 Replication May Fail to Resume After Multiple System Failures in OKV Cluster

Issue: Due to GoldenGate Bug 29624366, after multiple system failures in an Oracle Key Vault cluster, replication from some nodes may fail to resume. Specifically, GoldenGate replicats will terminate and not be able to process new change logs in the GoldenGate trail file when it happens.

Workaround: Manually reposition such replicats to skip erroneous records in the trail file or forcefully delete the troubled Oracle Key Vault nodes from the cluster and add new nodes to replace them.

Bug Number: 29700647

1.3.4.3 Oracle Key Vault Should Prevent Enabling From Finishing If It Takes Longer Than MDND

Issue: If you enable or disable an Oracle Key Vault node before the Maximum Disable Node Duration time limit, but the enabling does not finish before the Maximum Disable Node Duration time limit expires, it is possible that there could be cleanup of archivelogs and trail files that would cause inconsistency in the cluster. Don't allow the enabling process to finish in this case.

Workaround: Delete or force delete the node from the cluster if it takes longer than the Maximum Disable Node Duration amount of time to finish enabling.

Bug Number: 30533066

1.3.4.4 RMAN Automatically Cleans Up Archivelogs Still Necessary for OGG

Issue: RMAN automatically manages the archivelogs in the fast recovery area. Under normal circumstances, RMAN will not delete archivelogs that may still be needed by Oracle GoldenGate. However, under space pressure, RMAN may clean up the needed archivelogs. These archivelogs getting cleaned up will break replication from the current node to all other nodes except the node's read-write peer node. Oracle Key Vault attempts to mitigate this issue by performing regular clean up of the fast recovery area, but under rare circumstances, the fast recovery area may be filled up and this issue may occur.

Workaround: Identify the source of space pressure in the fast recovery area and remedy the issue. You may identify space pressure in the fast recovery area by keeping tabs on the disk space. The fast recovery area is located under /var/lib/oracle/fast_recovery_area/. If replication has broken because of this issue, take a remote backup of the node and then force delete it from the cluster. Note that you may need to make sure that any keys or other objects created on that node are also on all other nodes in the cluster, and manually re-upload them if they are not.

Bug Number: 30558372

1.3.4.5 Read-Write Nodes in Read-Only Restricted Mode After a Reboot

Issue: After rebooting a read-write node, sometimes the node or its read-write peer will become stuck in read-only restricted mode.

Workaround: When you reboot a node, it is normal for a node's read-write peer node to temporarily run in read-only restricted mode. However, soon after the node finishes booting, the read-write peer should transition back to read-write mode within a few minutes. The node that was rebooted may come up in read-only restricted mode, but should also transition back to read-write mode within a few minutes. However, if either a node or its read-write peer does not leave read-only restricted mode, redo shipping may be stuck. It may be fixed by rebooting the node still in read-only restricted mode.

Bug Number: 30589921

1.3.4.6 Certificate Must Be Rotated Before Converting To Cluster If Upgrading From 12.2 BP4 or Older

Issue: If you upgrade from Oracle Key Vault 12.2 BP4 or older and do not generate a new certificate before converting the upgraded Oracle Key Vault server to a cluster node, you will receive the following error message:

Failed to convert server to cluster node, detected use of weak signature
algorithms in OKV server credentials. Please perform a certificate rotation
operation before converting this server to a cluster node.
Workaround: Upgrade to Oracle Key Vault release 18.4 in two steps:
  1. Upgrade from Oracle Key Vault 12.2 BP4 to 12.2 BP11, and perform a certificate rotation operation.
  2. Upgrade from Oracle Key Vault 12.2 BP11 to Oracle Key Vault release 18.4.
For more information on how to perform a certificate rotation in Oracle Key Vault 12.2 BP11, refer to the Oracle Key Vault Administrator's Guide for release 12.2.

Bug Number: 30673249

1.3.4.7 After Force-Deleting A Read-Write Node In 18.1 Cluster And Then Upgrading, May Not Be Able To Replace Force-Deleted Node In Higher Version

Issue: When force-deleting a read-write node, it should be shutdown first. However, due to GoldenGate bug 30413969, if the force-deleted node is shut down, the downstream extract on the deleted node's read-write peer node is not fully cleaned up. The workaround for this bug is present in Oracle Key Vault versions 18.2 and higher. However, if upgrading from an Oracle Key Vault 18.1 multi-master cluster that has had a read-write node force-deleted, if attempting to replace it after upgrade, it will still not succeed because the cleanup was not executed when the force-delete happened in version 18.1.

Workaround: The following steps are to be executed with caution. Executing these steps on the wrong Oracle Key Vault server will break replication and result in having to force-delete the node on which they were executed.

Example scenario: Nodes A and B are read-write peers. Node B was force-deleted from the cluster. Node A may not have been fully cleaned up due to GoldenGate bug 30413969. Before or after upgrading, but before attempting to add another node as Node A's read-write peer, execute the following steps on node A to finish the cleanup.
ssh support@Oracle_Key_Vault_IP_address
su - root
su - oracle
/var/lib/oracle/dbfw/bin/sqlplus / as sysdba
exec sys.dbms_xstream_adm.drop_outbound('OGG$OKV_DEXT');
exec sys.dbms_streams_adm.remove_queue('OGG$Q_OKV_DEXT', TRUE, TRUE);

After the above steps are successfully executed on Node A, it can be used as the controller node to add another node to the cluster as Node A's read-write peer.

Bug Number: 31216736

1.3.4.8 Backup From Oracle Key Vault 18.1, 18.2 or 18.3 Cluster Node That Is Then Upgraded and Used To Make Another Cluster May Not Be Able To Add A Read-Write Peer

Issue: When restoring a backup taken on a cluster node to a standalone Oracle Key Vault server, the global_name of the database on Oracle Key Vault may be either DBFWDB.DBFWDB or DBFWDB_HA2.DBFWDB, depending on the global_name of the cluster node on which the backup was taken. If the global_name is DBFWDB_HA2.DBFWDB, and the standalone Oracle Key Vault server is converted to a cluster node, then it will not be able to successfully add a read-write peer node due to the global_name mismatch. The global name is fixed during backup restore in versions 18.4 and higher, but if the backup was taken and restored on a lower version, the issue will persist even after upgrading to 18.4 or higher.

Workaround: After restoring the backup to a standalone Oracle Key Vault server, execute these steps before converting it to a cluster node. The first select statement is to confirm that the global_name is DBFWDB_HA2.DBFWDB. Do not proceed with the global_name update if the global_name returned by the below select statement is not DBFWDB_HA2.DBFWDB or if the server has already been converted to a cluster node.
ssh support@Oracle_Key_Vault_IP_address
su - root
su - oracle
/var/lib/oracle/dbfw/bin/sqlplus / as sysdba
select global_name from global_name;
alter database rename global_name to DBFWDB.DBFWDB;

Bug Number: 31241245

1.3.4.9 Cluster Service Status Is Down After Rotating Server Certificate

Issue: Rotating the server certificate will stop multiple processes in order to replace the certificates. However, under normal circumstances, they are restarted soon after they are stopped. During or after certificate rotation, on the Monitoring page under the Cluster tab, the Cluster Services Status may show a downward arrow, indicating that one or more cluster services are not running. This will cause replication to be broken to and from this node. If it persists for more than a few minutes, it is likely that this bug has occurred.

Workaround: If this issue occurs, try to restart the cluster services by clicking the Restart Cluster Services button on the Monitoring page. After a few minutes, refresh the page. If the Cluster Service Status still shows a red downward arrow, contact Oracle Support.

Bug Number: 31371440

1.3.4.10 OGG Bug 32079454 Causes Intermittent Distribution Path Failures Causing Replication to Break

Issue: Due to Oracle GoldenGate bug 32079454, distribution paths will intermittently encounter SSL errors that cause them to fail and not automatically restart. This can cause replication between non-read-write peer nodes to break. Other side effects are objects that transition from PENDING to ACTIVE status to be stuck in PENDING status. You will also get replication lag alerts. You can see if a distribution path has potentially failed by checking the Monitoring page under the Cluster tab, and looking for red, downward arrows in the Cluster Link State section.

Workaround: If restarting the cluster link on the Monitoring page under the Cluster tab does not resolve the issue, restart the node and verify that there are no red, downward arrows on the Monitoring page afterward.

Bug Number: 32079491

1.3.5 Software Development Kit Issues

The Oracle Key Vault Software Development Kit (SDK) has the following issues when working with Oracle Key Vault server.

  • If an unsupported attribute for an object type is added using the attribute addition or registration APIs, the server adds the attribute to the KMIP object, and no error is given by server.

    APIs impacted are: okvAddAttribute, okvCreateKey, okvRegKey, okvRegSecretData, okvRegOpaqueData, okvRegTemplate

  • If a positive integer is passed as value for attribute index for a single instance attribute in request TTLV for registration, addition, modification and delete APIs , the server will add/modify/delete the attribute at index 0 and will return success irrespective of the index passed in the request.

    APIs impacted are : okvAddAttribute, okvModifyAttribute, okvDeleteAttribute, okvCreateKey, okvRegKey, okvRegSecretData, okvRegOpaqueData

  • If an invalid integer is passed as value for attribute index for a multi instance attribute in request TTLV for modification and deletion APIs, the server will not modify or delete the attribute value and also will not throw any error.

    APIs impacted are : okvModifyAttribute, okvDeleteAttribute

  • The server doesn’t validate the cryptographic usage mask value passed in request TTLV and the value will be added in case of attribute addition and modification.

    APIs impacted are: okvModifyAttribute, okvAddAttribute, okvCreateKey, okvRegKey, okvRegSecretData

  • Oracle Key Vault server currently stores type of name attribute as Uninterpreted Text string in all cases irrespective of the name type in request TTLV.

    APIs impacted are: okvAddAttribute, okvModifyAttribute, okvDeleteAttribute, okvCreateKey, okvRegKey, okvRegSecretData, okvRegOpaqueData

  • If an invalid custom attribute identifier is passed in request TTLV for modification or deletion, the server returns success and no attribute value is modified or deleted.

    APIs impacted are: okvModifyAttribute, okvDeleteAttribute

  • If multiple single instance attributes are added to request TTLV, then no error will be thrown and the first value will get added.

    APIs impacted are: okvCreateKey, okvRegKey, okvRegSecretData, okvRegOpaqueData, okvRegTemplate

  • The objects created using SDK are not downloadable by okvutil as of today. Attempting to run the okvutil download command may show the warning WARNING: Could not store <Unique Identifier of the object>. This warning can be safely ignored as we are skipping the download of keys, secrets & objects created through SDK (if found) to the given wallet. Support for the same will be provided in future releases.

  • Whenever a server operation is performed using C SDK, Java SDK, or RESTful utility, the configuration file okvclient.ora does not get updated with the latest state. The configuration file remains static as it is when the endpoint software is installed. For example, if there is a new node added to the Oracle Key Vault cluster, the client configuration file okvclient.ora will not get updated with the new node information whenever a server operation is performed using Oracle Key Vault SDK or RESTful utility.

1.4.1 Oracle TDE and Oracle Key Vault Integration

Depending on the Oracle Database version used and on the feature of TDE used, there might be a need to patch the Oracle database for smooth operations.

Refer to the MOS-NOTE with Doc ID 2535751.1 to ascertain if your deployment needs a database patch.

The MOS-NOTE lists known issues with Oracle Database Transparent Data Encryption (TDE) feature when it is configured to use Oracle Key Vault as the keystore. The document also lists the fixes that resolve the issues enabling smoother integration between Oracle Database TDE and Oracle Key Vault. The issues could be defects, reducing the user burden with simplified operations, or improving the integration between TDE and OKV. The document is for Database Administrators and others tasked with managing the TDE Master Keys with Oracle Key Vault.

1.4.2 Reports are Affected by Audit Replication in a Multi-Master Cluster

Oracle Key Vault reports and details in the home page are generated from Oracle Key Vault audit records. Each node will show reports of the operations specifically done on that node if audit replication is turned off. Each node will show reports of the operations done on all nodes in the cluster if audit replication is turned on.

The recommendation is to turn off audit replication and use a security information and event management (SIEM) solution like Oracle Audit Vault and Database Firewall (AVDF) to collect audit records from all nodes.

1.4.3 Updates in a Multi-Master Cluster are Slower Than in a Single Instance

An update in a multi-master cluster might check for an object's existence, which may result in a scan of all nodes in the cluster slowing down the update operation. The time will increase proportional to the number of nodes in the cluster. The update could take several minutes to complete.

Setting and rotating the TDE master encryption key are examples of update operations.

1.5 Supported Database Versions

The following versions of Oracle Database are supported with Oracle Key Vault 21.1:

  • Oracle DB 11.2 with the compatible parameter set to 11.2
  • Oracle DB 12.1 with the compatible parameter set to 11.2
  • Oracle DB 12.2
  • Oracle DB 18c
  • Oracle DB 19c

1.6 Critical Patch Updates Included in Release 21.1

Oracle Key Vault release 21.1 updated the underlying infrastructure to incorporate the October 2020 Release Update for Oracle Database 19 (19.9 DB RU) - October Release Update. Please sign in for full details.

https://www.oracle.com/security-alerts/cpuoct2020.html

Oracle Key Vault release 21.1 also includes security and stability fixes for Java and Oracle Linux (OL7U9) operating system.

1.7 Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.