1. Administrator's Guide
  2. Monitoring and Auditing Oracle Key Vault

17 Monitoring and Auditing Oracle Key Vault

Oracle Key Vault administrators can monitor and audit the Oracle Key Vault system, configure alerts and use reports.

  • Managing System Monitoring
    System monitoring refers to tasks such as configuring SNMP connections, email notifications, the syslog destination, and system diagnostics.
  • Configuring Oracle Key Vault Alerts
    You can select the type of alerts that you want to see in the Oracle Key Vault dashboard.
  • Managing System Auditing
    Auditing entails tasks such as capturing audit records in a syslog file or downloading the audit records to a local file.
  • Using Oracle Key Vault Reports
    Oracle Key Vault collects statistical information on a range of activities that impact Key Vault operations.

17.1 Managing System Monitoring

System monitoring refers to tasks such as configuring SNMP connections, email notifications, the syslog destination, and system diagnostics.

Parent topic: Monitoring and Auditing Oracle Key Vault

17.1.1 Configuring Remote Monitoring to Use SNMP

With Simple Network Management Protocol (SNMP) enabled, system administrators can remotely monitor the Oracle Key Vault appliance and its services.

The collected data can be further processed and presented for the needs of the enterprise.

Parent topic: Managing System Monitoring

17.1.1.1 About Using SNMP for Oracle Key Vault

You can use the Simple Network Management Protocol (SNMP) to monitor devices on a network for resource usage.

Monitoring Oracle Key Vault is an important aspect how critical Oracle Key Vault's availability is when hundreds or thousands of Oracle and MySQL databases store their TDE master encryption keys in an Oracle Key Vault multi-master cluster. The types of resource usage that you should monitor include memory, CPU utilization, and processes. Even though Oracle Key Vault provides continuous key availability by allowing up to 16 (geographically distributed) instances to be connected to a single cluster, the health of each individual node contributes to the performance and availability of the entire cluster.

You can use Simple Network Management Protocol (SNMP) third-party tool to monitor remote systems that access Oracle Key Vault. The benefits of using SNMP to monitor Oracle Key Vault are as follows:

  • There is no need to allow SSH access to Oracle Key Vault. (SSH access should only be enabled for the window of time in which it is being used.)
  • You do not need to install additional tools to perform an SNMP monitoring operation.

Oracle Key Vault uses SNMP version 3 for user authentication and data encryption features. Unlike SNMP versions 1 and 2 that communicate in readable, insecure plaintext, SNMP 3 authenticates users and encrypts data on the communication channel between the monitoring server and the target. The information from Oracle Key Vault is unreadable to an intruder, even if the communication channel is intercepted.

In addition, with SNMP enabled on Oracle Key Vault, you can determine whether the key management server (KMIP daemon) is running. To track this information, you must use a third-party SNMP client to poll the Oracle Key Vault instance, because Oracle Key Vault does not provide SNMP client software.

Oracle Key Vault audits the creation and modification of SNMP credentials.

You must be a user with the System Administrator role to configure the SNMP account with a user name and password. These SNMP credentials are needed to access SNMP data.

In a multi-master cluster, the SNMP account with a user name and password can be set for all nodes of the cluster at once. It can also be set for each individual node.

Note:

You must ensure that the SNMP username and password is not the same username and password as any of the Oracle Key Vault administrative user accounts with the System Administrator, Key Administrator, or Audit Manager role.

17.1.1.2 Granting SNMP Access to Users

You can grant any user, including users who are not Oracle Key Vault administrators, access to SNMP data.

  1. Log in to the Oracle Key Vault management console as a user with the System Administrator role.
  2. Select the System tab, and then select Monitoring Settings from the left side bar.
  3. In the Monitoring page, enter the following information:
    • SNMP Access: Select All to enable a client at any IP address to poll Oracle Key Vault for information, Disabled to prevent any client, regardless of the client IP address, to poll Oracle Key Vault for information, or IP Address(es) if you want to restrict polling to clients with specific IP addresses. If you select IP Address(es), then enter the IP addresses of the users you want to grant access to in the IP Address field. Separate multiple IP addresses by a space. You cannot enter a range of IP addresses. You must list each IP address individually.
    • Username: Enter a name to associate with the SNMP configuration that will perform the monitoring.
    • Password and Confirm Password: Enter a secure password for this user that is at least 8 or more characters and contains at least one of each of the following: an uppercase letter, a lowercase letter, a number, and a special character from the set: period (.), comma (,), underscore (_), plus sign (+), colon (:), space. The SNMP password must not be the same as the password used to login into the Oracle Key Vault management console in any of the administrative roles.
  4. Click Save.
17.1.1.3 Changing the SNMP User Name and Password

You can change the SNMP user name and password for a node at any time.

  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then select Monitoring Settings.
  3. In the Username, Password, and Reenter Password fields, enter the user name and password information.
  4. Click Save.
17.1.1.4 Changing SNMP Settings on the Standby Server

You change the SNMP settings from the command line on the standby server.

To add SNMP support in a primary-standby environment, you should configure SNMP on both the primary and standby servers before pairing them. This is because the standby server is no longer accessible from the Oracle Key Vault management console because all requests are forwarded to the primary server. However, you can change SNMP settings on the standby server in a primary-standby environment.

  1. Log in to the standby server as the support user.
  2. Switch to the root user. 
    su -
  3. Go to the Oracle Key Vault bin directory.
    cd /usr/local/okv/bin/
  4. Run the stdby_snmp_enable script.
    ./stdby_snmp_enable parameter "options"
    In this specification:
    • parameter can be the following:
      • -a, which sets the SNMP access. It accepts the following options:
        • all grants SNMP access.
        • disabled disables SNMP access.
        • IP_addresses specifies one or more IP addresses to be granted SNMP access. Separate each IP address with a space.
      • -u sets the user's SNMP name.
      • -p sets the user's SNMP password.
    • options is only used with the -a parameter.

The following examples show how to change SNMP settings on a standby server:

To grant SNMP access to all IP addresses and assign a user name snmpuser and password password:
./stdby_snmp_enable -a "all" -u "snmpuser" -p "password"
To disable SNMP access from all IP addresses: 
./stdby_snmp_enable -a "disabled"
To grant SNMP access to certain IP addresses and assign user name snmpuser and password password:
./stdby_snmp_enable -a "192.0.2.1 192.0.2.3 192.0.2.3" -u "snmpuser" -p "password"
17.1.1.5 Remotely Monitoring Oracle Key Vault Using SNMP

SNMP enables you to monitor the vital components of Oracle Key Vault remotely without having to install new software in Oracle Key Vault.

Though there are third-party tools that graphically display the information that SNMP extracts from Oracle Key Vault, the examples shown here are given with snmpwalk and snmpget from the command line on a remote computer that has a network connection into the SNMP account in Oracle Key Vault.
  1. Log in to the remote host that will monitor Oracle Key Vault.
  2. Confirm that the UCD-SNMP-MIB is installed on the remote host from which Oracle Key Vault is monitored.
  3. Query the object ID for an Oracle Key Vault-supported SNMP Management Information Base (MIB) variable.
    For example, suppose you wanted to track the number of processes running for the SNMP host. You can use a third-party SNMP client utility to query the status of the KMIP MIB whose object ID is 1.3.6.1.4.1.2021.2, as follows:
    third_party_snmp_client_command -v 3 OKV_IP_address -u SNMP_user -a SHA -A SNMP_password -x AES -X SNMP_password -l authPriv iso.3.6.1.4.1.2021.2.1.2

    The output is similar to the following:

    iso.3.6.1.4.1.2021.2.1.2.1 = STRING: "mwecsvc"              <== Event collector
iso.3.6.1.4.1.2021.2.1.2.2 = STRING: "httpd"                <== httpd
iso.3.6.1.4.1.2021.2.1.2.3 = STRING: "kmipd"                <== KMIP daemon
iso.3.6.1.4.1.2021.2.1.2.4 = STRING: "ora_pmon_dbfwdb"      <== embedded DB
iso.3.6.1.4.1.2021.2.1.2.5 = STRING: "ServiceManager"       <== Golden Gate Service Manager (Monitors other processes and reports status)
iso.3.6.1.4.1.2021.2.1.2.6 = STRING: "adminsrvr"            <== Golden Gate Admin Server (Communicates with the DB to perform certain maintenance/admin tasks)
iso.3.6.1.4.1.2021.2.1.2.7 = STRING: "distsrvr"             <== Golden Gate Distribution Server (Sends the OGG changes to other nodes)
iso.3.6.1.4.1.2021.2.1.2.8 = STRING: "recvsrvr"             <== Golden Gate Receiver Server
17.1.1.6 SNMP Management Information Base Variables for Oracle Key Vault

Oracle Key Vault provides a set of SNMP Management Information Base (MIB) variables that you can track.

The following table lists the MIB variables that are supported.

Table 17-1 MIBs That SNMP Tracks for Oracle Key Vault

MIB Variable Object ID Description

hrSystemUptime

1.3.6.1.2.1.25.1.1

Tracks the amount of time that an Oracle Key Vault instance has been running

ifAdminStatus.x

1.3.6.1.2.1.2.2.1.7

Tracks if the Oracle Key Vault network interface (x) are running, not running, or being tested. Values are as follows:

  • 1: Instance is running

  • 2: Instance is down

  • 3: Instance is being tested

memAvailReal

1.3.6.1.4.1.2021.4.6

Tracks the available RAM

memTotalReal

1.3.6.1.4.1.2021.4.5

Tracks the total amount of RAM being used

ssCpuRawIdle

1.3.6.1.4.1.2021.11.53

For CPU monitoring; tracks the number of ticks (typically 1/100s) spent idle

ssCpuRawInterrupt

1.3.6.1.4.1.2021.11.56

For CPU monitoring; tracks the number of ticks (typically 1/100s) spent processing hardware interrupts

ssCpuRawKernel

1.3.6.1.4.1.2021.11.55

For CPU monitoring; tracks the number of ticks (typically 1/100s) spent processing kernel-level code

ssCpuRawNice

1.3.6.1.4.1.2021.11.51

For CPU monitoring; tracks the number of ticks (typically 1/100s) spent processing reduced-priority code

ssCpuRawSystem

1.3.6.1.4.1.2021.11.52

For CPU monitoring; tracks the number of ticks (typically 1/100s) spent processing system-level code

ssCpuRawUser

1.3.6.1.4.1.2021.11.50

For CPU monitoring; tracks the number of ticks (typically 1/100s) spent processing user-level code

ssCpuRawWait

1.3.6.1.4.1.2021.11.54

For CPU monitoring; tracks the number of ticks (typically 1/100s) spent waiting for input-output (IO)

UCD-SNMP-MIB.prTable

1.3.6.1.4.1.2021.2

Tracks the number of processes running under a certain name. Names we monitor are httpd (the http server), kmipd (the kmip daemon), and ora_pmon_dbfwdb (an indicator if the DB is down)

See Also:

For more information refer to the Net-SNMP documentation at http://www.net-snmp.org

Parent topic: Configuring Remote Monitoring to Use SNMP

17.1.1.7 Example: Simplified Remote Monitoring of Oracle Key Vault Using SNMP

In Linux, you can simplify the SNMP commands you manually enter to find Oracle Key Vault information, yet still have useful and detailed output.

The configuration in this section assumes that you have granted SNMP access to a trusted user. It also assumes that the you have installed the SNMP Management Information Base (MIB) variables on the remote host that will monitor Oracle Key Vault.

For example, a lengthy version of the snmpwalk command for an SNMP user named snmp_admin is as follows: 

snmpwalk -v3 OKV_IP_address -n "" -l authPriv -u snmp_admin -a SHA -A snmp_user_password -x AES -X snmp_user_password

This command lists the vital services that are running on Oracle Key Vault. However, you can modify the command (and other SNMP commands) to be not only shorter, but to show additional information, such as whether the services are running or not running.

To simplify this type of command, you can edit the /etc/snmp/snmp.conf configuration file so that the SNMP commands you enter will automatically include commonly used settings, such as the default user or the default security level. The example in this topic omits password parameters so that users can enter the password at the command line interactively.

  1. Log in to the remote host that will monitor Oracle Key Vault.
  2. Edit the /etc/snmp/snmp.conf, which appears as follows: 
    # As the snmp packages come without MIB files due to license reasons, 
# loading MIBs is disabled by default. If you added the MIBs you 
# can reenable loading them by commenting out the following line. 
  mibs :
  3. Comment out the # mibs : line and then add the following lines, as follows:
    # loading MIBs is disabled by default. If you added the MIBs you 
# can reenable loading them by commenting out the following line. 
# mibs : 
defSecurityName snmp_admin 
defSecurityLevel authPriv 
defAuthType SHA 
defPrivType AES

    In this example:

    • defSecurityName: Enter the name of the user to whom you granted SNMP access. This example uses snmp_admin.
    • defSecurityLevel: Enter the default security level to use. This example uses authPriv, which enables communication with authentication and privacy.
    • defAuthType: Enter the default authorization type. This example uses SHA.
    • defPrivType: Enter the default privilege type. This example uses AES.
  4. Restart snmpd to load the configuration file.

    For example, for Linux 7:

    systemctl restart snmpd

    For Linux 6:

    service snmpd restart
  5. To run the simplified version of the snmpwalk command that was shown earlier, enter the following command: 
    snmpwalk okv_ip_address prNames -A snmp_user_pwd -X snmp_user_pwd

    In this command, prNames refers to "process names", which displays the names of processes instead of numbers. For example: 

    $ snmpwalk 192.0.2.254 prNames -A snmp_user_pwd -X snmp_user_pwd
UCD-SNMP-MIB::prNames.1 = STRING: mwecsvc
UCD-SNMP-MIB::prNames.2 = STRING: httpd
UCD-SNMP-MIB::prNames.3 = STRING: kmipd
UCD-SNMP-MIB::prNames.4 = STRING: ora_pmon_dbfwdb
UCD-SNMP-MIB::prNames.5 = STRING: ServiceManager
UCD-SNMP-MIB::prNames.6 = STRING: adminsrvr
UCD-SNMP-MIB::prNames.7 = STRING: distsrvr
UCD-SNMP-MIB::prNames.8 = STRING: recvsrvr
An example of running the snmptable command now becomes the following. 
snmptable okv_ip_address prTable -A snmp_user_pwd -X snmp_user_pwd

Output similar to the following appears. 

SNMP table: UCD-SNMP-MIB::prTable 
prIndex         prNames prMin prMax prCount prErrorFlag prErrMessage prErrFix prErrFixCmd
      1         mwecsvc     1     1       1     noError      noError            
      2           httpd     1    20       9     noError      noError                
      3           kmipd     1     2       2     noError      noError                
      4 ora_pmon_dbfwdb     1     1       1     noError      noError                
      5  ServiceManager     1     1       1     noError      noError                
      6       adminsrvr     1     1       1     noError      noError               
      7        distsrvr     1     1       1     noError      noError                 
      8        recvsrvr     1     1       1     noError      noError

The next example shows how you would now run the snmpdf command: 

snmpdf okv_ip_address -A snmp_user_pwd -X snmp_user_pwd

Output similar to the following appears. 

Description                Size (kB)      Used   Available   Used% 
/                          20027260    7247856    12779404     36%  
/usr/local/dbfw/tmp         6932408      15764     6916644      0%  
/var/log                    5932616      19932     5912684      0% 
/tmp                        1999184       3072     1996112      0% 
/var/lib/oracle           143592160   35023900   108568260     24%

17.1.2 Configuring Email Notification

You can use email notifications to directly notify administrators of Key Vault status changes without logging into the Oracle Key Vault management console.

Parent topic: Managing System Monitoring

17.1.2.1 About Email Notification

Email notifications alert users of status changes and are used to complete the processes of endpoint enrollment and user password reset operations.

To enable email notification you must set your email preferences in Oracle Key Vault. You can choose the events that you want updates to. The events include Oracle Key Vault system status like disk utilization, backup, and primary-standby, or user and endpoint status like expiration of user passwords, endpoint certificates, and keys, or cluster status like the heartbeat lag, naming conflicts, cluster-wide HSM status, and others.

Oracle Key Vault supports anonymous and insecure connections to the SMTP server. By default, Oracle Key Vault uses the default Java truststore packaged with Oracle Key Vault's Java library to validate the server certificate. Optionally, you can upload a custom truststore in order to use a specific certificate or certificate chain at the same time you configure SMTP settings.

You can modify the SMTP server configuration at any time. If a custom SMTP certificate was used initially, and you later decide to use the default, you can modify the trust store setting to default, instead of custom.

For example:

  • The enrollment token generated during endpoint enrollment can be mailed directly to the endpoint administrator from Oracle Key Vault.

  • An Oracle Key Vault system administrator can send the random temporary password directly to the user when the user password is reset.

To enable email notifications successfully, there must be a connection between Oracle Key Vault and the SMTP server.

You can disable email notifications at any time.

17.1.2.2 Configuring Email Settings

You can configure the Simple Mail Transfer Protocol (SMTP) server properties to receive email notifications from Oracle Key Vault.

  1. Log in to the Oracle Key Vault management console as a user with the System Administrator role.
  2. Click the System tab, and then click Email Settings.
  3. In the Email Settings page, enter the following values:

    • SMTP Server Address: Enter a valid SMTP server address or host name for the user account. This setting should match the SMTP server setting of the user's email account. Ensure that the SMTP server or hostname is reachable from Oracle Key Vault. If you enter the SMTP hostname, you must configure DNS from the System Settings menu, so the host name can be resolved.

    • SMTP Port: Enter the SMTP port number of the outgoing SMTP server, usually 465. This port number can be another number, if expressly configured that way in your organization.

    • Name: Enter an alias for the SMTP user that will appear in the From field of the email.

    • From Address: Enter the email address that you want to provide as a sender.

    • If the SMTP server requires a secure connection, select Require Secure Connection. If you are using anonymous relay on Microsoft Exchange Server, or an external SMTP server such as Gmail or Office 365, do not select Require Secure Connection. Ensure that your firewall rules allow forwarding of SMTP requests to an external SMTP server.

      If Require Secure Connection is selected, the Authentication Protocol field is displayed with two options, SSL and TLS. Select the authentication protocol for the email server, either SSL or TLS. The default is TLS.

    • If you have an SMTP user account, then check the box Require Credentials. When checked, the input fields Username, Password, and Reenter Password appear:

      • Enter the username of the SMTP user account.

      • Enter the password for the SMTP user account.

      • Reenter the password for the SMTP user account.

      Caution:

      Oracle strongly recommends that you have a secure connection to the SMTP server, because auto-generated tokens are sent over email for operations such as the creation of administrative users and Oracle Key Vault system alerts.

      Do not check Require Credentials for non-secure connections.

    • If Custom SMTP Server Certificate is checked, then the field Upload Certificate File appears with the Choose File button to its right. Select this option if you want to upload a custom SMTP server's certificate to establish a TLS session between SMTP and Oracle Key Vault. This is how you can add a custom truststore in cases where the default Java truststore does not contain a necessary certificate. After Upload Certificate File, click Browse to upload a custom certificate file.

  4. Click Configure.

    On successful configuration, a SMTP successfully configured message is displayed.

    If the configuration fails, then check that the SMTP server settings of the user email account are correct. Error messages highlight the field where the error has occurred to help isolate the problem.

17.1.2.3 Testing the Email Configuration

Oracle Key Vault management console enables you to send test emails to test the email configuration.

You can test the email configuration of the SMTP user account any time after you save the configuration. If you change an existing SMTP configuration, then you must save the configuration before you can test it.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then select Email Settings.

    The Email Settings page appears.

  3. Configure the user's SMTP settings.
  4. Save the configuration.
    You must save the configuration before you can test it.
  5. In the Send Test Email section, enter the user email address in the Email Address field. Then click Test.

    An email is sent to the user with Oracle Key Vault: Test Message in the subject line.

    Depending on the Oracle Key Vault server timestamp, the email notification may not show up as the latest email.

    The email notification may also not show up in your inbox, in which case you must check the spam folder.

    If the email notification is not received, click the Reports tab and select System Reports from the left sidebar. On the System Reports page, click Notification Report. Check the list to determine the issue encountered while sending the email notification.

17.1.2.4 Disabling Email Notifications for a User

You can use the Oracle Key Vault management console to enable or disable email notifications.

An Oracle Key Vault user may elect not to receive email alerts. Only a user with the System Administrator role, or a user managing his own account can disable email notifications.
  1. Log in to the Oracle Key Vault management console as a user with the System Administrator role.
  2. Select the Users tab.

    The Manage Users page appears.

  3. Click User Name of the user.

    The User Details page appears.

  4. Check the box to the left of text Do not receive email alerts.
  5. Click Save.

17.1.3 Configuring the Syslog Destination for Individual Multi-Master Cluster Nodes

On each node, you can forward syslog entries to a remote service such as Splunk or SIEM.

Parent topic: Managing System Monitoring

17.1.3.1 Setting the Syslog Destination Setting for the Node

You can set the syslog destination to use either Transmission Control Protocol (TCP) or User Datagram Protocol (UDP).

  1. Log into any Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then System Settings from the left navigation bar.
  3. In the Syslog section, select one of the following options:
    • TCP: Enables syslog using the TCP protocol.
    • UDP: Enables syslog using the UDP protocol.
  4. Enter the syslog destination IP addresses and port numbers in the Syslog Destinations field, in the format IP_address:port.
    You can enter multiple destinations, each separated by a space.
  5. In the Syslog section, click Save.
17.1.3.2 Clearing the Syslog Destination Setting for the Node

You can clear the syslog destination setting for the node and then reset the node to the cluster setting.

  1. Log into any Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then System Settings from the left navigation bar.
  3. In the Syslog section, click the Use Cluster Settings button.
    Clicking Use Cluster Settings is immediate for this setting. You do not need to click Save afterward.

17.1.4 Capturing System Diagnostics

To troubleshoot problems that may arise, you can generate a system diagnostics file.

Parent topic: Managing System Monitoring

17.1.4.1 About Capturing System Diagnostics

The Oracle Key Vault diagnostics file provides advanced debug and troubleshooting information for problems that you may encounter while using Oracle Key Vault.

You can download this file and provide it to Oracle support for further analysis and debugging. The diagnostics file includes information about free space and disk usage reported is space available to Oracle Key Vault and not based on total disk size.

Diagnostics reporting is not enabled by default. You must enable the feature to generate diagnostics reports. After you have enabled diagnostics, you can configure the necessary information to be captured in diagnostics reports. You then can customize and package diagnostics reports with flexibility. Be aware that the first time you run the diagnostic utility or after the Oracle Key Vault system's internal database has been restarted, it can take longer that it will in future runs because it must gather all the diagnostic information in the system.

If you plan to perform an upgrade of Oracle Key Vault, then you must remove the diagnostic generation utility before performing the upgrade.

17.1.4.2 Installing the Diagnostics Generation Utility

You can use the Oracle Key Vault management console to download instructions for installing and using the diagnostics generation utility.

The instructions also explain how you can customize the output in the reports to accommodate different categories.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select System.
  3. Click Download Diagnostics.
    If the diagnostics generation utility is not installed, then you will be prompted to download the diagnostics-not-enabled.readme file.
  4. Save the diagnostics-not-enabled.readme file to a local directory.
  5. Follow the directions in this readme file to install and run the diagnostics generation utility, and to customize the report output.
    The readme file includes the following instructions, but you should double-check this file in case these instructions have changed:
    1. Use SSH as to connect as user support , then switch user (su) to root .
    2. Install the diagnostics generation utility:
      /usr/local/dbfw/bin/priv/dbfw-diagnostics-package.rb --install
    3. Enable the collection of diagnostics:
      /usr/local/dbfw/bin/priv/dbfw-diagnostics-package.rb --enable ALL
17.1.4.3 Generating a System Diagnostics File

The system diagnostics file that you download is in a .zip file.

  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select System.
    The Status page appears.
  3. Click Download Diagnostics.
    You should be prompted to download a .zip file, which will contain the diagnostics reports. If you are prompted to download the diagnostics-not-enabled.readme file, then the diagnostics generation utility has not been installed and you will need to install it.
  4. Download the .zip file that contains the diagnostic reports to a secure location.
17.1.4.4 Removing the Diagnostic Generation Utility Temporary Files

Removing the diagnostic generation utility temporary files frees up space on your server.

After you have run diagnostic reports, temporary files will accumulate. You should periodically remove these files. You can execute the command to remove these files from any directory.
  1. Log in to the server where you downloaded and installed the diagnostic generation utility.
  2. Use SSH as to connect as user support , then switch user (su) to root .
  3. Execute the following command: 
    /usr/local/dbfw/bin/priv/dbfw-diagnostics-package.rb --clean
    This command removes any .zip files that are found in the /usr/local/dbfw/tmp directory.
17.1.4.5 Removing the Diagnostic Generation Utility

If you no longer need to generate system diagnostic reports, then you can remove the diagnostic generation utility.

If you plan to upgrade Oracle Key Vault, then you must remove the diagnostic generation utility before you perform the upgrade. Removing this utility does not remove its temporary files.
  1. Log in to the server where you downloaded and installed the diagnostic generation utility.
  2. Use SSH as to connect as user support , then switch user (su) to root .
  3. Execute the following command: 
    /usr/local/dbfw/bin/priv/dbfw-diagnostics-package.rb --remove

17.1.5 Configuring Oracle Audit Vault Integration for the Node

You can configure the integration of Oracle Audit Vault (but not the Database Firewall component) for a node.

  1. Log into any Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the System tab, and then System Settings from the left navigation bar.
  3. Select the Enable check box to Oracle Audit Vault Integration for the node.
  4. In the Password and Reenter password fields that appear after you click Enable, enter the password of the user in the database that Audit Vault and Database Firewall will use to extract the audit records.
  5. Click Save.

Parent topic: Managing System Monitoring

17.2 Configuring Oracle Key Vault Alerts

You can select the type of alerts that you want to see in the Oracle Key Vault dashboard.

  • About Configuring Alerts
    System administrators can configure alerts from the Oracle Key Vault dashboard, but all users can see alerts for their security objects.
  • Configuring Alerts
    You can configure alerts in the Reports page of the Oracle Key Vault management console.
  • Viewing Open Alerts
    Users can view alerts depending on their privileges.

Parent topic: Monitoring and Auditing Oracle Key Vault

17.2.1 About Configuring Alerts

System administrators can configure alerts from the Oracle Key Vault dashboard, but all users can see alerts for their security objects.

The Oracle Key Vault dashboard is the first page you see on logging into to the management console. You can navigate to this page by clicking the Home tab. All users can see the alerts on security objects they have access to, but only users with the System Administrator role can configure alerts.

Oracle Key Vault has 17 alerts, including alerts for an HSM-enabled Oracle Key Vault server, that you can configure with appropriate thresholds according to your requirements.

You can configure the following alerts:

Table 17-2 Available Alerts

Alert Type Applicability Purpose
Cluster FIPS Not Consistent Cluster-wide Raised when at least one, but not all, ACTIVE nodes in the cluster are in FIPS mode
Cluster Heartbeat Lag Node specific Raised when a node has not received a heartbeat from another ACTIVE node in the cluster for over the threshold value (default 5 minutes)
Cluster HSM Not Consistent Cluster-wide Raised when at least one, but not all, ACTIVE nodes in the cluster are HSM-enabled
Cluster Naming Conflict Cluster-wide Raised when a naming conflict is resolved
Cluster Redo Shipping Status Node specific Raised when a read-write node is unable to ship redo to its read-write peer, and as a result, is in read-only restricted mode
Disk Utilization Node specific Raised when the free disk space percentage of the /var/lib/oracle partition is lower than the threshold value (default 25 percent)
Endpoint Certificate Expiration Cluster-wide Raised when an endpoint's certificate is expiring within the threshold value (default 30 days)
Failed System Backup Node specific Raised when the last backup did not complete successfully
Primary-Standby Data Guard Broker Status Primary-Standby Only Raised when the Oracle Data Guard Broker status is not ENABLED
Primary-Standby Data Guard Fast-Start Failover Status Primary-Standby Only Raised when the fast-start failover status is not SYNCHRONIZED
Primary-Standby Destination Failure Primary-Standby Only Raised when the switchover status is FAILED DESTINATION
Primary-Standby Restricted Mode Primary-Standby Only Raised when in primary-standby environment and the primary is running in read-only restricted mode
Primary-Standby Role Change Primary-Standby Only Raised when there is a role change
Key Rotations Cluster-wide Raised when a key's deactivation date is within the threshold value (default 7 days)
OKV Server Certificate Expiration Node specific Raised when the Oracle Key Vault server certificate is expiring within the threshold value (default 30 days)
SSH Tunnel Failure Node specific Raised when an SSH tunnel is not available
System Backup Node specific Raised when the last successful backup is over the threshold value (default 14 days)
User Password Expiration Cluster-wide Raised when a user's password will expire within the threshold value (default 14 days)
OKV Server Certificate Expiration Node specific Raised when the Oracle Key Vault server certificate is expiring within the threshold value (default 30 days)
Invalid HSM Configuration Node specific Raised when there is an error in the HSM configuration (checked by default every 5 minutes)
Cluster Replication Lag Node specific Raised when incoming replication lag is greater than the threshold value (default 60 seconds)

Related Topics

Parent topic: Configuring Oracle Key Vault Alerts

17.2.2 Configuring Alerts

You can configure alerts in the Reports page of the Oracle Key Vault management console.

  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Select the Reports tab.
  3. Select Configure Alerts from the left sidebar.

    The Configure Alerts page appears, listing various alert types and for some, information such as the days until expiration (such as a user password expiration). If you are using a multi-master cluster, then the Configure Alerts page will provide cluster-specific alerts, such as the cluster heartbeat lag, redo shipping status, or whether naming conflicts resolution is enabled. The following image shows how the Configure Alerts page appears in a non-multi-master cluster environment.


    Description of configure_alerts_18-2.png follows
    Description of the illustration configure_alerts_18-2.png

  4. Check the boxes in the Enabled column to the right of the alert types to enable the alert.
    Then set the threshold value in the box under Limit. This value determines when the alert will be sent. You can uncheck the boxes by alerts that you do not want to appear in the dashboard.
  5. Click Save.

Related Topics

Parent topic: Configuring Oracle Key Vault Alerts

17.2.3 Viewing Open Alerts

Users can view alerts depending on their privileges.

Users with the System Administrator role can view all alerts. Users without system administrator privileges can only view alerts related to objects they can access.
  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Click the Reports tab.
    The Audit Trail appears.
  3. Click Alerts from the left sidebar.

    The Alerts page appears displaying all the alerts that have not been resolved. When you resolve the issue stated in the alert message, the alerts are automatically removed. You cannot explicitly remove them.

    Description of screenshot-14.7.3-step-3.png follows
    Description of the illustration screenshot-14.7.3-step-3.png

    Oracle Key Vault sends all system alerts to the syslog. The following is an example of a system alert in syslog:

    Mar 29 18:36:29 okv080027361e7e logger[13171]: No successful backup done for 4 day(s)

    The following table lists the conditions that trigger alerts, and the accompanying system alert message:

    Condition System Alert Message

    Disk utilization

    Free disk space is below threshold value (currently current value)

    Endpoint certificate expiration

    Endpoint endpoint_name certificate expiration date

    Failed system backup

    Most recent backup failed!

    Key rotations

    Key unique_ID expiration: <date>

    Primary-standby destination failure

    One or more standy servers are in an error state. HA destination failure.

    Primary-standby Oracle Data Guard Broker status

    Data Guard Broker is disabled

    Primary-standby Oracle Data Guard fast-start failover status

    HA FSFO is not synchronized. FSFO status is HA_status

    Primary-standby restricted mode

    HA running in read-only restricted mode

    Primary-standby role change

    HA role changed. Primary IP Address: IP_address

    SSH tunnel failure

    SSH tunnel (IP IP_address) is not available

    System backup

    No successful backup for number day(s)

    User password expiration

    User user_name password expiration: date

    Invalid HSM Configuration

    HSM configuration error. Please refer to the HSM Alert section in the Oracle Key Vault HSM Integration Guide

    Cluster Replication Lag

    Replication lag from node node_name to node node_name is greater than number seconds.

Parent topic: Configuring Oracle Key Vault Alerts

17.3 Managing System Auditing

Auditing entails tasks such as capturing audit records in a syslog file or downloading the audit records to a local file.

Parent topic: Monitoring and Auditing Oracle Key Vault

17.3.1 About Auditing in Oracle Key Vault

Oracle Key Vault records and time-stamps all endpoint and user activity.

The audit records include endpoint groups and user groups, from endpoint enrollment and user password reset, to the management of keys and wallets, and changes to system settings and SNMP credentials. The audit trail captures details on who initiated which action, with what keys and tokens, and the result of the action. In addition, it records the success or failure of each action.

Only a user who has the Audit Manager role can manage the audit trail for Oracle Key Vault activity. Each user can see audit records of the objects that the user can access.

Auditing in Oracle Key Vault is enabled by default.

A user with the Audit Manager role can see and manage all the audit records. Other users can see only those audit records that pertain to security objects that they have created, or have been granted access to.

The audit manager can export audit records to view system activity off line. After exporting the records, the audit manager can delete them from the system to free up resources.

Related Topics

Parent topic: Managing System Auditing

17.3.2 Configuring Syslog to Store Audit Records

You can configure the Oracle Key Vault syslog to store audit records if the System Administrator has enabled this functionality.

  1. Log in to the Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Click the Reports tab.

    The Audit Trail page appears.

  3. Click the Audit Settings button.

    The Audit Settings page appears.


    Description of config_syslog.png follows
    Description of the illustration config_syslog.png

  4. Enter the following settings:
    • Scope: Select Node to restrict the audit records to those that were generated in the current node, or select Cluster to capture audit records for the entire multi-master cluster environment.
    • Send Audit Records To Syslog: Click Yes.
  5. Click Save.
    If syslog is not configured, then the Syslog forwarding to remote machines not enabled error message appears. If this error appears, dismiss the error dialog and go to the next step.
  6. If syslog is configured, then do the following:
    1. Select the System tab, and then select System Settings.
    2. In the Settings page, go to the Syslog pane.
    3. Select the protocol to use to transfer syslog files: TCP or UDP.
    4. Enter the IP address of the remote system where the syslog files will be stored.
  7. Click Save.

Related Topics

Parent topic: Managing System Auditing

17.3.3 Configuring Audit Settings for a Multi-Master Cluster

You can enable or disable auditing for a multi-master cluster.

You can also enable or disable replicating the audit records to other nodes in the cluster and saving the syslog to its configured destination.
  1. Log into any Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Select the System tab, and then Audit Settings from the left navigation bar.
  3. For Scope, select Cluster.
  4. In the Cluster Audit Settings section, select the Yes or No option for each of these actions:
    • Enable Auditing: Enables or disables auditing for all nodes in the cluster.
    • Replicate Audit Records: Enables or disables replication of audit records to all nodes in the cluster.
    • Send Audit Records to Syslog: Enables or disables sending of audit records to the configured syslog location, as configured by a user with the System Administrator role.
  5. Click Save.
    Saving settings in the Node scope overrides the cluster settings for this node.

Parent topic: Managing System Auditing

17.3.4 Viewing Audit Records

To view audit records, access the Oracle Key Vault management console Audit Trail page.

The reports page shows the Audit Trail page by default. The Audit Trail page lists all system activity with details on who performed an operation, when the operation was performed, what object was used to perform the operation, and the result.
  1. Log in to the Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Click the Reports tab.

    The Audit Trail page appears. Optionally, filter records by selecting the table column heads, and from the drop-down list, select the type of sort order that you want.

Parent topic: Managing System Auditing

17.3.5 Exporting and Deleting Audit Records

Oracle Key Vault audit records are stored in a .csv file.

A user with the Audit Manager role can export the audit trail to a .csv file that can be downloaded to the user's local system. The .csv file contains the same details found in the audit trail on the Reports page. The timestamp in the .csv file reflects the time zone of the particular Oracle Key Vault server whose records were exported. After you export the records, you can delete them from the Oracle Key Vault server to free up space.
  1. Log in to the Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Click the Reports tab.

    The Audit Trail appears.

  3. Click Export/Delete Audit Records on the top right.

    The Export/Delete Audit Records page appears.

    Description of screenshot-export-delete-audit.png follows
    Description of the illustration screenshot-export-delete-audit.png

  4. Select the date by clicking the calendar icon.

    Based on the date that you select, the number of records appears after the Number of records to be exported/deleted label.

  5. Click Export to download the audit records in .csv file format to a local folder.

    After you export the records, you can delete them from Oracle Key Vault to free up resources.

  6. Click Delete to remove the audit records.
  7. Click OK to delete or Cancel to stop.

Parent topic: Managing System Auditing

17.3.6 Audit Consolidation with Audit Vault and Database Firewall

Oracle Key Vault audit data can be forwarded to Audit Vault and Database Firewall (AVDF) for audit consolidation.

  1. Log in to the Oracle Key Vault management console as a user who has the System Administrator role.
  2. Click the System tab, then System Settings.

    The Settings page appears.

  3. Click the box to the right of Enable in the Oracle Audit Vault Integration pane.
    Two password fields appear: Enter Password and Reenter Password.
  4. Enter the password and confirm it.

    Store this password in a safe place. You will need it when you create a secured target on the Audit Vault and Database FirewallAVDF side.

Related Topics

Parent topic: Managing System Auditing

17.4 Using Oracle Key Vault Reports

Oracle Key Vault collects statistical information on a range of activities that impact Key Vault operations.

Parent topic: Monitoring and Auditing Oracle Key Vault

17.4.1 About Oracle Key Vault Reports

The reports cover system activity, certificate expiration, keys, passwords, entitlement status, and metadata.

Oracle Key Vault provides four types of reports for endpoints, users, keys and wallets, and system. In a multi-master cluster, some reports contain additional information, such as the node ID, node name, and IP address.

The four report types are as follows:

  • Endpoint reports contain details of all endpoint and endpoint group activity, certificate and password expiration, and access privileges.

  • User reports contain details of all user and user group activity, their certificate and password expiration, and access privileges.

  • Keys and wallets reports list the access privileges granted to all keys and wallets, and the details of TDE master encryption keys managed by Oracle Key Vault.

  • System reports contain a history of system backups taken and scheduled, details of remote restoration points, and RESTful API usage.

A user who has the Audit Manager role can view all reports, including reports that are accessible from the Audit Trail pages in the Oracle Key Vault management console. A user with the Key Administrator role can view user reports and keys and wallets reports. Users with the System Administrator role can view endpoint, user, and system reports.

Related Topics

Parent topic: Using Oracle Key Vault Reports

17.4.2 Viewing Endpoint Reports

You must have the Audit Manager role to view the four categories of endpoint reports.

Oracle Key Vault offers four endpoint reports: Endpoint Activity, Endpoint Certificate Expiry, Endpoint Entitlement, and Endpoint Metadata.
  1. Log in to the Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Click the Reports tab to display the Reports page.
  3. Click Endpoint Reports under Reports in the left sidebar.

    The Endpoint Reports page appears displaying four endpoint report types.

    Description of endpoint_reports.png follows
    Description of the illustration endpoint_reports.png

  4. Click the link under Name to view the report that you want.

    For example, the Endpoint Certificate Expiry report appears similar to the following:

    Description of rpt_ep_cert_exp.png follows
    Description of the illustration rpt_ep_cert_exp.png

Parent topic: Using Oracle Key Vault Reports

17.4.3 Viewing User Reports

You must have the Audit Manager role to view the four categories of user reports.

Oracle Key Vault offers four user reports: User Activity, User Entitlement, User Expiry, and User Failed Login.
  1. Log in to the Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Click the Reports tab.
  3. Click User Reports to see user specific reports.

    The User Reports page appears displaying the four types of user reports.

    Description of user_reports.png follows
    Description of the illustration user_reports.png

  4. Click the report name to see the corresponding user report.

Parent topic: Using Oracle Key Vault Reports

17.4.4 Viewing Keys and Wallets Reports

You must have the Audit Management role to view the two categories of keys and wallets reports.

Oracle Key Vault offers two reports for keys and wallets: Entitlement and TDE Key Metadata.
  1. Log in to the Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Click the Reports tab.
  3. Click Keys and Wallets Reports under the REPORTS heading.

    The Keys and Wallets Reports page appears displaying the reports.

  4. Click the report name to see the corresponding report.

Parent topic: Using Oracle Key Vault Reports

17.4.5 Viewing System Reports

You must have the Audit Manager role to view the three categories of system reports.

Oracle Key Vault offers three system reports: Backup History, Backup Restoration Catalog, and RESTful API Usage.
  1. Log in to the Oracle Key Vault management console as a user who has the Audit Manager role.
  2. Click the Reports tab.
  3. Click System Reports under the REPORTS heading.

    The System Reports page appears displaying the system reports available.

    Description of system_rpts.png follows
    Description of the illustration system_rpts.png

  4. Click the report type to see the corresponding system report.

Parent topic: Using Oracle Key Vault Reports