M Troubleshooting Oracle Database Security Central

Oracle Database Security Central provides troubleshooting advice for a range of scenarios.

RAC - SAML SSO user login fails

If you encounter intermittent failures when logging in to Oracle Database Security Central with the error, *Forbidden You don’t have permissions to access this resource”, then complete the following steps:

  1. On both nodes, compare the configured value:

    grep '^SSO_DOMAIN' /usr/local/dbfw/etc/dbfw.conf

    The value will be empty on one of the nodes and defined on the other.

  2. Insert the defined value into the same empty property on the node that does not have it defined:

    sudo -u dbfw /usr/local/dbfw/bin/dbfwconf_editor.rb /usr/local/dbfw/etc/dbfw.conf SSO_DOMAIN='<value>'

  3. Regenerate the Apache configuration and restart HTTPD:

    /opt/avdf/bin/configure-networking run-only=apache

SSO login should now work.

RAC - Register firewall displays Certificate Validation Failed

When registering a database firewall with a 2 node RAC AVS system, certificate validation fails. To workaround this issue, run the following command on Database Firewall as root:

/opt/avdf/config-utils/bin/config-rac_cluster register_dbfw

RAC - CRS Not starting after crsctl start crs

The CRS has not started on the node or attempts to restart have failed. To address this issue:

  1. Try to stop CRS normally:

    ORACLE_HOME=$(su - grid -c 'echo $ORACLE_HOME')
    $ORACLE_HOME/bin/crsctl stop crs

  2. If you’re not able to stop CRS normally, force the services to stop:

    $ORACLE_HOME/bin/crsctl stop crs -f

  3. Wait for CRS services to quiesce.

  4. Restart CRS:

    $ORACLE_HOME/bin/crsctl start crs -wait

RAC - Setting Time on RAC Instances

The server was set up without NTP configuration or NTP configuration was removed and the server has now slipped significant time. To address this issue:

  1. Disable CRS on the affected node(s):

    ORACLE_HOME=$(su - grid -c 'echo $ORACLE_HOME')
    $ORACLE_HOME/bin/crsctl stop crs

  2. Set the correct NTP servers. 1-3 servers can be specified.

    /opt/avdf/config-utils/bin/config-ntp set servers="ntp1,ntp2,ntp3" enabled=true sync_on_save=true

    Optionally you can instead set the system time manually, this is inadvisable as it is extremely difficult to get appropriate levels of synchronisation between the nodes.

  3. Restart the CRS on the node:

    $ORACLE_HOME/bin/crsctl start crs -wait

  4. Check the status of the cluster to ensure all services online :

    $ORACLE_HOME/bin/crsctl stat res -t

Information to Provide Support When Filing a Service Request

Review this list of information to provide support when filing a service request.

Note: Diagnostics data, especially trace files, often contains sensitive information. Protect it accordingly and only gather and send the information that’s required.

Before contacting support, the Audit Trail Transaction Log should follow these guidelines:

Related Topics

Using Oracle Trace File Analyzer

If you request support from Oracle Support, they may ask you to run Oracle Trace File Analyzer on the Oracle DBSecCentral to collect diagnostic information. Oracle Trace File Analyzer is already installed on the Oracle DBSecCentral.

  1. Log in to the Audit Vault Server through SSH and switch to the root user.

    See Logging In to Oracle DBSecCentral Appliances Through SSH.

  2. Run tfactl command to collect diagnostics, as needed. For example:

    tfactl diagcollect -avs -noclassify  -noinsight

    The avs parameter should be used to ensure the Oracle DBSecCentral application layer logs will also get collected.

  3. Securely copy the collected diagnostic file to a location from which you can upload the file to the service request. For example:

    scp /var/opt/oracle/ahf/oracle.ahf/data/repository/<diagnostic_zip_file> <new_location>

    Oracle Trace File Analyzer on the Audit Vault Server will automatically collect logs in /var/opt/oracle/ahf/oracle.ahf/data.

    If you have modified the IP address of the Audit Vault Server and are encountering either the TFA-00104 Cannot establish connection with TFA Server. Please check TFA Certificates or TFA-00104 Cannot establish connection with TFA Server. Keystore password was incorrect'" error when running TFA commands, follow these steps to resolve the error:

    1. Log in to the Audit Vault Server through SSH and switch to the root user.

      See Logging In to Oracle DBSecCentral Appliances Through SSH.

    2. Run the following command:

      tfactl syncnodes -regenerate

Related Topics

Ability to Boot Into Rescue Mode When Troubleshooting

You can boot directly into rescue mode from the grub menu on both the DBSecCentral and the Database Firewall. Booting into the rescue mode does not run DBSecCentral processes and allows for easier troubleshooting. Running rescue mode is intended for use by and under the direction of Oracle Support.

Once the system has booted you can switch to rescue mode by running the following command:

- systemctl isolate avdf-minimal.target

You can switch back to usual runtime by running the following command:

- systemctl isolate avdf-runtime.target

Note: Switching from rescue mode on the DBSecCentral to the usual runtime mode can take a long time, around 15 minutes.

Audit Vault Agent or Host Monitor Agent Is Not Upgraded to the New Release

Learn how to upgrade the Audit Vault Agent or Host Monitor Agent manually.

Problem

After upgrading to Oracle AVDF 20.1 or later, some of the Audit Vault Agents or Host Monitor Agents are not upgraded.

Symptom - 1

Audit Vault Agent is in STOPPED state after Oracle DBSecCentral upgrade.

Symptom - 2

Host Monitor Agent is in NEEDS UPGRADE or UPDATE FAILED state after Oracle DBSecCentral upgrade.

Solution - 1

The symptom indicates that the Audit Vault Agent has failed to auto upgrade during the Oracle DBSecCentral upgrade. Run the following steps as the user who installed Agent previously:

  1. Check for any Agent processes on the host machine. Ensure there are no Agent related processes currently running.

  2. Remove the existing agent.jar file and the Agent folder from the host machine.

  3. Download the new agent.jar file from the upgraded Oracle DBSecCentral.

  4. Run the following command:

    java -jar agent.jar [-d <AgentHome>]

  5. Verify the Agent is in RUNNING state.

Solution - 2

The symptom indicates that the Host Monitor Agent has failed to auto upgrade during the Oracle DBSecCentral upgrade. Run the following steps as root user:

  1. Check for any Host Monitor Agent related processes on the host machine. Ensure there are no hostmonitor, hmdeployer, or hostmonmanager processes currently running.

  2. Navigate to the directory outside of hm where the Host Monitor Agent is installed.

  3. Run the following command to uninstall the Host Monitor Agent:

    ./hm/hostmonsetup uninstall

  4. Download the new Host Monitor Agent installable bundle from the Oracle Database Security Central console, for the specific platform on which it will be reinstalled.

  5. Extract the Host Monitor Agent bundle inside the hm directory.

  6. Run the following command to reinstall the Host Monitor Agent in a root owned location:

    ./hostmonsetup install

Failure While Building a Host Monitor Agent or Collecting Oracle Database Trails

Learn what to do when you experience a failure while building Host Monitor Agents or collecting Oracle Database trails.

Problem

This problem may manifest with various symptoms:

Solution

  1. Unset all environment variables except the following:

    • PATH

    • TERM

    • PS1

    • LANG

    • LC_*

    • JAVA_HOME

    Then run the java -jar agent.jar command again on the host machine.

    See Also: Deploying the Audit Vault Agent

  2. If you deployed the Audit Vault Agent in a Linux environment, then ensure that the host machine name appears in the /etc/hosts file.

Error When Running Host Monitor Agent Setup

Review the resolutions for errors that occur when running Host Monitor Agent setup.

Problem

I am setting up a Host Monitor Agent. When I run the command $HOSTMON_HOME/hm/hostmonsetup install, the following error is displayed:

Failed to generate executables for Host monitor

This means the host computer does not have the required libraries for the Host Monitor Agent. Install the required libraries mentioned in Host Monitor Agent Requirements.

Symptom 1

Even after installing the required libraries, if the Host Monitor Agent installation fails with above error message, then examine the makelogerror file which is available in the Host Monitor Agent installation directory. The following errors, may appear in the file:

/bin/ld: cannot find -laio

/bin/ld: cannot find -lnsl

/bin/ld: cannot find -lpcap

/bin/ld: cannot find -lcap

Solution 1

To resolve the issue on Linux (64 bit) systems, follow these steps:

  1. Search where the actual binaries (not symlinks) are present – libnsl,libaio, libpcap, libcap. In most scenarios it should be present either in /lib64 or /usr/lib

  2. Create below symlinks if not already present in /lib64 or /usr/lib

    1. libcap binary

      • ln -s *`<location from step 1>`*/libcap.so.<version> /lib64/libcap.so.1

      • ln -s *`<location from step 1>`*/libcap.so.1 /lib64/libcap.so

    2. libaio binary

      • ln -s *`<location from step 1>`*/libaio.so.<version> /lib64/libaio.so.1

      • ln -s /lib64/libaio.so.1 /lib64/libaio.so

    3. libnsl binary

      • ln -s *`<location from step 1>`*/libnsl.so.<version> /lib64/libnsl.so.1

      • ln -s /lib64/libnsl.so.1 /lib64/libnsl.so

    4. libpcap binary

      • ln -s *`<location from step 1>`*/libpcap.so.<version> /lib64/libpcap.so.1

      • ln -s /lib64/libpcap.so.1 /lib64/libpcap.so

Symptom 2

The following error is observed in the HOSTMON_HOME/makelogerror file:

Undefined first referenced symbol in file
__1cG__CrunKex_dealloc6Fpv_v_       ./libhostmon19.a(Class.o)
ld: fatal: symbol referencing errors
make: Fatal error: Command failed for target `hostmonitor'

Solution 2

This error is observed when attempting to install Host Monitor Agent on Solaris 11.3 host machine. Some of the Solaris OS libraries are corrupt. Upgrade the operating system to Solaris 11.4 or contact the Solaris team for further assistance.

Host Monitor Agent Fails to Start

Learn what to do when the Host Monitor Agent fails to start.

Problem

The Host Monitor Agent network trail does not start after installation. The collection framework (collfwk) log file contains one of the following errors:

-

java.io.IOException: Cannot run program "<AgentHome>/hm/hostmonmanager" (in directory "<AgentHome>/hm"): error=13, The file access permissions do not allow the specified action.

-

HMCommandExecutor : startTrail :  binary is not found here: <AgentHome>/hm/hostmonmanager

Solution

This issue may arise due to insufficient privileges while starting Host Monitor Agent. Ensure the Audit Vault Agent user belongs to the group that owns hm (Host Monitor Agent installation) directory. Ensure that the following permissions are given:

In the event that assigning the above permissions to the group did not work, use Access Control Lists (ACL) to ensure that the following permissions are given:

Host Monitor Agent Network Trail is in STOPPED State

Learn how to fix the issue when Host Monitor Agent network trail is in STOPPED state.

Problem

After starting the Host Monitor Agent network trail it goes into a STOPPED state.

Symptom

The following error is observed in the HOSTMON_HOME/log/av.hostmonitor*.log file:

[2022-01-27 13:40:57,061] [PID: <ID>, TName: main] [WARNING]
- Failed to perform SSL handshake using TLS protocol TLS 1.2 Error Msg: SSL error:
Error in system call. Details: error:00000000:lib(0):func(0):reason(0)
Retrying with lower protocol

Solution

The Host Monitor Agent certificate is corrupt. Follow the steps in the topic Using Mutual Authentication for Communication Between the Database Firewall and the Host Monitor Agent to regenerate the certificate.

Network Audit Trail Does Not Start on Unix Platforms

Learn the resolution when the network audit trail fails to start on Unix platforms.

Problem

The network audit trail does not start on Unix platforms.

Symptoms

Solution

  1. Connect to the host machine on which the Audit Vault Agent and Host Monitor Agent are installed.

  2. In the Agent Home location there is an hm symlink pointing to Host Monitor Agent installation location.

  3. Run the following command from the Agent Home as the user who installed Audit Vault Agent:

    ls -lrt hm

  4. Verify that it’s possible to list the contents of the Host Monitor Agent installation directory.

  5. Check the permissions of all directories in the hierarchy of the path under which the Host Monitor Agent is installed.

    Note:

    The entire directory hierarchy must be owned by the root user. All of the directories in this hierarchy must have read and execute permission for other users or groups, but not write permission.

    In addition, the hostmonitor and hostmonmanager binaries should have execute access for the user who owns the Host Monitor Agent. These permissions should be granted by using an access control list (ACL).

  6. Grant the necessary permissions according to the preceding note.

  7. Restart the network audit trail.

Partial or No Traffic Seen for an Oracle Database Monitored by Oracle Database Firewall

Review the troubleshooting advice for when you see limited or no traffic for an Oracle Database that is monitored by Oracle Database Firewall.

Problem

I see no traffic, or only partial traffic, captured in reports for an Oracle Database monitored by the Database Firewall.

Solutions

Go through the following checks to find the trouble:

  1. In the Audit Vault Server, check that the report filters are set correctly, including the time slot.

  2. Check that the system time on the Database Firewall is synchronized with the time on the Audit Vault Server and the target system.

  3. Check that the target’s network traffic is visible to the Database Firewall using the Live Capture utility on the firewall.

  4. Check that the Oracle Database service name or SID is used correctly. If you specified an Oracle Database service name in the monitoring point settings for this target, you will only see traffic for that service name. To see all traffic, remove the service name from the monitoring point settings.

    If you have entered a service name in the monitoring point, and see no traffic, check to see that the service name is entered correctly in the monitoring point settings.

    For monitoring points set to use monitoring only mode, the Database Firewall may be monitoring traffic for existing client connections to the database. Since these connections were in place before you deployed the Database Firewall, it will not be able to detect the service name you specify in the monitoring point. In this case, restart the client connections to the database.

  5. Check that the correct Database Firewall policy is deployed.

    See Also:

## SAML SSO enablement fails for expired SAML signing certificate {#GUID-28C23044-0E78-4F71-85CD-0B1CE317919D}

If an SAML Signing Certificate configured in the Azure/IdP Enterprise Application expires, enabling SAML SSO on AVS fails with the following error: OAV-46599: internal error Failed to enable SSO.

You may try the following to fix this issue:

Incomplete or Missing SQL Statements or Network Traffic in Oracle DBSecCentral Reports

Learn about the probable causes that may result in missing SQL statements in Oracle DBSecCentral Reports.

Problem

Sometime there may be SQL statements missing or incomplete network traffic information in Oracle DBSecCentral Reports. This topic contains the probable causes and some tips to troubleshoot.

Symptoms

Although there may be multiple reasons that may cause this issue, the following are some of the probable causes:

Solution

Take necessary steps to resolve depending on the issue and diagnostic information. The following table contains some of the probable issues and some measures for the resolution:

   
Issue Resolution
The database client is unable to connect to the Database Firewall proxy port.
  1. Check the IP address and TCP port used by the database client to connect to the Database Firewall. Does that belong to the specific Database Firewall instance or monitoring point?
  2. Ping the Database Firewall server from the database client's host machine. In case there is a failed response, then contact the network administrator.
  3. Is there any other firewall between the database client and Oracle Database Firewall which may block the connection?
  4. Is the Database Firewall listening to the expected or configured port? Log in to the Database Firewall instance through SSH as support user and then switch to root user. Run the command netstat -alnpt | grep LISTEN | grep nnnnn where nnnnn is the port number the database client is attempting to connect. If there is no port displayed in the output, then check the proxy port settings for the configured target.
The Database Firewall deployed in Monitoring / Blocking (Proxy) mode is unable to connect to the target database.
  1. Check the IP address and TCP port of the target database. Verify if this information is properly entered when registering the target in the Oracle Database Security Central console.
  2. Is there any other firewall between the database client and Oracle Database Firewall which may block the connection?
  3. Ping the target database from the Database Firewall server. In case there is a failed response, then contact the network administrator.
  4. Log in to the Database Firewall instance through SSH as support user and then switch to root user. Access and examine the contents of the /var/log/messages file. Check if the string com.oracle.dbfw.fw ERROR - ODF-10501: Failed connecting to the Target exists. See Database Firewall Messages for more information on the error message. The message also contains additional information for the failure. For example: No route to host.
Missing SQL statements that were expected to be captured by the Database Firewall.

  1. Check if the Database Firewall monitoring point is running. In the Oracle Database Security Central console, check the status of the specific Database Firewall monitoring point.

  2. Check if the Database Firewall instance is generating the required log files. There are two ways of doing this:

    1. Check the Database Firewall policy applied to the specific target. See if the option Pass-All is selected. This does not log any traffic. In case of a custom policy, analyze and revert to Log-All policy option and check if the expected traffic data is reflecting in the Oracle DBSecCentral reports. If records are present, then examine the details of the custom policy and redefine as per the requirements.
    2. Log in to the Database Firewall instance through SSH as support user and then switch to root user. Run the following command watch ls -ltr /usr/local/dbfw/va/*/log/. Examine the logs displayed and search for the log files named kernel.nnnnnnnnnn.nnn.n.nnnn.dat.gz. In case the traffic is being logged, then these files are displayed. Later these files disappear in a few minutes as their contents are transferred to the Audit Vault Server.

  3. Check if the Oracle Database Security Central console is requesting the transfer of the traffic log files from the Database Firewall. Follow these steps:

    1. The network traffic is being logged in Database Firewall and the log files are not removed after a while. Oracle Database Security Central console may not be requesting the transfer of the traffic log files from the Database Firewall. Log in to the Database Firewall instance through SSH as support user and then switch to root user.
    2. Run the following command tail -f /var/log/httpd/ssl_request_log.

    3. Examine the output of the command. Some of the regular entries that end as follows exist:

      GET /logs/2/list HTTP/1.0" 78

      GET /logs/2/kernel.1655297472.734.0.0000.dat.gz HTTP/1.0" 1321

    4. If these entries are not displayed, then further troubleshooting in the Oracle Database Security Central console is required. Raise a bug and attach the diagnostics of both Audit Vault Server and Database Firewall.

Agent Activation Request Returns ‘host is not registered’ Error

Read the troubleshooting advice if you receive a ‘host is not registered’ error.

Problem

I used the following two commands to register the Oracle Audit Vault Agent’s host computer (where the agent is deployed), and to request Audit Vault Agent activation:

From the Oracle Database Security Central console:

avcli> register host 'host_name'

From the host computer:

agentctl activate

But the agentctl activate command returns: Agent host is not registered

Solution

Your agent host may be multi homed. In this case, the agent hostname to IP address resolution may resolve to the NIC/IP that is not used by the agent while connecting to the AV server. To resolve this issue, try to register the agent host using the with ip option and then try activating the agent again.

From the Oracle Database Security Central console, use the following command:

avcli> register host 'host_name' with ip 'host_ip_address'

If you still have issues, try finding the IP address used in the database session when you connect to the Oracle Database Security Central console from the agent host, using these commands:

Start SQLPlus* connection as sqlplus /nolog without the username or password.

In SQLPlus* run the command: connect <user>. Enter the password when prompted.

sqlplus username/password@"(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=Audit_Vault_Server_IP)(PORT=1521))(CONNECT_DATA= (SERVICE_NAME=dbfwdb)))"
sqlplus> select SYS_CONTEXT('USERENV','IP_ADDRESS') from dual;

Use the IP address from the above query to register your host.

Unable to Deploy Agent on the Secondary Audit Vault Server

Learn the resolution if you are unable to deploy an agent on a secondary Oracle Audit Vault server.

Problem

When I try to deploy the Audit Vault Agent on the secondary Audit Vault Server in a high availability pair, I get an error that the host is not registered.

Cause

After you pair two Audit Vault Servers for high availability, you do all configuration on the primary server in the pair only, including Audit Vault Agent deployment.

‘java -jar agent.jar’ Failed on Windows Machine

Review the resolution procedures when the java -jar agent.jar command fails on Windows machines.

Problem

The command java -jar agent.jar failed on my Windows target machine, and I noticed in the log files that the Audit Vault Agent services installation/un-installation failed.

Solution

  1. Follow the instructions for unregistering the agent in Registering and Unregistering the Audit Vault Agent as a Windows Service.

    If Method 1 fails, then try Method 2.

  2. Run the java -jar agent.jar command again.

Unable to Install the Agent or Generate the agent.jar File

Determine the steps to perform if you are unable to install the agent or generate the agent.jar file.

Problem

Unable to install the Audit Vault Agent. Attempts to regenerate the agent.jar file are also unsuccessful.

Solution

Follow these steps to regenerate the agent.jar file:

  1. Log in to the Audit Vault Server through SSH as user oracle.

  2. Go to the directory /var/lib/oracle/dbfw/av/conf/ location.

  3. Delete the bootstrap.prop file.

  4. Run the following command:

    /var/lib/oracle/dbfw/bin/avca configure_bootstrap

  5. Check the avca.log file that is available at /var/lib/oracle/dbfw/av/log/ to check if the above command was executed successfully.

  6. Switch the user (su) to avsys.

  7. Run the following query:

    select agent_gen_ts from file_repos where file_name='agent.jar';

  8. The above query displays the current time in case the agent.jar file is generated successfully.

Unable to Un-install the Oracle Audit Vault Agent Windows Service

Review the troubleshooting advice if you are unable to un-install the Oracle Audit Vault Agent Windows Service.

Follow the instructions for unregistering the Agent in Registering and Unregistering the Audit Vault Agent as a Windows Service.

If Method 1 fails, then try Method 2.

Access Denied Error While Installing Agent as a Windows Service

Learn how to resolve access denied errors when installing Oracle Audit Vault agent as a Windows service.

Problem

I got an error during installation of Oracle Audit Vault Agent on Windows, and I noticed the following error in the AGENT_HOME\av\log\av.agent.prunsvr log file:

[2013-05-02 11:55:53] [info] Commons Daemon procrun (1.0.6.0 32-bit) started
[2013-05-02 11:55:53] [error] Unable to open the Service Manager
[2013-05-02 11:55:53] [error] Access is denied.
[2013-05-02 11:55:53] [error] Commons Daemon procrun failed with exit value:
7 (Failed to )
[2013-05-02 11:55:53] [error] Access is denied.

Solution

The above message means that the logged in user does not have privileges to install the Audit Vault Agent as a Windows Service. If you get the above message, try launching the command shell with the Run As Administrator option, and then run java -jar agent.jar in that command shell.

Unable to Start the Agent Through the Services Applet on the Control Panel

Review how to resolve being unable to start the agent through the services applet on the control panel.

Problem

I did the following:

  1. Installed the Audit Vault Agent using the java -jar agent.jar command.

  2. Activated the Audit Vault Agent.

  3. Started the Audit Vault Agent using the agentctl start -k key command.

    The agent started up and is in RUNNING state.

  4. Stopped the Audit Vault Agent.

  5. Tried to start the Audit Vault Agent using the Services Applet on the Windows Control Panel.

    The Audit Vault Agent errored out immediately.

Solution

This means that the Audit Vault Agent is configured to use a Windows account that does not have privileges to connect to the Audit Vault Server.

Take the following steps:

  1. Go to Control Panel, then to Services Applet.

  2. Select the Oracle Audit Vault Agent service.

  3. Right select and select the Properties menu.

  4. Select Log on.

  5. Select This account: and then enter a valid account name and password.

  6. Save and exit.

  7. Start the Audit Vault Agent through the Services Applet.

Error When Starting the Agent

Resolved errors that occur when starting the agent.

Problem

After I installed the Audit Vault Agent, I set the username and password in the OracleAVAgent Windows Service Properties Log On. However, when I try to start the OracleAVAgent service, I see the following error in the Agent_Home\av\log\av.agent.prunsvr.date.log file:

[info]  Commons Daemon procrun (1.0.6.0 32-bit) started
[info]  Running 'OracleAVAgent' Service...
[info]  Starting service...
[error] Failed creating java
[error] ServiceStart returned 1
[info]  Run service finished.
[info]  Commons Daemon procrun finished

Solution

This means that the OracleAVAgent service is not able to launch the Java process. Try the following:

  1. Uninstall all JDKs and/or JREs in the system.

  2. Reinstall JDK SE or JRE and then start the OracleAVAgent service.

  3. If this doesn’t help, you can install 32 bit JDK SE or JRE and then start the OracleAVAgent service.

Alerts on Oracle Database Targets Are Not Triggered for Extended Periods of Time

Learn what to do when alerts on targets are not triggered for a long time.

Problem

I configured an Oracle Database target to audit to XML files, configured an audit trail in Oracle DBSecCentral of type DIRECTORY, and then configured an alert to trigger on certain events. My alert did not get triggered for a long time.

Solution

This issue can occur if the Oracle Database target is not flushing the audit records to the file immediately. Contact Oracle Support in order to access support note 1358183.1 Audit Files Are Not Immediately Flushed To Disk.

Error When Creating an Audit Policy

Resolve errors that can occur when you create an audit policy.

Problem

I received this error message when I tried to create a new audit policy setting for Oracle Database:

-ORA-01400: cannot insert NULL into ("AVSYS"."AUDIT_SETTING_ARCHIVE_MAP"."ARCHIVE_ID")

Cause

The Oracle Database must have at least one audit policy setting before you can create and provision new audit settings using Oracle Database Security Central. Oracle Database comes with a predefined set of audit policy settings. You must not manually remove these settings. If the audit settings have been removed, then you can manually create at least one audit setting in the Oracle Database. Then try again to create new audit settings using Oracle Database Security Central.

See Also: Oracle Database Security Guide for detailed information on Oracle Database auditing.

Connection Problems When Using Oracle Database Firewall Monitoring and Blocking

Resolve the connection problems that might occur when using Oracle Database Firewall monitoring and blocking.

Problem

In monitoring and blocking mode, my client application cannot connect to the target database.

Solution

  1. Log in as root on the Database Firewall server.

  2. Run this command using the target database IP address or host name:

    ping -I <secured_target_ip_address_or_hostname>

    If you do not receive a response, then ensure that the DNS is configured on Oracle Database Firewall.

    If a response is received, check:

Audit Trail Does Not Start

Learn the resolution to use when the audit trail does not start.

Problem

An audit trail does not start. For example, in the Oracle Database Security Central console, in the Audit Trails page, the Collection Status column indicates that the trail is Stopped or Unreachable.

Solution

When a trail does not start, you can show the associated error in two ways:

If the cause is still unclear, or the grep command returns no results, raise an SR with Oracle Support and include Audit Vault Agent log files.

Cannot See Data for Targets

Learn what to do when you cannot see the data for a target.

Problem

Data for my Target does not appear on reports.

Solution

If you cannot see the data you expect to see in the Audit Vault Server, you can troubleshoot by trying one or more of the following:

Problems Pairing Oracle Database Firewall and Oracle Audit Vault Server

Review the procedure to follow when you have problems pairing Oracle Database Firewall with Oracle Audit Vault Server.

Problem

I encounter errors when I try to associate a Database Firewall with the Audit Vault Server.

Solution

Check the following:

User Names Do Not Appear on Database Firewall Reports

Learn what to do when names do not appear on Database Firewall reports.

Problem

When I generate a Database Firewall report, I do not see user names.

Solution

Check the following possibilities:

Note: Sometimes unknown_username is displayed in the User field of Database Firewall reports for SQL server. This can be resolved by enabling Retrieve session information from target DB option under the Advanced for the Database Firewall monitoring point. The report may also display unknown_osusername in the OS User field of Database Firewall reports for SQL server. This information is available to Database Firewall only if the client uses Windows authentication or a trusted connection.

See Also:

Alerts Are Not Generated

Review the resolution to use when alerts that you created are not generated.

Problem

Alerts I have created are not being generated.

Solution

Try the following:

See Also:

Problems Retrieving or Provisioning Audit Settings on Oracle Target

Learn what to do when you encounter problems while retrieving or provisioning Oracle target audit settings.

Problem

I have a problem either retrieving audit settings form an Oracle Database target, or provisioning audit settings to an Oracle Database target.

Solution

If you have problems retrieving audit settings, try the following:

If you have problems provisioning audit settings, and the Oracle Database target has Database Vault enabled, confirm that the Oracle Database Security Central user you created on this database has the AUDIT SYSTEM and AUDIT ANY privileges.

Operation Failed Message Appears When Attempting to Enable Oracle Database Security Central Policies

Learn how to resolve operation failures when you try to enable Oracle Database Security Central policies.

Problem

I Oracle Database Security Central configured for a backup and restore operation. After I completed the procedure, I could not enable an Oracle Database Security Central policy. The error message Operation failed. Please contact Oracle Support appeared.

Solution

During the backup and restore process, Oracle Database Security Central must perform a restart of the Oracle Audit Vault Server database. The internal tool Java Framework may need to be restarted. To remedy this problem:

  1. Log in to Oracle Audit Vault Server.

  2. At the command line, run the following command to check the status of the Java Framework:

    /usr/local/dbfw/bin/javafwk status

  3. If the output says Java framework process is stopped, then restart it as follows:

/usr/local/dbfw/bin/javafwk start

Out of Memory Error Message During Restore

Learn the resolution when you receive an out of memory error message during a restore.

Problem

Encounter out of memory error while performing restore task.

Solution

Prior to initiating the restore task, ensure that the RAM size and Disk size in the new system is equal or bigger than the original system. This ensures that the out of memory error is not encountered while performing the restore task.

JAVA.IO.IOEXCEPTION Error

Learn how to resolve a JAVA.IO.IOEXCEPTION error.

Problem

SSL peer shuts down incorrectly with the following error:

JAVA.IO.IOEXCEPTION: IO ERROR:SSL PEER SHUT DOWN INCORRECTLY

Solution

  1. Access the target through SSH.

  2. Change to the following location using the command:

    cd $ORACLE_HOME/network/admin

  3. Edit the sqlnet.ora file. Add parameter sqlnet.recv_timeout=100000 in the file.

  4. Restart the target listener.

  5. Once the target listener is started, start the agent, and the audit trail.

Failed to Start ASM Instance Error

Learn what to do when you receive a Failed to start ASM instance error.

Problem

The avdf-upgrade --confirm command stops and results in an error. The command may fail for many reasons. The error mainly occurs due to failure in starting or stopping of a service.

The following is an example of Failed to start ASM instance error:

}

Solution

Rerun the command avdf-upgrade --confirm

Executing this command again will get past the Failed to start ASM instance error.

Internal Capacity Exceeded Messages Seen in the /var/log/messages file

Learn how to resolve Internal capacity exceeded messages that appear in the /var/log/messages file.

Problem

Not all the expected traffic is being captured or logged by the Database Firewall, and error messages are present in the /var/log/messages file containing the text Internal capacity exceeded.

Solution - 1

Increase the processing resources available for the target on which the issue is observed through the setting of the MAXIMUM_ENFORCEMENT_POINT_THREADS collection attribute.

See Also: Registering Targets

Solution - 2

The size of the buffer used for inter-process communication on the Database Firewall can be increased to improve throughput, though at the cost of more memory being allocated by the relevant processes. Please note that this setting is in units of Megabytes, and has a default value of 16. To change the configuration for this value perform the following steps:

  1. Log in to the Oracle Database Security Central console as the root user.

  2. Edit the file /usr/local/dbfw/etc/dbfw.conf. Look for an entry with the key IPC_PRIMARY_BUF_SIZE_MB. If it exists, this is the line to change. If it does not exist, add a new line beginning with IPC_PRIMARY_BUF_SIZE_MB.

  3. Change the IPC_PRIMARY_BUF_SIZE_MB line to reflect the required buffer size. For example, if you wished to change the buffer size to 24 megabytes, the configuration line should be IPC_PRIMARY_BUF_SIZE_MB="24". Save the changes.

  4. From the command line restart the Database Firewall processes so that the new setting is used with the command line /usr/local/dbfw/bin/dbfwctl restart.

There is also a second setting available to alter the maximum size that the inter-process communication buffer can grow to. It’s units are in megabytes, and has a default value of 64 megabytes. To change the configuration for this value, perform the following steps:

  1. Log in to the Oracle Database Security Central console as the root user.

  2. Edit the file /var/dbfw/va/N/etc/appliance.conf, where N is the number of the Database Firewall monitoring points in question. Look for an entry with the key IPC_BUF_SIZ_MB. If it exists, this is the line to change. If it does not exist, add a new line beginning with IPC_BUF_SIZ_MB.

  3. Change the IPC_BUF_SIZ_MB to reflect the desired maximum buffer size. For example, if you wished to change the buffer size to 80 megabytes, the configuration line should be IPC_BUF_SIZ_MB="80". Save the changes.

  4. From the command line restart the Database Firewall processes so that the new setting is used with the command line /usr/local/dbfw/bin/dbfwctl restart.

If the problem persists and after altering the above settings the Internal capacity exceeded error is still encountered, then further investigation by support is required.

Perform the following:

  1. Log in to the Oracle Database Security Central console as the root user.

  2. Edit the file /usr/local/dbfw/etc/logging.conf

  3. Find the line log4j.logger.com.oracle.dbfw.Metrics=ERROR

  4. Comment out this line by placing a # character at the beginning of the line log4j.logger.com.oracle.dbfw.Metrics=ERROR. Save the changes.

  5. From the command line restart the Database Firewall processes so that the new setting is used with the command line /usr/local/dbfw/bin/dbfwctl restart

  6. Leave the Database Firewall running for several hours under load even while the Internal capacity exceeded error is still encountered.

  7. After this period, get the diagnostics output from the Database Firewall as detailed in MOS note How to Collect Diagnostic Logs From Audit Vault Server (Doc ID 2144813.1). Provide the diagnostics output to support for further analysis.

First Archive Or Retrieve Job After Upgrade

Learn what to do if after an upgrade, the first archive or rervireve job submission displays the status of Starting.

Problem

After upgrade the first archive or retrieve job submission may display the status as Starting.

Solution

Submit the job again. This is a known issue and subsequent submission of job succeeds.

Audit Vault Agent Installation Fails After HA Pairing Or Separation

Learn what to do after the Oracle Audit Vault installation fails after an HA pairing or separation.

Problem

Installation of Audit Vault agent fails after performing pairing or separation (un-pairing) of Oracle Audit Vault server.

The following command generates agent debug logs during agent installations.

java -jar agent.jar -v

Symptoms

The following errors may be found during agent installation in the agent log file:

PKIX path validation failed

signature check failed

Solution

After the pairing or separating of Oracle Audit Vault servers, you must download the Audit Vault agent from the GUI and install the agent again after removing the existing Audit Vault Agent.

See Also: Post High Availability Pairing Steps

If the Audit Vault agent fails to install after pairing or separating of Audit Vault server, then install the Audit Vault agent using -v option.

To resolve the above errors, follow the steps mentioned below:

  1. Log in to the Audit Vault server as user root.

  2. Run the following script to generate a new agent.jar file.

    /usr/local/dbfw/bin/priv/update_connect_string_ip.sh

  3. Download the new agent.jar file from the GUI.

  4. Install the newly downloaded agent.jar file.

Error in Restoring Files

Learn what to do when you encounter errors while restoring files.

Problem

An attempt to restore the data files results in a failure. The restore job completes successfully, however the data files are not restored. There is no information in the restore job log file.

Solution

Check for the following to troubleshoot the issue:

DB2 Collector Fails Due to Source Version NULL Errors

If the DB2 collector fails due to source version NULL errors, then follow these steps.

Problem

The following error or trace is displayed in the collector log file.

Caused by: java.lang.ClassNotFoundException:

sun.io.MalformedInputException

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

Solution

Check the Java version on the host system This failure is due to Java SE version 8. Attempt to use Java SE 7.

Note: This issue may be encountered in releases prior to 12.2.0.11.0.

DB2 Collector Fails Due to Database Connection or Permission Issues

If the DB2 collector fails due to database connection or persmission issues, then follow these steps.

Problem

The following error or trace is displayed in the collector log file.

Caused by: oracle.ucp.UniversalConnectionPoolException: Cannot get Connection from Datasource: java.sql.SQLSyntaxErrorException: [Audit Vault][DB2 JDBC Driver][DB2]<User> DOES NOT HAVE PRIVILEGE TO PERFORM OPERATION EXECUTE ON THIS OBJECT NULLID.DDJC360B

Solution

Run the following command for successful execution of DB2 collector:

grant execute on package NULLID.DDJC360B to <User> (user while registering the target)

ORA-12660 Error While Registering Target

Learn how to resolve the ORA-12660 error.

Problem

Audit Vault agent fails with ORA-12660 error.

Solution

The server encryption is set to REQUIRED in on-premises by default. Set the server encryption to ACCEPTED or REQUESTED or REJECTED.

Note: REJECTED is not a recommended option. The following table describes these options in detail.

Table 1 Server Encryption Types

Option Description
ACCEPTED The server does not enable both encrypted and non-encrypted connections. This is the default value in case the parameter is not set.
REJECTED The server does not enable encrypted traffic.
REQUESTED The server requests encrypted traffic if it is possible, but accepts non-encrypted traffic if encryption is not possible.
REQUIRED The server accepts only encrypted traffic.

Audit Trail Performance Issues Occur After Audit Vault Server Upgrade

Learn what to do when audit trail performance issues occur after upgrading Oracle Audit Vault Server.

Problem

You might experience audit trail performance issues after upgrading Oracle Audit Vault Server.

Solution

The audit_trail_id_idx index that is created resolves the performance issues encountered. However, you must retain sufficient disk space if there is large amount of event data for the period prior to upgrading Oracle Audit Vault Server. The amount of disk space required is about 5% of the total event log data size.

Failures Due to Dropping Users

Learn how to resolve failures that occur when dropping users.

Problem

Failed to drop the user with an error message and the user was not listed in the Audit Vault Server GUI.

Solution

Contact Oracle Support for the best workaround and to drop the user manually using SQL*Plus.

Failure of Agent Automatic Upgrades

Learn what to do when agent automatic upgrades fail.

Problem

The automatic upgrade of the Agent fails with the following error. This is because the Agent is unable to connect to the Audit Vault Database.

Message: Exception occurred while updating Agent.
Cause: Unable to connect to AV Server.
Note: Agent will try to re-connect automatically in 10 seconds.

Solution

The Agent attempts to connect to the Audit Vault Database and auto upgrade after 10 seconds. Check the Oracle Audit Vault Database connection or contact Oracle Support.

Some Services May Not Start After Backup

Learn what to do when services fail to start after a backup.

Problem

The system may not be stable after a cold backup operation failed to complete.

Solution

Oracle recommends that you reboot the system if there is a failure while performing a cold backup operation.

Data Overflow Issues in the Oracle Audit Vault UI

Learn how to resolve data overflow issues in the Oracle Audit Vault UI.

Problem

The Recently Raised Alerts Report region appears on your dashboard and displays the list of alerts with data overflowing in the Audit Vault GUI. This may occur when you launch the GUI using Internet Explorer and the Microsoft Windows Server operating system.

Solution

To fix this issue and to display the data properly on the Audit Vault GUI, you should make minor changes to the Internet Explorer browser settings. Press F12 and select Emulation.

Change the Document mode and Browser profile fields from the default settings. For example, change the Document mode value to 10 from the drop down menu and change the Browser profile field to Desktop.

Oracle Audit Vault Agent is Unreachable and the Transaction Log Audit Trail is Frozen in Starting Status

Learn what to do when the Oracle Audit Vault Agent is unreachable and the transaction log audit trail is frozen in Starting status.

Problem

The status of Oracle Audit Vault Agent is unreachable from the AV GUI. The status of the Transaction Log audit trail persistently remains in the Starting status.

This may be due to a user application that is blocking the creation of streams by ORAAUDIT user.

Symptom

The Transaction Log audit trail does not start. The following information may be found in the thread dump that is taken using jstack tool:

oracle.av.platform.agent.collfwk.impl.redo.RedoCollector.sourceSetup(RedoCollector.java:634)

Solution

Terminate the user application that is blocking the creation of streams. Restart the Transaction Log audit trail.

Scheduled PDF or XLS Reports Result in a Hung State

To resolve a hung state that occurs for scheduled PDF or XLS reports, follow these recommendations.

Problem

Scheduled PDF or XLS reports remain incomplete for an extended period of time or ramin in q RUNNING state.

Solution

You can schedule reports to be sent to other users in PDF or XLS formats. Avoid triggering or scheduling concurrent long-running reports at the same time. Producing PDF and XLS reports occupies a lot of system resources because there is a significant amount of data involved. Scheduled concurrent long-running reports can remain in a hung state indefinitely. The reports must be scheduled with staggered intervals in between. For example, run the reports at intervals of 5, 10, or 20 minutes.

Pending Reports Remain in Scheduled Status

To resolve pending reports that remain in scheduled status, follow these steps.

Problem

Many reports are stuck in scheduled or pending status. These reports may never be completed and may be stopped.

Solution

This may be due to an issue with the Java Framework process in the background. Use these steps to check and resolve this issue:

  1. Log in to the CLI as support user.

  2. Switch to root user using the command:

    su root

  3. Run the following command to check the status of the Java Framework:

    systemctl status javafwk

  4. Stop the Java Framework even if it is running. Run the following command:

    systemctl stop javafwk

  5. Run the following command to start the Java Framework:

    systemctl start javafwk

  6. Run the following command to restart the Java Framework:

    systemctl restart javafwk

Use the following procedure to check the status of the reports from the operating system logs after running one of the procedures mentioned above and restarting the Java Framework:

  1. Log in to AVCLI as admin user.

  2. Run the following command to enable diagnostics for the reports:

    ALTER SYSTEM SET loglevel=ReportLog:DEBUG|JfwkLog:DEBUG;

  3. The diagnostics can also be enabled using the Oracle Oracle Database Security Central console by following these steps:

    1. Log in to the console as admin user.

    2. Select Settings.

    3. Select Diagnostics on the left navigation menu.

    4. Select Debug against Report Generation.

    5. Select Save.

  4. Run a PDF report. For example, Activity Overview.

    1. Log in to the Oracle Oracle Database Security Central console as auditor.

    2. Select Reports.

    3. Select Activity Reports under Built-in Reports.

    4. In the Activity Reports on the screen, you can schedule a report and view the generated report.

  5. After a while, check on the /var/lib/oracle/dbfw/av/log file. For example, av.report* file. It contains the PDF/XLS report generation debug logs.

Error OAV-47409 While Managing Archive Locations

Learn what to do when you receive the OAV-47409 error while managing archive locations.

Problem

The following error message displays in the Auto Archive Message column under Manage NFS Locations:

OAV-47409: Absolute path does not exist on remote filesystem
ORA-06510: PL/SQL: unhandled user-defined exception

The configured path of the archive location is either missing or outside of the remote filesystem.

Cause

The NFS export directory configured on the DBSecCentral server did not have read/write permissions assigned to the Oracle user.

Solution-1

The value under Auto Archive Order column is set to 0 [zero]. The system has set this value as the archive location is problematic. You must ensure that the NFS location issue is resolved to a valid directory on the remote filesystem. Upon resolving this issue, set the value under Auto Archive Order column to 1 or higher. This sets the appropriate priority for the auto archive order.

Solution-2

Check if the owner of the NFS archive location is oracle:oinstall.

  1. If not, run the following commands:

    # chown oracle:oinstall /archive/archive

  2. Change the directory to the archive location:

    # cd /archive

  3. On the AV server, run the following command to check the existence and the permissions:

    # ls -lrth
    drwxrwxrwx. 2 root   root      16K Jan  2 15:54 lost+found
drwxrwxr-x. 2 oracle oinstall 4.0K Jan  4 11:30 archive

  4. Use the pwd command to know the present working directory and change to /archive/archive directory by using the following command:

    # cd archive

    Now try to configure the NFS, it is expected to function correctly.

Error OAV-47402 While Defining Archive Locations Using NFS Mount Point

Learn what to do when you receive the OAV-47402 error while defining archive locations.

Problem

An error is observed after registering the archive location using NFS mount point through AVCLI. The created remote file system shows inaccessible when running the SHOW STATUS command. The following error is observed when running ALTER REMOTE FILESYSTEM <file system name> MOUNT command. However, the process of defining or creating the archive location is successful.

OAV-47402: Unable to mount export /exabackup from host <host Ip address>

Solution

This issue is observed when using NFS version v3 only. Reach out to the NAS storage support or NFS administrator support team to verify if the mount point in the NFS server is properly configured. It must support both v3 and v4 to integrate with Oracle DBSecCentral.

Follow the steps documented in My Oracle Support Doc ID 2232033.1 to verify if the mount point in the NFS server is properly configured.

See Defining Archive Locations for complete information.

Audit Trail Stopped After Relocating Windows Event Log Files

Use this procedure when the audit trail stops after you relocate the Windows event log files.

Problem

Windows event log relocation causes audit trail to be stopped.

Solution

Follow this procedure to resolve this problem:

  1. Stop the audit trail.

  2. Drop the audit trail.

  3. Restart the audit trail. The new trail recognizes the new location for event logs.

Missing or Incomplete Client Information in Oracle Database Firewall Logs

Learn how to resolve missing or incomplete client information in Oracle Database Firewall logs.

Problem

Empty client information in the Oracle Database Firewall logs after upgrading Oracle Database Security Central. The logs that are generated are missing some of the client information such as the user name.

Note: This issue occurs only when you are in DAM mode deployment of Oracle Database Firewall. You will not experience this issue in the Proxy mode deployment.

Cause

Oracle Database Firewall records information that is related to the TCP sessions during inspection and it saves this data to disk. This recorded information includes client user names and other metadata about the connection. When Oracle Database Firewall processes are restarted after a configuration change or an upgrade, Oracle Database Firewall continues to generate logs accurately by re-reading this cached information.

The format of the cache file has changed in the recent releases. Oracle Database Firewall may not be able to read the file in the old format. Therefore, existing client connections to the database that were established before performing the upgrade may not retain certain information such as client user names. This can lead to logs missing information such as the client username.

Solution

Restart the database clients.

Issues with Retrieving Session Information Through Clients Connecting to Microsoft SQL Server

Learn what to do when you have issues retrieving session information through clients that connect through Microsoft SQL Server.

Problem

Database Firewall is unable to retrieve session information through some clients (for example, MS SQL Server Management Studio) as the information is encrypted. You can retrieve session information for non Oracle databases to obtain the name of the database user, operating system, and client program that originated a SQL statement.

Symptom

Audit Reports show unknown user names and unknown program names where the target is Microsoft SQL Server.

Solution

Ensure the following steps are accurate while registering Microsoft SQL Server as a target.

  1. In the User Name field, enter the user name of the system administrator.

  2. In the Password field, enter the password of the system administrator.

  3. In the Host Name / IP Address field, enter the IP address of the SQL Server.

  4. In the Port field, enter the port of the SQL server listening port.

  5. In the Service Name field, enter a valid database service name on SQL Server. In case the database service name is not correct, then SQL server DDI requests fail on the SQL Server with invalid request error.

Note: If the secured type is not Oracle, then the Service Name field must be empty. This field is designated for a specific Oracle Service Name (OSN) and is not applicable to any other database type. If this field is not blank, then no traffic will be recorded, as per the reported symptoms.

See Also:

Performance Issues Due to High Memory Usage

Learn how to address performance issues in Oracle DBSecCentral with very large deployments.

Problem

Audit Vault Server in large deployments may have performance issues due to increased memory usage.

Solution

httpd Crash Issue on Database Firewall

Learn how to fix httpd crash issue in Database Firewall.

Problem

The httpd process in Database Firewall may crash under some circumstances.

Symptom

The status of the Database Firewall instance appears Down in the Oracle Database Security Central console. The Database Firewall logs are not transferred to the Audit Vault Server.

The following is observed in the log files of the impacted Database Firewall instance. The httpd.service file in /etc is symlinked to the file in /usr path.

# ls -l /etc/systemd/system/multi-user.target.wants/httpd.service

lrwxrwxrwx. 1 root root 37 Nov 27 09:26 /etc/systemd/system/multi-user.target.wants/httpd.service -> /usr/lib/systemd/system/httpd.service

# ls -lL /etc/systemd/system/multi-user.target.wants/httpd.service

-rw-r--r--. 1 root root 752 Nov 10 20:33 /etc/systemd/system/multi-user.target.wants/httpd.service

#

Solution

Follow these steps to change the configuration of the system and restart the httpd process:

  1. Log in to the Database Firewall instance as root user.

  2. Check and confirm that the above mentioned symptom exists.

  3. Copy the base file from /usr to /etc by running the following command:

    # install -m 0644 -o root -g root /usr/lib/systemd/system/httpd.service /etc/systemd/system/httpd.service

  4. Edit the file in /etc and find the below mentioned Service block:

    # vi /etc/systemd/system/httpd.service

    ...

    [Service]

    Type=notify

    EnvironmentFile=/etc/sysconfig/httpd

    ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND

    ...

  5. Modify the file and add the following code to include the restart failure directive. The file looks like the folllowing:

    ...
[Service]
Restart=on-failure
Type=notify
EnvironmentFile=/etc/sysconfig/httpd
ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND
...

  6. Save the file.

  7. Disable and re-enable the service to fully apply the following changes:

    # systemctl disable httpd
Removed symlink /etc/systemd/system/multi-user.target.wants/httpd.service.
# systemctl enable httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /etc/systemd/system/httpd.service.
#

  8. Verify the following changes:

    # sha256sum -c - <<EOF
eac607c17f2c122619b3e1459eafdfef6bde003d24964891aa506735df4f55c2  /etc/systemd/system/multi-user.target.wants/httpd.service
EOF
/etc/systemd/system/multi-user.target.wants/httpd.service: OK
#

  9. Reload the systemd configuration and restart httpd by running the following commands:

    # systemctl daemon-reload
    # systemctl restart httpd

  10. Verifying the service is enabled by running the following command:

    # systemctl list-unit-files | grep http

  11. Observe the following output:

    httpd.service enabled

    #

  12. If the daemon subsequently fails, the systemd will restart it, and write the following example audit trail to the system log:

    Nov 27 08:38:09 example systemd: httpd.service: main process exited, code=killed, status=11/SEGV

    Nov 27 08:39:40 example systemd: httpd.service stop-sigterm timed out. Killing.

    Nov 27 08:39:40 example systemd: Unit httpd.service entered failed state.

    Nov 27 08:39:40 example systemd: httpd.service failed.

    Nov 27 08:39:40 example systemd: httpd.service holdoff time over, scheduling restart.

    Nov 27 08:39:40 example systemd: Stopped The Apache HTTP Server.

    Nov 27 08:39:40 example systemd: Starting The Apache HTTP Server...

    Nov 27 08:39:40 example systemd: Started The Apache HTTP Server.

Issue with Retrieval of Return Row Count

Learn how to fix the issue related to retrieval of return row count.

Problem

Database Firewall captures the number of rows returned by a SELECT query and display them in reports under the column Row Count.

If the database takes a while to generate response result set, then return row count may not be extracted due to timeout configuration.

Workaround

Follow these steps to adjust the timeout interval:

  1. Log in to the Database Firewall through SSH and switch to the root user.

    See Logging In to Oracle DBSecCentral Appliances Through SSH.

  2. Change to /var/dbfw/va directory.

  3. Identify the Database Firewall monitoring point by searching for the target name configured in the Audit Vault Server. Run the following command:

    grep -lr <TARGET NAME> *

  4. Find the monitoring point number from the output which contains the name and path of the configuration file. For example: 1/etc/appliance.conf. In this example, 1 is the monitoring point number.

  5. Change the directory to the identified monitoring point and open configuration file of the appliance.

  6. Search for the following entry in the file:

    MAX_LOG_FILE_TIMERANGE

  7. Modify the MAX_LOG_FILE_TIMERANGE line to reflect the required time range in seconds. For example, if you wish to change the time range to 5 minutes, then the configuration line should be MAX_LOG_FILE_TIMERANGE=="300".

  8. Save the changes.

  9. Run the following command to restart the Database Firewall processes so that the new setting takes effect:

    /usr/local/dbfw/bin/dbfwctl restart <monitoring point number>

    In this case the monitoring point number was 1.

    Hence, the command should be:

    /usr/local/dbfw/bin/dbfwctl restart 1

Note: Increasing the timeout configuration delays the availability of captured SQL statements in the reports and any alerts configured for the same. Use your discretion while configuring the above value close to the actual query completion time.

Unable to Log in to the Oracle DBSecCentral Appliance through SSH

Learn how to fix log in issue to Oracle DBSecCentral appliance.

Problem

The user is unable to log in to the Oracle DBSecCentral appliance through SSH. This may be because of using old SSH clients to log in to the Oracle DBSecCentral appliance.

Workaround

Log in to ARU (Automated Release Updates). Apply the patch number 32287150 that solves the problem.

Error When Changing IP Address of Management Interface

Learn how to resolve the error encountered when changing the IP address of the Management Interface.

Problem

The Management Interface IP address is the IP address of the Database Firewall which was used to register the Database Firewall in the Oracle Database Security Central console.

In Oracle AVDF 20.1, the following error may be encountered while attempting to change the IP address of the Management Interface:

Operation failed OAV-46981: Unable to connect to Database Firewall with IP <ipaddress>

Solution

This error may come up because the IP Address of the Database Firewall is changed successfully. However, there may be a delay in the response from Database Firewall. It may take a few seconds for the network update on the Database Firewall and for the system to settle.

Select Save and Close buttons to exit the dialog. Do not select cross (X) mark in the top right corner of the dialog.

Transaction Log Audit Trail Stops Due to an Error While Parsing XML File Containing Emoji

Problem

Transaction Log audit trail stops while parsing a file that contains emoji. The following error is observed in the Agent logs:

javax.xml.stream.XMLStreamException: ParseError at [row,col]

Solution

Follow these steps to resolve this error:

  1. Run the following command to stop the Audit Vault Agent:

    AGENT_HOME/bin/agentctl stop

    .

  2. Delete the sjsxp.jar file present in the AGENT_HOME/av/jlib directory.

  3. Run the following command to start the Audit Vault Agent:

    AGENT_HOME/bin/agentctl start

    .

Unable to Find the FIPS Status for Database Firewall Instance

Learn how to fix the error when the FIPS status for a Database Firewall instance is not displayed in the Oracle Database Security Central console.

Problem

The FIPS status for the Database Firewall instance could not be determined from the Oracle Database Security Central console.

Solution

Perform the following checks to determine the root cause of the problem:

If none of the above points are helpful in identifying the cause of the problem, then contact Oracle Support.

Unable to Modify the Database Firewall FIPS Mode Through Oracle Database Security Central Console

Learn how to fix the error when the FIPS mode cannot be modified through the Oracle Database Security Central console.

Problem

This could be caused due to a communication issue between the Audit Vault Server and the Database Firewall instances.

Solution

Perform the following checks to determine the root cause of the problem:

If none of the above points are helpful in identifying the cause of the problem, then contact Oracle Support.

The FIPS Status on Both the Database Firewall Instances is Different

Learn how to fix the error when the FIPS mode is different on both the Database Firewall instances.

Problem

The FIPS mode is different on both the Database Firewall instances. This could be caused when FIPS mode is manually changed on one of the Database Firewall instances. It can also be caused when such an attempt to manually change the FIPS mode failed.

Solution

All the Database Firewall instances that are part of high availability must have the same FIPS 140-2 mode. The FIPS 140-2 status of the Database Firewall instances must either be Off or On.

FIPS 140-2 mode can be disabled or enabled on both the Database Firewall instances. In case, these two instances have different FIPS mode, then an error message is displayed on the screen.

Verify the high availability status of the Database Firewall instances, and change the FIPS mode again.

After Restarting Secondary Audit Vault Server, the Primary Instance Fails to Switchover

Learn how to fix a switchover issue on the primary Audit Vault Server, after the secondary instance is restarted.

Problem

After restarting the secondary Audit Vault Server, the switchover status of the primary Audit Vault Server shows NOT ALLOWED state.

This status of the primary Audit Vault Server is not recoverable and the following error messages appear and are repeated every 50 seconds on the secondary Audit Vault Server:

<Date> <avs-instance-name> observerctl: com.oracle.avs.observerctl DEBUG - DGMGRL:[W000 <date and timestamp>] The primary database has requested a transition to the UNSYNC/LAGGING state with the standby database DBFWDB_HA2.
<Date> <avs-instance-name> observerctl: com.oracle.avs.observerctl DEBUG - DGMGRL:[W000 <date and timestamp>] Permission granted to the primary database to transition to LAGGING state with the standby database DBFWDB_HA2.
<Date> <avs-instance-name> observerctl: com.oracle.avs.observerctl DEBUG - DGMGRL:[W000 <date and timestamp>] Reconnect interval expired, create new connection to primary database.
<Date> <avs-instance-name> observerctl: com.oracle.avs.observerctl DEBUG - DGMGRL:[W000 <date and timestamp>] The primary database has been in LAGGING state for 7138 seconds.

Solution

In case the primary Audit Vault Server’s switchover status goes into NOT ALLOWED status after restarting the secondary instance, then follow the steps mentioned in MOS Note (Doc ID 1258074.1) to restart the standby Audit Vault Server.

Incorrect Syntax Near Connectivity Entry in Audit Logs

Learn how to fix incorrect syntax error entry in audit logs.

Problem

When attempting to add an audit trail for Microsoft SQL Server, the Audit Vault Agent attempts to acquire a target connection using JDBC driver. After the connection is established, a test query is sent to validate the connection by the JDBC driver.

This test query may generate the following error:

Incorrect syntax near 'Connectivity'

This error is visible in the database audit records.

Solution

To avoid unnecessary logging of records or events due test queries in the target database, define the collection attribute as follows:

av.collector.validateConnectionOnBorrow = false

Certificate Regenerate Failure Error

Learn how to fix a certificate regenerate failure error.

Problem

In case the certificate regenerate operation fails, then one of the possible reasons can be the incorrect date and time of the appliance (Audit Vault Server or Database Firewall).

Solution

Specify the correct time, and then run the following command to regenerate the certificate:

/usr/local/bin/gensslcert create-certs

To retrieve the details about certificate expiry date, run the following command:

openssl x509 -enddate  -startdate -noout -in {certificate path}

For example:

openssl x509 -enddate  -startdate -noout -in /usr/local/dbfw/etc/cert.crt
notAfter=Oct 17 17:44:53 2022 GMT
notBefore=Sep 14 17:44:53 2021 GMT

Note: The audit trails go to UNREACHABLE state for about 45 minutes after the certificates are rotated and all the relevant services are restarted. The trails continue to work normally after that. This behavior is observed in Oracle AVDF release 20.6 only.

User Entitlement, Audit Policy, or Firewall Policy Job Stuck in Running State

Learn how to manage the user entitlement, audit policy or firewall policy job stuck in RUNNING state.

Problem

A user entitlement job, audit policy job, or firewall policy job is stuck in the RUNNING state for a long time. The job is not progressing and must be manually stopped.

Workaround

This issue may be due to an issue with the Java Framework process in the background. Follow these steps and submit the job again:

  1. Log in to the Audit Vault Server as support user through SSH.

  2. Switch to root user by running the following command:

    su root

  3. Restart the Java Framework by running the following command:

    systemctl restart javafwk

Displaying Job Status Takes Lot of Time in the Oracle Database Security Central Console

Learn how to resolve the Jobs dialog issue.

Problem

The Jobs dialog in the System takes lot of time to load and to display the jobs and their current status.

Solution

Delete unwanted or old data from the Status column. This resolves the issue and the Jobs dialog displays the required information.

For example: Delete unwanted or old data from the avsys.job_status table that is more than 30 days old using the following SQL query:

Delete from job_status
where status = 'Completed'
and status_time < sysdate - 30;

Microsoft SQL Server Database Audit Trails are in Stopped State After Upgrading Java

Learn how to fix issue when audit trails belonging to Microsoft SQL Server database go to stopped state after upgrading Java version u291 or greater.

Problem

Audit trails that belong to Microsoft SQL Server database are not collecting audit data. This issue is observed after upgrading the Java version to u291 or greater and when Microsoft SQL Server target’s connect string is one of the following:

Solution

Modify the connect string for Microsoft SQL Server database (in Oracle Database Security Central console or AVCLI) to one of the following:

Issue in Language Setting of the Audit Vault Agent

Learn how to fix the language setting in Audit Vault Agent.

Problem

Unable to change or set the language in Audit Vault Agent. Audit Vault Agent supports languages other than English.

Audit Vault Agent uses the language specified in the locale settings of the host machine (Agent machine), provided the language is supported. In case the specific language is already set on the system, then there is no need to change the settings for the Agent to use the specific language.

Solution

The locale settings for the Windows platform can be changed through the Control Panel on the Windows host machine.

To change the locale settings on Linux/Unix/AIX/Solaris platform, set the LC_ALL and LANG environment variables.

For example:

export LC_ALL=fr_FR.iso88591
export LANG=fr_FR.iso88591

Unable to Create a Database Firewall Monitoring Point

Learn how to fix an error while creating a Database Firewall monitoring point.

Problem

An attempt to create a Database Firewall monitoring point using the target host name does not succeed.

Symptom

Solution

DNS is not configured and hence the above error is observed. Configure the DNS and attempt to create the Database Firewall monitoring point again.

In case DNS is configured, verify the DNS settings. Attempt to resolve the host name is made once every minute.

Unable to Connect to Audit Vault Server through Console or SSH

Learn how to resolve if you are unable to log in to Audit Vault Server through the console or SSH.

Problem

Unable to log in to the Oracle Database Security Central console or through SSH as opc user.

The following error is displayed when attempting to connect through SSH as opc user:

remote side unexpectedly closed network connection

The following error is displayed when attempting to connect through the Oracle Database Security Central console:

internal server error 500

Solution

Oracle DBSecCentral OCI Marketplace image has a password expiry setting. Check if the password for the opc user has expired. The following message is displayed when attempting to connect through SSH from another Linux VM to the Audit Vault Server instance:

ssh -i av_key opc@<IP address>
Audit Vault Server 20.x.0.0.0

DO NOT CHANGE ANY CONFIGURATIONS IN Audit Vault Server APPLIANCE WITHOUT GUIDANCE FROM ORACLE SUPPORT.
ANY CHANGES SHOULD BE TRACEABLE TO APPROPRIATE SR REFERENCE.
Your account has expired; please contact your system administrator.
Authentication failed.

Follow these steps to resolve the issue on the:

Related Topics

Audit Vault Agent Fails with the ORA-01745 Error

Learn how to resolve the ORA-01745 error for Audit Vault Agent.

Problem

Audit Vault Agent fails with the ORA-01745 error.

Solution

Modify the firewall rules to ensure that communication between Audit Vault Agent and Audit Vault Server is allowed.

Oracle Directory or Table Audit Trail Stops with Error PLS-00201

Learn how to resolve error PLS-00201 in the collector logs.

Problem

The Oracle directory or table audit trail stops and the collector logs display the following error:

PLS-00201: identifier 'SYS.DBMS_AUDIT_MGMT' must be declared

Solution

Grant permissions to the target user and start the trail again. See Oracle Database Setup Scripts.

Error with Potential Insecure Path

Learn how to solve error java.lang.IllegalArgumentException:Potential insecure path found : <path>.

Problem: Audit Vault agent fails with error java.lang.IllegalArgumentException: Potential insecure path found : <path>.

Solution:

Error OAV-47112 When Trying to Delete an Existing Archive Location

Learn what to do when you receive the OAV-47112 error when trying to delete an existing archive location.

Problem

The OAV-47112 error appears when you try to delete an existing archive location.

Solution

You might receive this error when you try to delete an archive location that is currently in use to store archive tablespaces or data files. Wait until the tablespace or data file archive period expires before deleting the archive location. If needed, you can create a new archive location to use for the tablespace or data file archiving and then retrieve the tablespaces from the previous location and archive them to the new location.

Transaction Log Audit Trail Stops Due to XML Parsing Error

Learn how to fix issue when Transaction Log Audit Trail goes to stopped state due to XML parsing error.

Problem: Transaction Log Audit Trail stops due to XML parsing error. This is because of invalid XML record generated by Oracle GoldenGate.

Solution:Contact Oracle Support to create a Merge Label Request for applying the patch 32175609, 32063871, 33701099, and 34014874. This patch needs to be applied on Oracle GoldenGate installation.

“-bash: permission denied” Error When Trying to Run Custom Backup Script from /home/oracle

Scripts in /home/oracle cause permission errors when trying to run the scripts.

Problem: “-bash: permission denied” error when trying to run script from /home/oracle. The reason is executing script under /home/oracle is not allowed.

Solution: Move scripts to a different location.

Issues Deleting Target Database With Audit Trail Still Running

Secured target could not be deleted as the audit trail had not officially been stopped via the console or command line.

Problem: Unable to delete a target while it’s trails are running. This is a safety function to prevent removal of active audit trails by accident.

Solution: Stop all the trails on that target through GUI, or AVCLI before dropping the target.

Deleting Audit Records Requires Applying Retention Period to Purge Records

Learn how to apply retention periods to audit data so that it can be purged.

Problem: No mechanism to delete audit data.

Solution: Apply small retention period to audit data to be deleted so that it is purged with time. See Configuring Archive Locations and Retention Policies for retention periods.

Unable to Mount NFS on New AVDF 20.3 Server

Learn how to fix inability to Mount NFS on s System running with Oracle DBSecCentral.

Problem: AVDF Client is unable to mount due to bug or parameter settings on NAS Storage and/or NFS Server.

Symptoms:

Note: nfs-client01 is the Oracle DBSecCentral system. Oracle DBSecCentral has no mechanism to collect the sosreport.

Solution:

  1. Engage Vendor NAS Storage Support or NFS Admin Support Team to verify if the mount point at the NFS Server side is properly set-up. See the output of the command below.

    [root@nfs-client01 ~]# showmount -e nfs-server01
/avdf_archive_vol01 nfs-client01 <== This is the limited NFS configuration of the NFS Server, which can be seen from DBSecCentral NFS Client.
[root@nfs-server01 ~]# cat/etc/exports
/avdf_archive_vol01 nfs-client01(rw,no_subtree_check,no_root_squash)

  2. Test whether the NFS mount point can be mounted properly on NFS client, please use the command below.

    [root@nfs-client01 ~]# mount -vvvv -t nfs -o nolock nfs-server01:/avdf_archive_vol01/avdf_archive_backup
mount.nfs: timeout set for Mon Oct 29 14:05:37 2018
mount.nfs: trying text-based options 'nolock,vers=4.1,addr=<IP ADDR>,clientaddr=<IP ADDR>'
[root@nfs-client01 ~]# df -hP
Filesystem Size Used Avail Use% Mounted on
devtmpfs 412M 0 412M 0% /dev
tmpfs 432M 0 432M 0% /dev/shm
tmpfs 432M 6.0M 426M 2% /run
tmpfs 432M 0 432M 0% /sys/fs/cgroup
/dev/mapper/ol-root 9.8G 7.3G 2.5G 75% /
/dev/sdb1 16G 11G 5.7G 65% /yum
/dev/sda1 1014M 171M 844M 17% /boot
tmpfs 87M 0 87M 0% /run/user/0
nfs-server01:/avdf_archive_vol01 11G 2.2G 7.4G 23% /avdf_archive_backup <== This the sample AVDF NFS mount point. The one in use might be different.

    Note:

    vvvv - this is the debugging mode in NFS to test which layer of NFS is failing.

    -o nolock - to test if the NFS mount can mount using nolock.

    If the above command is able to mount the ADVF NFS mount point, hence, there is no issue on the NFS at the Linux OS level.

For more information refer to My Oracle Support Doc ID 2466520.1.

Alert Email Notifications Are Not Received from Oracle DBSecCentral Server

Learn what checks need to be performed when the email alerts are not received from the DBSecCentral server.

Problem: Alert email notifications are not received from Oracle DBSecCentral Server.

Solution:

  1. Log in with a super administrator user in AVCLI and check SMTP server settings.

    AVCLI> connect avadmin;
Connected.
AVCLI> LIST ATTRIBUTE OF SMTP SERVER;

--------------------------------------------------------------------------------------------------------
| HOST | PORT | SENDER_ID | SENDER_EMAIL | SECURE | TRUSTSTORE | STATE |
========================================================================================================
| <mail server host> | 25 | user1 | user1@<mail server host> | UNSECURED | | ENABLED |

--------------------------------------------------------------------------------------------------------
1 row(s) selected.
The command completed successfully.

  2. From AVCLI interface send an email to test if the connection with the SMTP server works:

    AVCLI> TEST SMTP SERVER SEND EMAIL TO user2@<mail server host>;
Request submitted successfully.

  3. There are multiple reasons why the connection to the SMTP server might not work including:

    • The SMTP server is configured using DNS name and it cannot be resolved by DBSecCentral server

    • DBSecCentral server cannot communicate with the mail server

    • There are invalid objects in the database

    • Java processes are stuck

    • There are scheduled jobs by auditor user to retrieve audit settings or user entitlements

    View other problem causes and their solutions at My Oracle Support Doc ID 2232033.1

Audit Vault Agent is Stuck in Starting State: Error OAV-46573

Problem:

Audit Vault agent installed as a service on Windows is stuck in starting state after restarting the agent host.

Error OAV-46573: Agent is UNREACHABLE on host "hostname". Please try after some time. Audit trail is not eligible for auto start.

Solution:

Set JAVA_HOME. Audit Vault agent needs to find Java Runtime Environment.

SSH Becomes Disabled After Enabling FIPS Mode

If SSH becomes disabled after enabling FIPS mode, update the SSH keys to be compliant with FIPS.

Problem

In Oracle AVDF 20.9, SSH becomes disabled after enabling FIPS mode.

Solution

Before enabling FIPS 140-2, ensure that your SSH keys are compliant with FIPS. If your SSH keys are not compliant with FIPS, the SSH connection with the appliance might be lost after enabling FIPS.

For Oracle AVDF on Oracle Cloud Infrastructure (OCI), before enabling FIPS mode, ensure that the opc user has FIPS-compliant keys registered to /home/opc/.ssh/authorized_keys.

Follow these steps to resolve this issue:

  1. Log into the Oracle Database Security Central console and disable FIPS mode.

  2. Log back into the appliance through SSH and check or update the user keys for SSH-enabled users in ~/.ssh/authorized_keys to be compliant with FIPS.

    It can take several minutes for the console to become available after enabling or disabling FIPS mode.

  3. Enable FIPS mode.

Related Topics

Audit Vault Agent Is Not Reachable from the Oracle Database Security Central Console

Problem

Oracle Database Security Central console reports an agent as “not reachable.” When trying to start the agent, a message similar to the following appears:

C:\AUDIT_VAULT_AGENT_3\bin>agentctl.bat start An instance of the agent is already running.
[2015-08-26T10:51:25.345+03:00] [agent] [ERROR] [] [] [tid: 10] [ecid:
172.xx.1.xxx:69595:1440575485345:0,0] OAV-10: Failed to release connection to DB[[
Failed to release connection to DB at
oracle.av.platform.common.dao.ConnectionManagerImpl.destroy(ConnectionManagerI
mpl.java:578) at oracle.av.platform.agent.AgentController.doStop(AgentController.java:1966)
at oracle.av.platform.agent.AgentController.doProcess(AgentController.java:2037)
at oracle.av.platform.agent.AgentController.main(AgentController.java:2046)
Nested Exception: oracle.ucp.UniversalConnectionPoolException: The Universal
Connection Pool cannot be null at
oracle.ucp.util.UCPErrorHandler.newUniversalConnectionPoolException(UCPErrorHandler.java:368) at
oracle.ucp.util.UCPErrorHandler.newUniversalConnectionPoolException(UCPErrorHandler.java:336) at
oracle.ucp.util.UCPErrorHandler.newUniversalConnectionPoolException(UCPErrorHandler.java:350) at
oracle.ucp.admin.UniversalConnectionPoolManagerBase.destroyConnectionPool
UniversalConnectionPoolManagerBase.java:469) at
oracle.av.platform.common.dao.ConnectionManagerImpl.destroy(ConnectionManagerImpl.java:574)
at oracle.av.platform.agent.AgentController.doStop(AgentController.java:1966)
at oracle.av.platform.agent.AgentController.doProcess(AgentController.java:2037)
at oracle.av.platform.agent.AgentController.main(AgentController.java:2046)

Cause

The lock file is still present. This is a protection mechanism to prevent starting multiple agents from the same host.

Solution

  1. Make sure that the java.exe processes for the agent are terminated. Use Task Manager to terminate them, if necessary.

  2. Remove the <agent home>\av\conf\agent.lck file. For example:

    del C:\AUDIT_VAULT_AGENT_3\av\conf\agent.lck

  3. Start the agent normally. For example:

    C:\AUDIT_VAULT_AGENT_3\bin> agentctl.bat start

Proxy Error When Opening DBSecCentral Console in Web Browser

Problem

While opening the DBSecCentral console in a web browser, the following error is shown:

Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /console/f.

Solution

To fix the proxy error:

  1. Check if the database and the Automatic Storage Management (ASM) instance is running or not. If not, then reboot the Audit Vault Server once and then check again.

  2. If Java framework is not running, then start it by running the following command:/usr/local/dbfw/bin/javafwk start

Prevent a Terminal Login Session from Expiring When Connecting to an Audit Vault Server or a Database Firewall Server

Problem

When performing an Audit Vault Server or Database Firewall Server backup or upgrade, sometimes the connection to the server through the terminal timeouts.

Solution

To prevent a terminal login session from expiring when connecting to an Audit Vault Server or a Database Firewall Server.

  1. Connect to the AV Server as root using a terminal session (like putty).

  2. Run the following command: cd /etc/ssh

  3. Run the following command: vi sshd_config

  4. Run the following command: /ClientAliveCountMax

  5. Set the value from 0 to 1000

  6. Save the file by running the following command: :wq!

  7. Run the following command at the OS prompt: service sshd restart

  8. Run the following command: cd /usr/local/dbfw/templates

  9. Run the following command: vi template-ssh-sshd-conf

  10. Run the following command: /ClientAliveCountMax

  11. Set the value from 0 to 1000

  12. Save the file by running the following command: :wq!

  13. Exit out of the terminal session.

  14. Connect to the Audit Vault server or Database Firewall server again.

Microsoft SQL Server Database Audit Trails Are Unreachable

Problem

When you start an audit trail, it fails with the following message:

OAV-46573: Agent is UNREACHABLE on host "****.XXX.com". Please try after some time. Audit trail is now eligible for auto start.

This may occur for EVENT LOG and DIRECTORY audit trails for Microsoft SQL Server on Microsoft Windows Server 2012.

Cause

The Audit Vault Agent was stopped. To verify this, use the agentctl status command. For example:

PS C:\Agent_Home\bin> ./agentctl status
Agent is stopped.

Solution

Start the Audit Vault Agent by using the agentctl start command. For example:

PS C:\Agent_Home\bin> ./agentctl start
Agent started successfully.

The audit trails are configured for automatic startup. After you start the Audit Vault Agent, the audit trails should start automatically. Check the status to verify that the audit trails are started and collecting audit data.

Note: You can also configure the Audit Vault Agent to restart automatically. See Configuring Agent Auto Restart Functionality.

Database Firewall Error ODF-10507: TCP Session Re-use

Problem

The Database Firewall reports an error similar to the following in /var/log/messages:

Dec 23 08:41:20 dp-svif-odb-n001 dbfw2.0:
com.oracle.dbfw.dbfw_server WARN - ODF-10507: TCP session re-use:
Session reuse observed for session 10.8.130.107:35699-10.2.129.152:8521
Connection observed 61 seconds since last access

Cause

A closed TCP session to the database has been reopened. This could lead to the state from the previous session being applied to the new session.

Solution

No action is required.

Automate Archivelog Deletion in the Audit Vault Server Repository By Using the oracle User

Problem

You can’t automate archivelog deletion in the Audit Vault Server repository because crontab is not enabled for the oracle user.

Cause

Crontab is disabled by default for the oracle user in Oracle DBSecCentral.

Workaround

Use the root user to log in as the oracle user and issue the required command. For example: su -l oracle -c bash.

To enable the oracle user’s crontab, as the root user, update /etc/cron.allow and change the command to ensure that the oracle user password has not expired. This results in configuration errors for using crontab.)

OAV-46511: Missing Plug-in for Trail at Agent on Host

Problem

Adding an audit trail fails after unregistering and re-registering a host. The following error appears:

OAV-46511: missing plugin for trail at agent on host "<hostname>"

Solution

  1. Stop the Audit Vault Agent.

  2. Make sure that no processes are running from the Audit Vault Agent home.

  3. Log into the Oracle Database Security Central console and stop any audit trails that are using this Audit Vault Agent. These should already have been be stopped when the agent was stopped, but check again.

  4. In the Oracle Database Security Central console, select Agents.

  5. Select the host name that appears in the error.

  6. Select Deactivate.

  7. Select the same host name, and select Activate.

    A new key is created.

  8. Select Downloads in the left navigation menu.

  9. Download the agent.jar file to the target host.

  10. Create a new home (or remove all files from the old Audit Vault Agent home).

  11. Redeploy the Audit Vault Agent.

    java -jar agent.jar -d <AGENT_HOME>

  12. Start the Audit Vault Agent.

    cd <AGENT_HOME>/bin ./agentctl start -k

Related Topics

Initiate Pairing for High Availability Fails with OAV-46599: Internal Error

Problem

When setting up high availability, the Initiate Pairing command fails with the following error:

OAV-46599: internal error Error: Failed to execute HTTPS request on the remote Audit Vault Server.

The messages log shows errors similar to the following:

Jun 4 16:07:10 avs00001702d420 setup_ha.rb[5272]: com.oracle.avs.high_availability ERROR - ODF-10001: Internal error: Error: Failed to execute HTTPS request on the remote Audit Vault Server.
Jun 4 16:18:04 avs00001702d420 setup_ha.rb[9959]: com.oracle.avs.high_availability ERROR - ODF-10001: Internal error: Error: Failed to execute HTTPS request on the remote Audit Vault Server.

Cause

The ports that are required for network connectivity between the primary and secondary Audit Vault servers in high availability mode are not open in the firewall.

Solution

Open port 7443 in the firewall.

Related Topics

Archive Error OAV-46599 and Internal Error ORA-14400: Partition Key Not Mapped

Problem

When archiving the data, the following errors appear:

OAV-46599: Internal error ORA-14400: partition key not mapped to any partition

Cause

The EVENTDATA disk group doesn’t have enough space.

Solution

  1. Check the current status of the existing Oracle Automatic Storage Management (Oracle ASM) disks and disk groups.

    set pagesize 1000
set linesize 1000
COLUMN NAME format A25
COLUMN MOUNT_STATUS format A10
COLUMN HEADER_STATUS format A20
COLUMN MODE_STATUS format A20
COLUMN STATE format A20
COLUMN PATH format A40
COLUMN LABEL format A20
SELECT GROUP_NUMBER,NAME,TOTAL_MB,FREE_MB FROM V$ASM_DISKGROUP;
SELECT MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,TOTAL_MB,FREE_MB,NAME,PATH,LABEL FROM V$ASM_DISK;

  2. Add space to the EVENTDATA disk group.

    ALTER DISKGROUP EVENTDATA add disk 'path';

SYSLOG Forwarding for Alerts Isn’t Working

Problem

SYSLOG forwarding for alerts isn’t working.

Cause

This may happen if the SYSLOG forwarding queue has many alerts without the old events backlog.

Solution

  1. Bounce the database.

  2. Purge the av_alert queue table as the AVSYS user.

    DECLARE
po_t dbms_aqadm.aq$_purge_options_t;
BEGINdbms_aqadm.purge_queue_table('AVSYS.AV_ALERT_QT', NULL, po_t);
end;
/

SYSLOG Forwarding to SIEM Isn’t Working

Problem

SYSLOG forwarding to SIEM isn’t working.

Cause

rsyslog.conf file has incorrect configuration entries.

The rsyslog.conf says to forward alerts, while also stating to filter out alerts. Because of this, the alert will never get written to local /var/log/messages and can’t be forwarded to SIEM.

#FORWARDED SYSLOG CATEGORIES:system, alerts, debug, info
*.info;local0.none;local1.none;local2.none;user.crit;user.warn;*.=debug;local1.info @<SIEM
      server IP address>
# This filters out DBSecCentral alerts, which are either user.crit, or user.warn
user.crit;user.warn ~

Solution

  1. Modify /etc/rsyslog.conf to not filter out alerts. Change user.crit;user.warn ~ to:

    user.crit;user.warn /var/log/DBSecCentral_alerts

    Note: This change to /etc/rsyslog.conf will revert after a server reboot.

  2. As root services rsyslog restart.

  3. If there are no alerts being forwarded but the following “logger” command works, then either no alerts are generated or SYSLOG forwarding is not setup fully in WebConsole.

    #Force it something log to rsyslog to process to send to remote @IP (and write to the DBSecCentral_alerts file)
logger -p user.crit DBSecCentral Alert dummy test
#You should see this dummy test being logged in the /var/log/dbalerts file

    For more information see, Configuring Audit Vault Server Syslog Destinations.

To monitor what is being sent by rsyslog off DBSecCentral server via rsyslog, run one of the following commands on port 514:

tcpdump -nn -i <eth#> | grep <IP of SIEM>
tcpdump -i eth0 tcp port 514
tcpdump -i lo -A udp and port 514
tcpdump -A dst
tcpdump -nnvvXS dst

Oracle DBSecCentral Reports For Oracle Database Shows UNKNOWN For Session Info If Native Network Encryption Is Enabled On the Database

Problem

If Native Network Encryption is enabled on an Oracle Database the Oracle DBSecCentral reports show UNKNOWN for session info. Examples include unknown_username and unknown_client.

Solution

See Monitor Native Network Encrypted Traffic Through Database Firewall for Oracle Databases for how to resolve this issue.

Error: Kernel Out of Memory

Problem

The following error appears:

kernel: Out of memory: Kill process nnnnn (oracle) score nnn or sacrifice child

The following output is a partial example:

<kern.warning> xxxxav01aud kernel: java invoked
      oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0<kern.info> xxxxav01aud
      kernel: java cpuset=/ mems_allowed=0-1<kern.warning> xxxxav01aud kernel: Pid: 19085, comm:
      java Not tainted 2.6.39-400.250.6.el5uek #1<kern.warning> xxxxav01aud kernel: Call
      Trace:<kern.warning> xxxxav01aud kernel: [<ffffffff81113e04>]
      dump_header+0x94/0xe0<kern.warning> xxxxav01aud kernel: [<ffffffff81113f4d>]

After this error occurs once, similar errors are logged intermittently and the audit trail may be stopped or a repository database may be terminated suddenly.

Cause

When this error occurs, the memory usage of oraagent.bin becomes very high. When this type of memory usage occurs, the Linux: Out-of-Memory (OOM) Killer may stop some processes. See Doc ID 452000.1 in My Oracle Support for more information about this process.

The root cause of the oraagent.bin high memory usage is related to an Oracle Database issue where the dependent listener is removed or renamed. See Doc ID 1640721.1 for more information about this issue.

Solution

Stop the oraagent.bin process periodically.

Increasing the Logical Volume Capacity for a File System

If an Oracle DBSecCentral file system runs out of space, you can allocate more space to the logical volume that holds the file system.

Use the lvextend command to increase the logical volume capacity. The vg_root volume group normally has unallocated space for this purpose.

  1. Log in to the appliance through SSH and switch to the root user.

    See Logging In to Oracle DBSecCentral Appliances Through SSH.

  2. Run vgs to check the volume group free space. For example:

    /usr/sbin/vgs
VG      #PV #LV #SN   Attr   VSize  VFree
vg_root   1  13   0 wz--n- 149.84G 10.72G

    For more detailed volume group information, run vgdisplay.

  3. Increase the logical volume capacity.

    For example, the following command adds 2 GB to the /tmp folder from the VG_ROOT volume group:

    /usr/sbin/lvextend -r -L+2G /dev/mapper/vg_root-lv_tmp

Related Topics

Can’t Install Host Monitor with Error: Failed to Generate Executables for Host Monitor

Problem

When installing the Host Monitor Agent, you receive one of the following errors:

[root@hm]# ./hostmonsetup install
/usr/bin/ld: cannot find -lpcap
collect2: ld returned 1 exit status
make: *** [hostmonitor] Error 1
Line 751: Failed to generate executables for Host monitor.
[root@hm]# ./hostmonsetup install
/usr/bin/ld: cannot find -lcap
collect2: ld returned 1 exit status
make: *** [hostmonitor] Error 1
Line 751: Failed to generate executables for Host monitor.

The libcap and libpcap packages are already installed. For example:

rpm -qa|grep cap

The output lists the following:

libcap-2.16-5.5.el6.x86_64
compat-libcap1-1.10-1.x86_64
libcap-ng-0.6.4-3.el6_0.1.x86_64
libpcap-1.0.0-6.20091201git117cb5.el6.x86_64
perl-Pod-Escapes-1.04-119.el6_1.1.x86_64

Cause

The libcap and libpcap packages should be installed.

Also, the -devel packages for libcap and libpcap packages must be installed.

Solution

Run the following commands to verify whether the packages are installed:

rpm -q libcap
rpm -q libcap-devel
rpm -q libpcap
rpm -q libpcap-devel

The output of each command should display the location where the library is installed. If any package is not installed, you should see a prompt stating that the package is not installed.

If a package isn’t installed, then install it by using the following command:

yum -y install <package_name>

To install the -devel packages, use the following commands:

yum -y install libpcap libpcap-devel
yum -y install libcap libcap-devel

Audit Trail Error: Unable to Connect to Target to Get Timezone Offset

Problem

In Oracle DBSecCentral, audit collection stops with the following error:

OAV-8015: Error initializing AuditEventCollector instanceCollectionController : run : AuditException from process()

In the Oracle Database Security Central console, when you start the audit trail, the state changes to “Stopped” with the following error:

Unable to connect to target to get Timezone Offset

Cause

The AV.COLLECTOR.TIMEZONEOFFSET audit collection attribute for the target is missing.

Solution

  1. Run the following SQL query on the target database:

    select systimestamp from dual;

    The output should look like the following example:

    SYSTIMESTAMP

---------------------------------------------------------------------------
AV.COLLECTOR.TIMEZONEOFFSET +7:00

  2. Log in to the Oracle Database Security Central console and an administrator.

  3. Modify the target and add the AV.COLLECTOR.TIMEZONEOFFSET audit collection attribute that you identified in the preceding step.

    For instructions, see Modifying Targets.

  4. Stop and start the audit trail.

    For instructions, see Stopping, Starting, and Autostart of Audit Trails in Oracle Audit Vault Server.

Diagnostic Report: Checking for Unknown Keys in /usr/local/dbfw/etc/dbfw.conf

Problem

The diagnostic report has the following message:

Checking for unknown keys in /usr/local/dbfw/etc/dbfw.conf: ["duplex", "speed"] - WARN

Cause

This warning may appear if the following entries are not configured in /usr/local/dbfw/etc/dbfw.conf:

speed=""
duplex=""

Solution

You can safely ignore this warning.

Agent IO Error: Network Adapter Can’t Establish Connection

Problem

In Oracle DBSecCentral 20.3 and later, the following error may occur after install when trying to start with this command: ` ./agentctl start -k`:

Internal Error. See log files for detail.

Within the av.agent log, the following error may appear:

[2021-07-29T16:25:36.956+07:00][agent] [ERROR] [] [] [tid: 1] [ecid: 1918831609:74227:1627550704887:0,0] Unable to connect to AV Server after 10 retries
[2021-07-29T16:25:36.959+07:00] [agent] [ERROR] [] [] [tid: 1] [ecid: 1918831609:74227:1627550704887:0,0] Error occurred in Agent.[[Failed to connect to DB at oracle.av.platform.common.dao.ConnectionManagerImpl.getConnection(ConnectionManagerImpl.java:548) at oracle.av.platform.agent.AgentController.doValidateKey(AgentController.java:3040) at oracle.av.platform.agent.AgentController.doProcess(AgentController.java:3595) at oracle.av.platform.agent.AgentController.main(AgentController.java:3614)]]

Similarly, within the av.common log, the following error may appear:

[2021-07-29T16:25:04.879+07:00][common] [ERROR] [] [] [tid: 1] [ecid: 1918831609:74227:1627550704887:0,0] [Thread]:main. Unable to get connection to the datasource through certificate and without credentials. Unable to start the Universal Connection Pool: oracle.ucp.UniversalConnectionPoolException: Cannot get Connection from Datasource: java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection

Cause

This error may occur if the external firewall is blocking network traffic from the secure target host to the audit vault server on port 1522.

Solution

To start correctly, you need to open the ports between Oracle DBSecCentralF and a secured target on 1521 and 1522. If there is a firewall in between the firewall ports, you must open it. After you open the ports, the error should no longer persist.

To prevent this issue, apply the patch to update Oracle DBSecCentral to the latest release update (RU). See Patching Oracle Database Security Central Release 20.

Error ORA-01403 No Data Found When Adding a Database Firewall Instance to a Target

Problem

In Oracle DBSecCentral, when configuring Database Firewall monitoring points for a target, you can add the first Database Firewall instance as a monitoring point, but when you try to add the second instance, you may get the following error:

ora-01403 no data found

If you remove the first Database Firewall instance and try to add the second instance as a new monitoring point, you may get the following error:

OAV-46593: secured target address does not exist. cannot drop secured target address.

Cause

This issue may happen when the ha_role for one of the Database Firewalls is set to 2 in the database. The ha_role needs to be set to 1.

This could happen if the Database Firewall instances were previously configured as a resilient pair.

Solution

  1. Connect to the Audit Vault Server database.

  2. Run the following SQL query:

    select id, name, is_active, ha_role from avsys.firewall where deleted_at is null;

  3. Find the row where ha_role is set to 2 and make a note of the Database Firewall ID.

  4. Run the following query by replacing the firewall_id with the ID that you identified in the preceding step.

    update avsys.firewall set ha_role = 1, is_active = 1, ha_role_changed_at = current_timestamp where id in (<firewall_id>);

    For example:

    update avsys.firewall set ha_role = 1, is_active = 1, ha_role_changed_at = current_timestamp where id in (2);

  5. Run the following command:

    commit;

  6. Add the second Database Firewall instance for the target by using the Oracle Database Security Central console.

The Order of IP Addresses Changes After Setting Up DNS Servers

Problem

After setting up DNS servers, the order of the IP addresses may change.

For example, you might set up the DNS servers in the following order:

  1. DNS server 1: xx.xxx.xx.14

  2. DNS server 2: xx.xxx.xx.15

  3. DNS server 3: xx.xxx.xx.16

After the configuration, the IP addresses might change to the following order:

  1. DNS server 1: xx.xxx.xx.14

  2. DNS server 2: xx.xxx.xx.16

  3. DNS server 3: xx.xxx.xx.15

Cause

The order depends on the behavior of the package that’s operating internally when registering the DNS servers.

Solution

No action is required. The IP addresses are not always registered in the order in which they are set.

NTP Is Unreachable on the Audit Vault Server

Problem

When configuring Network Time Protocol (NTP) on the Audit Vault Server, NTP is unreachable. It may be working fine on a Database Firewall server in the same network.

Cause

This may occur if the browser that you’re using to access the Oracle Database Security Central console is set to a language other than English.

Solution

To resolve this issue, change the browser to English and refresh the Oracle Database Security Central console.

Database Firewall Status Is Running but the Status Is Down on the Oracle Database Security Central Console

Problem

Oracle Database Security Central console shows that the Database Firewall is Down even though it’s up and running.

Cause

This may be caused by an application timeout that’s related to a bug that was fixed in Oracle AVDF 20.8.

Solution

To resolve this issue, complete the following steps on the Database Firewall sever:

  1. Rename /usr/local/dbfw/.bash_profile. For example:

    mv /usr/local/dbfw/.bash_profile /usr/local/dbfw/.bash_profile_old

  2. Restart Apache.

    systemctl restart httpd

    This should make it possible for Apache to spawn the web server APIs and for the system to start working again.

  3. Check the status of the Database Firewall in the Oracle Database Security Central console. The status should be Up.

Network Audit Trail Is Not Collecting Audit Data When Using the Host Monitor Agent

Problem

When using the Host Monitor Agent to capture network traffic, the network audit trail isn’t collecting audit data, even though the audit trail is running.

Note: The following instructions apply only when using the Host Monitor Agent to capture network traffic. Ensure that there are no connection issues between the Host Monitor Agent and the Database Firewall before proceeding.

Solution

  1. Log in to the target machine where the Host Monitor Agent is installed.

  2. Restart the Audit Vault Agent in debug mode by running the following command:

    <DBSecCentral AGENT HOME>/bin>./agentctl stop
<DBSecCentral AGENT HOME>/bin>./agentctl start -l debug

  3. Log in to the Oracle Database Security Central console as an administrator.

  4. Start the network audit trail and see if the network audit trail status changes to Collecting.

  5. Run the following command on the machine where the Host Monitor Agent is installed and ensure that the hostmonitor process is running.

    ps -ef | grep hostmonitor

  6. Navigate to the folder that contains the hostmonitor logs (for example, AGENT_HOME/hm/log), and run the following command:

    grep "Successfully sent data to Firewall machine" *

    Note: If “Successfully sent data to Firewall machine” appears in a log file, then the Host Monitor Agent is collecting and sending the network traffic to the Database Firewall successfully.

  7. If the preceding text doesn’t appear in the entry is not present in the hostmonitor log file, run the following command on the secured target machine to see which IP addresses and ports the target database is listening to.

    lsnrctl status

  8. Log in to to the Oracle Database Security Central console as an administrator and complete the following steps using the IP addresses and ports that you identified in the preceding step.

    1. Select Targets.

    2. Select the target for which the network audit trail configured.

    3. Verify that all the IP addresses and ports that the target database is listening to appear in the Connection Details column in the Database Firewall Monitoring section.

    4. If an IP address or port doesn’t appear, select link under Connection Details.

    5. In the Database Firewall Monitor dialog box, select Add to add any missing IP addresses and ports.

    6. Select Save twice.

  9. Run the following command on the target and verify the network interface card (NIC) to which all target database listening IP addresses belong.

    ifconfig -a

  10. Navigate to the folder that contains the hostmonitor logs (for example, AGENT_HOME/hm/log), and run the following command:

    grep "network_device_name_for_hostmonitor" *

    The output should be similar to the following example:

    The selected network device for capturing is: eth0. To change the device update the network_device_name_for_hostmonitor attribute at Collection Attributes to any one value from the list: eth0, nflog, nfqueue, any, lo and restart the trail.

    In the preceding example, the log shows that hostmonitor is listening on the eth0 NIC.

    Verify that hostmonitor is listening on the same NIC to which the target database listening IP addresses belong.

  11. If the target database listening IP addresses belong to a different NIC, perform the following steps:

    1. Log in to the Oracle Database Security Central console as an administrator.

    2. Select Targets.

    3. Select the target for which the network audit trail is configured.

    4. Select Modify.

    5. Select Audit Collection Attributes.

    6. Select Add and add the following attribute name and value pair:

      Name: network_device_name_for_hostmonitor

      Value: Enter the name of the NIC to which the target database listening IP addresses belong.

      Select Save twice.

  12. Restart the network audit trail from the Oracle Database Security Central console.

  13. Navigate to the folder that contains the hostmonitor logs (for example, AGENT_HOME/hm/log), and run the following command:

    grep "Successfully sent data to Firewall machine" *

    Note: If “Successfully sent data to Firewall machine” appears in a log file, then the Host Monitor Agent is collecting and sending the network traffic to the Database Firewall successfully.

  14. Log in to the target machine where the Host Monitor Agent is installed.

  15. Restart the Audit Vault Agent in normal mode by running the following command:

    <DBSecCentral AGENT HOME>/bin>./agentctl stop
<DBSecCentral AGENT HOME>/bin>./agentctl start

Internal Error When Deploying the Audit Vault Agent

Problem

Deploying the Audit Vault Agent fails with the following error:

Internal Error
Error occurred during install/upgrade. Check log files for more information.

The log file may contain entries similar to the following:

Unable to get connection to the datasource through certificate and without credentials. Exception occurred while getting connection: oracle.ucp.UniversalConnectionPoolException: Cannot get Connection from Datasource: java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection

Cause

The Audit Vault Agent was trying to connect to an incorrect IP address.

Solution

Cross-check the IP address of the Audit Vault Server and the secured target server on which you’re installing the Audit Vault Agent.

Related Topics

Agent Host Is Not Registered

Problem

Deploying the Audit Vault Agent fails with the following message, even though the agent was already registered:

Agent host is not registered.
Agent host must be registered before an agent can be installed or upgraded. Agent deployment failed.

Cause

This might happen on a multi-homed system when there are multiple routes from the Audit Vault Agent host to the Audit Vault Server. The SQLNet traffic might use an IP address that’s different from the one that was used to register the Audit Vault Agent host.

When registering a host in the Audit Vault Server, you have two choices:

This means that you must register the host with the same IP address that you see when using SQL*Plus to connect from the Audit Vault Agent host to the Audit Vault Server.

Solution

To verify the IP address with which the host should be registered, use SQL*Plus and connect using the connect string that’s defined in the <agent_home>/av/conf/bootstrap.prop file. For convenience, you can also add it to the tnsnames.ora file with the designation AV.

Use the following steps to determine which IP address to register:

  1. Determine which network interface card (NIC) is used for the Audit Vault Agent communication.

    1. Connect to the AV Server Database from the agent host using the following command.

      sqlplus <username>/<password>@"`cat <agent_home>/av/conf/bootstrap.prop | grep "SYS.CONNECT_STRING" | sed -e 's/SYS.CONNECT_STRING=//g' | sed -e 's/\\//g'`"

      For *`<username>`*, enter a valid user name in the database, such as avauditor.

      For *`<agent_home>`*, enter the path to the agent directory.

    2. Run the following query:

    select SYS_CONTEXT('USERENV','IP_ADDRESS') FROM dual;

  2. Use the IP address that was returned by the preceding query to register the Audit Vault Agent in the Oracle Database Security Central console. See Registering Hosts on the Audit Vault Server.

    Alternative, you can use the following AVCLI command:

    register host <hostname> with ip <ip address from the query>

On a multi-homed system, to register with another IP address, contact your network administrator and change the TCP routing configuration.

Related Topics

A Database Firewall Policy Is Not Blocking Statements Correctly

Problem

After creating a Database Firewall policy to block or substitute all queries from a specific database user, that user may still be able to run the SQL statements freely.

Cause

This could happen if the protected address that’s associated with the secured target doesn’t have an Oracle service name.

Solution

Make sure that all protected addresses contain an Oracle service name.

Related Topics

Having Automatic Archiving Enabled Is Giving OAV-47116 Error

Problem

AUTOMATIC ARCHIVING ENABLE is giving error OAV-47116.

Cause

Auto archive order should be greater than 0 (i.e 1 or more) to enable automatic archiving.

Solution

Change auto archive order to be greater than 0 from UI then try to enable automatic archiving.

Network Trail Fails To Be Started Due To Insufficient Permissions Error

Problem

The following errors are spotted in agent host monitor logs when the network trail is started:

startHostMonitor : exception while starting HostMonitor[[
Failed to start collector {0}:{1}
at oracle.av.platform.agent.collfwk.impl.factory.HMCommandExecutor.execute(HMCommandExecutor.java:380)
at oracle.av.platform.agent.collfwk.impl.factory.HMCommandExecutor.execute(HMCommandExecutor.java:311)
at oracle.av.platform.agent.collfwk.impl.factory.HMCommandExecutor.startHostMonitor(HMCommandExecutor.java:111)
at oracle.av.platform.agent.collfwk.impl.factory.HMCommandManager.startHostMonitor(HMCommandManager.java:679)
at oracle.av.platform.agent.collfwk.impl.factory.HMCommandManager.startTrail(HMCommandManager.java:736)
at oracle.av.platform.agent.collfwk.impl.factory.CollectionFactory.createCollection(CollectionFactory.java:565)
at oracle.av.platform.agent.collfwk.impl.factory.CollectionFactory.createCollection(CollectionFactory.java:392)
at oracle.av.platform.agent.StartTrailCommandHandler.processMessage(StartTrailCommandHandler.java:63)
at oracle.av.platform.agent.AgentController.processMessage(AgentController.java:585)
at oracle.av.platform.agent.AgentController$MessageListenerThread.run(AgentController.java:3075)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot run program "/u02/app/oracle/product/avdf12/av_agent/hm/hostmonmanager" (in directory "/u02/app/oracle/product/avdf12/av_agent/hm"): error=13,
Permission denied
Caused by: java.io.IOException: error=13, Permission denied

Cause

The DBSecCentral agent and the hostmonitor have been deployed as root and oracle users are not allowed to run the executables due to the binaries’ permissions and the hardcoded configuration.

Solution

  1. Redeploy the DBSecCentral agent with oracle user using documentation steps:

    Deactivate and Remove the Audit Vault Agent

  2. Copy the zip file for host monitor deployment:

    cd <Agent Installation Directory>/stage/plugins
cp agent-linux-x86-64-hmon-one.zip to /usr/local

  3. Unzip the file as root user:

    unzip the agent-linux-x86-64-hmon-one.zip filecd hm

  4. Install host monitor:

    ./hostmonsetup install agentuser=oracle agentgroup=oinstall

  5. Start the agent

    cd <agent home>/bin
./agentctl start

  6. Start the network trail.

How To Start an Audit Trail for Audit Trail Type DIRECTORY if the Database is Down

This document explains how to start audit trail for OS audit files even when the database is down or only in MOUNT state.

  1. For the collector to start you need to add 3 NLS attributes that the collector needs to parse the OS audit files.Collect the following information from the secured target database while it is running, if this is a standby database you can also collect this information from the primary database:

    select parameter, value from v$nls_parameters where parameter in ('NLS_LANGUAGE','NLS_TERRITORY','NLS_CHARACTERSET');

    For example:

    PARAMETER VALUE---------------- --------------------
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CHARACTERSET AL32UTF8

  2. Add these NLS attributes of the secured target database to the collector:

    ORCLCOLL.NLS_TERRITORY
ORCLCOLL.NLS_LANGUAGE
ORCLCOLL.NLS_CHARSET

    Adding these attributes manually is necessary to be able to start the audit trail and collect audit records from OS files even if the database is in MOUNT state or down.

    See more information at Oracle Database Audit Collection Attributes in the Plug-In Reference section.

After Setting the “SSH Acess” Setting, the SSH Connections are Dropped

Symptoms

After setting the “SSH Access” setting from the “Network Services” DBSecCentral configuration page, the SSH connections are dropped.

Cause

When the SSH connection was being created a telnet connection protocol was used.

Solution

Use the SSH protocol to connect to the DBSecCentral server.

DBSecCentral Directory Audit Trail Stays Up Collecting Audit Data Even When Target Database Is Shutdown

Question

Why does DBSecCentral directory Audit Trail stay up collecting audit data, even when target database is shutdown?

Answer

Directory audit trail collectors does not need target database to be up and running for collection. As long as the directory contains log files or audit files, directory trail collector collects. Hence non-availability of target database does not immediately translate to warning in DBSecCentral UI. This is the inherent nature of directory trail collector.

If the target database is down, directory trail can continue running as long as it has access to directory or audit logs.

This behavior is unlike table audit trails, where non-availability of target database immediately translates to warning in DBSecCentral UI.

ODF-10717 Is Logged In /var/log/messages File During The Starting Up of Database Firewall

Symptoms

ODF-10717 can be logged in /var/log/messages file during the starting up of database firewall.

Example)
Jan 18 00:45:51 <HOST> <EP>: com.oracle.dbfw.fw INFO - ODF-10102: Startup complete: Ready to process network traffic
Jan 18 00:45:51 <HOST> <EP>: com.oracle.dbfw.fw WARN - ODF-10717: Zero DAM packets processed: pcap_dispatch() processed zero packets out of 20 requested
Jan 18 00:45:51 <HOST> <EP>: com.oracle.dbfw.fw ERROR - ODF-10701: Network packets not intercepted: Maximum capacity of the system has been exceeded for Protected Databases '<SECURE TARGET1>', '<SECURE TARGET2>'

Cause

ODF-10717 can be logged when a empty network packet is detected on using DAM mode environment.

Also it is easly detected during the starting-up of database firewall or under heavy networking trafic.

This does not always mean that there are some kinds of crtical errors and it can be safely ignored in usual.

Solution

Safely ignore ODF-10717.

Error: Net::ReadTimeout occurred when executing Setup_ha.rb –disable_failover

Symptoms

Error: Net::ReadTimeout occurred when executing Setup_ha.rb --disable_failover.

$ /usr/local/dbfw/bin/setup_ha.rb --disable_ failover
Error: Net::ReadTimeout

Cause

After script was changing status to DISABLE, the other DBFW will be processed to reflect the settings, but at that point a timeout error has occurred.

Solution

If the result of /usr/local/dbfw/bin/setup_ha.rb --status is DISABLE, no other action is needed.

[root@avsxxxxx ~]#sudo -u oracle /usr/local/dbfw/bin/setup_ha.rb --status
HA mode: PRIMARYHA server 1: xx.xx.xx.xx
HA server 2: xx.xx.xx.xx
...
Automatic failover: DISABLED <<<<<

Processes Still Run After Stopping Audit Trails

Problem

Before upgrading to 20.1, audit trails should stop; however, some of the processes continue to run even after stopping the audit trails. These processes may lead to problems.

Solution

To prevent this issue, apply the patch to update Oracle DBSecCentral to the latest release update (RU). See Patching Oracle Database Security Central Release 20.

To resolve this issue, begin by identifying and removing the avorclcoll processes that might be persisting on the host machine. For each host machine where an Oracle Directory trail is configured, run the following command:

ps -ef | grep avorclcoll

If the avorclcoll process exists, then terminate the process by executing the following command:

kill -9 <pid>

Replace <pid> with the actual Process ID associated with the avorclcoll process.

Loss of Bonding Between Network Interface Cards Upon Creation of Proxy Port

Problem

When a network interface card (NIC) bonding is established, the creation of a proxy port on one of the NICs through the Oracle Database Security Central console, performed as a super administrator, leads to the unintended removal of the established NIC bonding.

Solution

To prevent this issue, apply the patch to update Oracle DBSecCentral to the latest release update (RU). See Patching Oracle Database Security Central Release 20.

After setting the port in the UI and encountering the loss of bonding, follow these steps to reconfigure the bond using the command-line interface (CLI):

  1. Configure Bonding via CLI: Use the CLI on the Database Firewall instance to configure the bonding between the relevant devices. See the CONFIG-BOND documentation for more details.

  2. Configure Proxy Ports: Set up the necessary proxy ports for the bonded device as required for your configuration. See the CONFIG-PROXY documentation for more details.

  3. Re-establish Bonding: Run the bonding command, as outlined in Step 1, to re-establish the bond between the network interface cards.

Significant Time Delay in Captured Traffic by the Database Firewall For Reporting

There may be a significant time delay from the moment traffic is captured by the Database Firewall to the time it is available for report generation.

Symptoms

There may be a significant time difference between the time when the traffic is captured by the Database Firewall and the time when it is available at Audit Vault Server (AVS) for report generation. For example, if some SQL is captured, it may not be available in AVS for a few hours to generate the reports.

The corresponding time at which the data becomes available to AVS can be checked from the AVSYS.EVENT_LOG table on the AVS server. This can be done by the following SQL command:

SELECT MAX(EVENT_TIME) FROM AVSYS.EVENT_LOG;

Cause

The possible causes may be one of the following:

  1. There could be a time zone mismatch on the UI and the visible time stamp is shifted.

  2. There is a significant load on one of the Enforcement Points and the Audit Vault Server is not able to insert the data at the appropriate rate.

  3. Some other problem.

Solution

The corresponding solutions to the above mentioned causes are as follows:

  1. Connect directly to the Audit Vault Server database and run a query on the AVSYS.EVENT_LOG table. Compare the time stamp with the expected one.

  2. To confirm there is a significant load on one of the Enforcement Points, check the number of files in the /usr/local/dbfw/va/*/log directories. If the number of kernel*.gz files is over 10, then this is a plausible cause.

  3. Collect the diagnostic package for further investigation.

ODF-10719 Error Logged In Messages File After Starting Database Firewall

The error code ODF-10719 is logged in the messages file when starting Database Firewall, indicating difficulties in loading session information from a file.

Problem

When starting Database Firewall, a ODF-10719 error may be logged into the /var/log/messages file.

The following error message is an example of what is displayed:

<HOST> auditd[4106]: Audit daemon rotating log files
<HOST> <EP>.24: com.oracle.dbfw.dbfw_server WARN - ODF-10719: Unable to load Session information from file: Could not acquire lock on file for instance=0 after 120 seconds.

Cause

Database Firewall collects the information of a session via a connect packet. Database Firewall manages the information of each session by using the connection information. A ODF-10719 error can occur when the Database Firewall cannot confirm the connect package information because it is unable to load the session information from its files. This may occur if sessions are already established before starting Database Firewall, which in turn means that Database Firewall cannot collect connect packets of the established session so it is missing this information in the logs.

Another possible cause is if the connection pooling feature is used on a target environment. This may result in an ODF-10719 error being logged in messages file after starting Database Firewall.

Solution

To prevent this issue, apply the patch to update Oracle DBSecCentral to the latest release update (RU). See Patching Oracle Database Security Central Release 20.

Please safely ignore the ODF-10719 error. Restart Database Firewall and then your session.

“Server Error 500” on Oracle DBSecCentral Server after Setting Network Time Protocol (NTP)

After setting the Network Time Protocol (NTP) option on the Oracle DBSecCentral server, the following error may occur: Server Error 500.

Problem

After setting the NTP using the setup page from the GUI, the following error message is observed:

Server Error 500

Cause

This error is caused by setting the NTP option on the Oracle DBSecCentral server. This causes the server to stop working and show the above error message.

Solution

This issue can be solved by completing the following steps:

  1. Disable the NTP setting.

  2. Set the time manually.

  3. Reboot the Oracle DBSecCentral Server

After completing these steps, the database and all other services should start successfully. After the application starts successfully, the NTP service can be enabled again without issues.

Audit Vault Agent Logs Report IO Error: The Network Adapter Could Not Establish Connection Due To Inactive Database Listener

Audit Vault Agent logs report IO Error saying the network adapter could not establish connection due to the inactivity of the database listener which leads to the disruption in the audit trail.

Problem

The Audit Vault Agent logs return an IO Error that states The Network Adapter could not establish the connection.

Cause

When the database listener is inactive, the audit trail loses its ability to establish communication with the database through the agent, leading it to enter a stopped state. It is crucial to ensure that the database is available and that the connected listener is active. Additionally, you should verify that the database services are correctly registered with the listener. If the listener is down, or in instances where the listener is active, but without any associated database services registered, this situation triggers a shutdown of the audit trails with the above error message.

Solution

To resolve this issue, you must ensure that the database listener is active and the database service is up. Use the command ps -ef|grep tns to check the status of the listener. Below is the output of this command when the listener is down:

root 10 2 0 Aug30 ? 00:00:00 [netns]
oracle 1673 1245 0 07:47 pts/1 00:00:00 grep tns

Next, use the command ps -ef|grep pmon to see the status of the database service. Below is the output of this command when the database service is up:

oracle 1670 1245 0 07:47 pts/1 00:00:00 grep pmon
oracle 3003 1 0 Aug30 ? 00:00:46 ora_pmon_orcl

See Creating and Configuring a Database Firewall Monitoring Point for more information.

Additionally, a TNSPING to the database service can verify the availability of the listener. If the listener is found to be inactive, start it using the LSNRCTL utility. Once the listener is running, the audit trail collector should initiate, and the status should display a green arrow pointing upwards.

Monitor the agent logs located at $AGENT_HOME/av/log. No further error logs should be reported upon successfully starting the collectors. Regularly checking these logs will help ensure the proper functioning of the collectors.

Authentication Processing Error When Logging in Due to Excessive Group String Length in Active Directory

An error in authentication processing is triggered by an excessive group string length in the Active Directory, leading to disruptions in user access.

Problem

Users receive an error message prompting them to contact their application administrator after there is an error processing authentication. This error disrupts the user’s ability to successfully authenticate and access the DBSecCentral system.

Cause

This issue arises when the group string associated with the AD/LDAP user in the Active Directory is too long. The group string length cannot exceed 8,000 characters; the system encounters an error processing authentication when the length surpasses this amount. See Integrating Oracle Database Security Central with Microsoft Active Directory or OpenLDAP for more information.

Solution

To prevent this issue, apply the patch to update Oracle DBSecCentral to the latest release update (RU). See Patching Oracle Database Security Central Release 20.

To resolve this issue and ensure successful authentication, it is important to adhere to the current group string length requirement. Reduce the number of groups for the AD/LDAP users so that the group string length remains within the 8,000 character limit. Additionally, administrators should manage the user group assignments within the Active Directory to ensure that users are only added to necessary groups. After reducing the number of groups, login to DBSecCentral with the AD user as planned.

Discrepancies When Registering a Target Using Internet Explorer as the Browser

Problem

When using Internet Explorer Audit Vault Server target registration screen is different from the manual.

Solution

To prevent this issue, apply the patch to update Oracle DBSecCentral to the latest release update (RU). See Patching Oracle Database Security Central Release 20.

You will also need to use a different broswer as Oracle Database Security Central console does not support Microsoft Internet Explorer 11 (and prior).

Datafiles Don’t Change to Read Only Mode After Entering Archive Period

Problem

A tablespace is not entering READ ONLY status even after the archive period has started.

Cause

The definition of ALERT_EVENT_MAP_TRANS is incorrect. The definition of ALERT_EVENT_MAP_TRANS should be the same as the definition of ALERT_EVENT_MAP.

Check the definition of ALERT_EVENT_MAP_TRANS and ALERT_EVENT_MAP by running the following commands:

desc AVSYS.ALERT_EVENT_MAP_TRANS
desc AVSYS.ALERT_EVENT_MAP

Solution

Change the definition of the ALERT_EVENT_MAP_TRANS table to match that of the ALERT_EVENT_MAP table.

Datafiles Don’t Change to Read Only Mode After Entering Archive Period

Problem

A tablespace is in ONLINE status even after the archive period has started, because the tablespace is not entering READ ONLY status.

Symptoms

You can use the following query to determine the date that the tablespace entered the archive period and it’s current status:

SQL> select <a.tablespace_name>, a.status, to_char(b.bytes,'999,999,999,999')"BYTES",
to_char(add_months(to_date('01-01-1970','MM-DD,YYYY'),substr(<a.tablespace_name>,9,3)+1),'DD-MON-YYYY') "WHEN PLACE OFFLINE",
to_char(add_months(to_date('01-01-1970','MM-DD,YYYY'),substr(<a.tablespace_name>,14,3)+1),'DD-MON-YYYY') "WILL BE DELETED",
(-1)*months_between(to_char(add_months(to_date('01-01-1970','MM-DD,YYYY'),substr(<a.tablespace_name>,9,3)),'DD-MON-YYYY'),
to_char(add_months(to_date('01-01-1970','MM-DD,YYYY'),substr(<a.tablespace_name>,14,3)),'DD-MON-YYYY')) "MONTHNS BETWEEN"
from dba_tablespaces <a>, dba_data_files <b>
where <a.tablespace_name> = <b.tablespace_name>
and <a.tablespace_name> like '%ILM%'
order by <a.tablespace_name>

The output will be in the format: TABLESPACE_NAME STATUS BYTES DATE PLACED OFFLINE DATE IT WILL BE DELETED MONTHNS BETWEEN.

For example, if the output is TABLESPACE_ABC ONLINE 104,857,600 01-OCT-2020 01-APR-2021 6, then it means that the tablespace “TABLESPACE_ABC” is online, contains 104,857,600 bytes, was placed offline on October 1, 2020, was deleted on April 1, 2021, and was in the archive period for six months between October and April.

Solution

  1. Check the following information:

    This should return 0:

    select count(*) from avsys.JOB_STATUS_TRANS;

    This should return AVSPACE:

    select <tablespace_name> from dba_tables where table_name ='JOB_STATUS_TRANS';

  2. Disable AVS_MAINTENANCE_JOB by running the following:

    exec dbms_scheduler.disable ('AVSYS.AVS_MAINTENANCE_JOB');

    Ensure that it is disabled by running the following:

    SELECT STATE,enabled FROM dba_scheduler_jobs where job_name='AVS_MAINTENANCE_JOB';

  3. Set event 14529 at level 512 by running the following:

    alter session set events '14529 trace name context forever, level 512';
alter system set events '14529 trace name context forever, level 512';

  4. Run the following on the Audit Vault database as the AVSYS user:

    DROP TABLE AVSYS.JOB_STATUS_TRANS;

CREATE TABLE AVSYS.JOB_STATUS_TRANS as
    SELECT * FROM AVSYS.JOB_STATUS
        WHERE 1=0;

ALTER TABLE AVSYS.JOB_STATUS_TRANS
    ADD CONSTRAINT CK_JOB_STATUS_TRANS_STATUS
    CHECK (STATUS IN ('Starting',
                    'Running',
                    'Stopping',
                    'Completed',
                    'Failed',
                    'Waiting'));
commit;

    Verify this completed successfully by running the following:

    This should return 0:

    select count(*) from avsys.JOB_STATUS_TRANS;

    This should return 1:

    select count(*)from dba_tables where table_name = 'JOB_STATUS_TRANS';

    This should return AVSPACE:

    select tablespace_name from dba_tables where table_name ='JOB_STATUS_TRANS';

  5. Disable event 14529 by running the following:

    alter system set events '14529 trace name context off';
alter session set events '14529 trace name context off';

    Confirm event 14529 is now disabled by running the following:

    SET SERVEROUTPUT ON
DECLARE
event_level NUMBER;
BEGIN
DBMS_SYSTEM.READ_EV(14529, event_level);
dbms_output.put_line (' 14529 is set at level '||TO_CHAR (event_level));
END;

  6. Re-enable the AVS_MAINTENANCE_JOB by running the following:

    exec dbms_scheduler.enable ('AVSYS.AVS_MAINTENANCE_JOB');

    Ensure that it is enabled by running the following:

    SELECT STATE,enabled FROM dba_scheduler_jobs where job_name='AVS_MAINTENANCE_JOB';

OAV-46599 Internal Error: The Data Guard Observer Is Not Present When Performing Manual Switchover of Audit Vault Server

The internal error OAV-46599 occurs in a High Availability (HA) setup where the data guard observer is found to be absent, preventing the switchover process. To resolve this issue, you should enable automatic failover so that the data guard observer status is set to YES.

Problem

When attempting to perform a High Availability switchover, an OAV-46599 Internal Error occurs, indicating the absence of the Data Guard observer. The following error message is an example of what is displayed:

OAV-46599: Internal Error: The Data Guard Observer is not present

When the automatic failover is disabled, both the primary and secondary (standby) servers will display blank Data Guard observer statuses, which inhibits the switchover process. The following is an example of the first several lines of the status:

$ /usr/local/dbfw/bin/setup_ha.rb --status
HA mode: PRIMARY
HA server 1: <IP 1>
HA server 2: <IP 2>
Unique database name:
Current database role: PRIMARY
Data guard broker: ENABLED
Data guard observer:
...

Note: Data Guard observer is blank instead of saying YES. This occurs in both the primary and secondary (standby) modes.

Cause

This error is caused by the absence of the Data Guard observer, which is necessary for the role switching process in the High Availability setup. The observer statuses are blank because automatic failover must be disabled.

Solution

To resolve this issue, please take the following steps:

  1. Enable automatic failover. Please see Disabling or Enabling Failover of the Audit Vault Server for more information.

  2. Ensure that the Data Guard observer status is set to YES.

  3. Now perform the switchover process.

Mail Notification Fails When Mailing Server is Configured with TLS/SSL

Problem

When the mailing server is configured with TLS/SSL, the mail notifications fail due to a failure during secure handshake.

Solution

To prevent this issue, apply the patch to update Oracle DBSecCentral to the latest release update (RU). See Patching Oracle Database Security Central Release 20.

DBSecCentral Agent Deployment Failure: Unable to Get Connection from Datasource

The DBSecCentral Agent deployment fails due to an error connecting to the datasource; the solution is to increase the init parameter processes value to 1000 for the AV repository database.

Problem

DBSecCentral Agent deployment fails with the following error message:

Unable to get connection to the datasource through certificate and without credentials.
Exception occurred while getting connection: oracle.ucp.UniversalConnectionPoolException:
Cannot get Connection from Datasource: java.sql.SQLRecoverableException:
IO Error: Got minus one from a read call

Cause

The init parameter processes is set to the default value of 500. This value is too low for the AV repository database.

Solution

To resolve this issue, increase the init parameter processes value to 1000 for the AV repository database. You can do this by running the following SQL query:

ALTER SYSTEM SET processes = 1000;

Once you have increased the value of this parameter, restart the AV repository database. The DBSecCentral Agent deployment should then succeed.

To learn more about the sizing guidelines, review the Database Security CentralBest Practices and Sizing Calculator forDBSecCentral12.2 and DBSecCentral20.1 (Doc ID 2092683.1).

Audit Vault Agent Installation Fails Due To File System Permissions

The Audit Vault agent installation fails when the file system on which the agent is being installed is mounted with the noexec option. This option prevents the execution of programs from the mounted file system.

Symptoms

When attempting to install the Audit Vault agent, the following error message appears:

Error occurred during install/upgrade. Check log files for more information.

The agent deployment log file contains the following error message:

java.io.IOException: Cannot run program "/home/audituser/avault/bin/agentctl":
java.io.IOException: error=13, Permission denied

Cause

The error occurs because the file system on which the agent is being installed is mounted with the noexec option. This option prevents the execution of programs from the mounted file system.

Solution

To resolve this issue, take the following steps:

  1. Check the Java version: Ensure that you have Java SE 6 or later installed on your machine. To check the Java version, run the following command: java -version

  2. Verify the file system mount options: Check whether the file system on which the agent is being installed is mounted with the noexecoption. Run the following command to check the mount options: mount. Below is sample output of the mount command showing the file system mounted with noexec option:

    # mount
/dev/sda5 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)

  3. Remount the file system: If the file system is mounted with the noexec option, remount it without this option. The specific command for remounting the file system will depend on the operating system and file system type. For example, to remount an ext4 file system named /dev/sda5 without the noexec option, you would run the following command:

    mount -o remount,noexec=off /dev/sda5

  4. Deploy the Audit Vault agent: After remounting the file system, deploy the Audit Vault agent. The installation should now proceed without encountering the permission error.

DBSecCentral Agent Deployment Fails on Target Host with RAC DB Due to Incorrect IP Address Registration

The agent deployment on the target host fails due to an incorrect IP address being used when registering the ‘secured host’ in the AV server. The outgoing IP address of the cluster should be used instead.

Problem

The agent deployment process fails on the target host, resulting in the following errors:

java -jar agent.jar -d <Agent_Home_Path>
Agent host is not registered.
Agent host must be registered before an agent can be installed or upgraded.
Agent deployment failed.

Cause

The target host has a RAC DB installed, and the host’s physical IP address was used when registering the ‘secured host’ in the AV server. This causes the agent deployment to fail.

Solution

To resolve this issue, ensure that the outgoing IP address of the cluster, rather than the physical IP address, is specified when registering the ‘secured host’ in the AV server.

To determine the outgoing IP address of the host:

  1. Connect to the AV database using SQLplus.

  2. Run the following query:

    select sys_context('userenv','ip_address') from dual;

    The result of this query will display the outgoing IP address of the host. Use this IP address when registering the ‘secured host’ in the AV server.

Host Monitoring Agent Installation Fails With Error About Inability to Retrieve Agent Details

Problem

Installation of Host Monitoring Agent fails with the following error:

/usr/local/hm# ./hostmonsetup install
Unable to retrieve - 1. Agent User 2. Agent Location 3. Platform Validation 4. HM Install State
Exception occured while creating AVS DB connection. Exception: Error while trying to retrieve text for error ORA-01804
Contact Oracle support.
:/usr/local/hm# ls -ltrh

/usr/local/hm/log# cat av.hmdeployer.log
[2023-09-16 11:06:21,784] [PID: 54294, TName: main] [ERROR] - Exception occured while creating AVS DB connection. Exception: Error while trying to retrieve text for error ORA-01804

[2023-09-16 11:06:21,784] [PID: 54294, TName: main] [ERROR] - Exception Occured: Unable to establish bootstrap connection to AV Server Database using connect string:(DESCRIPTION=(ENABLE=BROKEN)(FAILOVER=on)(R
[2023-09-16 11:06:21,851] [PID: 54300, TName: main] [ERROR] - Exception occured while creating AVS DB connection. Exception: Error while trying to retrieve text for error ORA-01804

Solution

Set the LD_LIBRARY_PATH environment variable as the Host Monitoring Agent installation path:

  1. Log in to the Audit Vault Server through SSH and switch to the root user.

    See Logging In to Oracle DBSecCentral Appliances Through SSH.

  2. Change the directory to the location of the Host Monitoring Agent:

    cd /user/local/hm

  3. Set the LD_LIBRARY_PATH environment variable:

    export LD_LIBRARY_PATH=/usr/local/hm

  4. Run the installation of the Host Monitoring Agent:

    ./hostmonsetup install -verbose

Audit Vault Agent Fails to Start from Windows Service

When trying to start the Audit Vault Agent from the Windows service, an error is returned.

Problem

The following error was logged in <DBSecCentral AGENT HOME>/av/log/av.agent.prunsvr.YYYY-MM-DD.log:

Unable to find Java Runtime Environment.
The system could not find the environment option that was entered.
reportServiceStatusE: dwCurrentState = 1, dwWin32ExitCode = 0, dwWaitHint = 0, dwServiceSpecificExitCode = 0.

This is because the JAVA_HOME variable was not set in the environment of the Windows operating system (OS).

Solution

  1. In the Windows OS, navigate to Control Panel.

  2. Select System.

  3. Select Advanced system settings.

  4. In the Advanced, select Environment Variables button.

    The Environment Variables dialog is displayed.

  5. Add a new JAVA_HOME variable that points to your JDK or JRE installation path.

    For example, C:\Program Files\Java\jdk1.8.0_65

  6. Start the Audit Vault Agent.

DBSecCentral Agent Management after OS Upgrade

After an OS upgrade, users may encounter problems with the DBSecCentral Agent; users should restart the agent to prevent problems.

Problem

When using Oracle DBSecCentral 20.1 and later, users may encounter issues with the DBSecCentral Agent after an operating system upgrade. The DBSecCentral Agent may be affected if specific precautions are not followed.

Solution

To mitigate potential issues after an OS upgrade, follow these steps:

  1. Stop DBSecCentral Agent before an OS upgrade:

    <DBSecCentral AGENT HOME>/bin>./agentctl stop

  2. After an OS upgrade, start the Agent:

    <DBSecCentral AGENT HOME>/bin>./agentctl start
<DBSecCentral AGENT HOME>/bin>./agentctl status

Additionally, ensure that the OS version being upgraded is certified and supported by the DBSecCentral Agent.

Starting a Monitoring Point Causes Error OAV-46649

Problem

After successfully creating a monitoring point, attempting to start it fails. Starting through the AVCLI results in error OAV-46649: Enforcement point is in resume state.

Cause

DNS is not properly configured in the Database Firewall.

Solution

  1. Log in to Oracle Database Security Central Console as a super administrator.

  2. Select Settings.

  3. Select System in the left menu.

  4. Under Status section, select System Settings.

  5. Configure DNS settings.

  6. Select Save.

  7. Start the monitoring point.

Related Topics

Database Firewall Not Capturing in DAM Mode

Problem

Database Firewall is not capturing in database activity monitoring mode on a VMware installation due to network misconfiguration.

Solution

ESX/VMware virtual switch has a property that does not allow VLAN traffic.

  1. The switch needs to be re-configured to allow VLAN traffic.

  2. Live capture should start working at this point, test and verify that.

  3. Check the reports to ensure they are being populated with data.

  4. Check to verify that alerts in the Oracle Database Security Central console are being generated.

How to Use Linux to Send E-mails From a DBSecCentral Appliance

Problem

How to use Linux to send e-mails from a DBSecCentral appliance?

Solution

  1. Log in to the appliance through SSH and switch to the root user.

    See Logging In to Oracle DBSecCentral Appliances Through SSH.

  2. Run the following command:

    Example: echo TEST | mailx -s "subject " -S smtp=10.10.10.10:25 username@oracle.com

Capture Bind Variables When Running the Database Firewall in DAM Mode

Problem

Is it possible to capture bind variables when running the Database Firewall in database activity monitoring (DAM) mode?

Solution

If the Database Firewall is only used to monitor the secured target through a monitoring point then the All Activity report will not capture bind variables involved in the SQL statement.

Audit Vault Agent Configuration for a Table Audit Trail in a RAC Environment

Problem

Learn how to configure the Audit Vault Agent for a table type audit trail in a real application cluster (RAC) environment.

Solution

Install the Audit Vault Agent in one of the following ways:

Related Topics

Database Firewall Certificate Validation Failed

If in the Oracle Database Security Central console the Database Firewall page shows a status of Certificate Validation Failed, follow these steps to resolve the issue.

  1. Update the certificate of the Database Firewall.

    For more information see Fetching an Updated Certificate from Database Firewall.

  2. If updating the certificate does not resolve the issue, rotate the Database Firewall certificate.

    For more information see Rotating Database Firewall Certificates.

  3. If the Database Firewall certificate can’t be rotated, it may be because the Audit Vault Server certificate authority is no longer valid on the Database Firewall. Follow these steps to resolve the issue:

    1. Log in to the Audit Vault Server through SSH and switch to the root user.

      See Logging In to Oracle DBSecCentral Appliances Through SSH.

    2. Run the following command:

      openssl x509 -noout -subject -in /usr/local/dbfw/etc/ca.crt

      Take note of the output.

    3. Log in to the Database Firewall through SSH and switch to the root user.

      See Logging In to Oracle DBSecCentral Appliances Through SSH.

    4. Run the following command:

      openssl x509 -noout -subject -in /usr/local/dbfw/etc/controller.crt

    5. If the outputs of the commands are different, then you need to add the Audit Vault Server certificate to the Database Firewall.

      For more information see Specifying the Audit Vault Server Certificate and IP Address.

Configuring ERSPAN for SQL Traffic Auditing in Monitoring (Out of Band) Mode

If Monitoring (Out of Band) is not collecting any SQL traffic audit, follow these steps to resolve the issue.

Problem

Monitoring (Out of Band) in DBSecCentral 20.7 is not collecting any SQL traffic audit. Even though the network interface cards (NICs) are correctly configured and the traffic is being captured in the pcap files, no SQL traffic audit is displayed on the DBSecCentral web page.

Cause

The Database Firewall does not process ERSPAN traffic by default in Monitoring (Out of Band) mode. This has to be enabled on the Database Firewall monitoring points, otherwise, the SQL traffic audits will not be displayed despite being correctly mirrored and captured.

Solution

To resolve this issue, you need to enable ERSPAN processing by setting DAM_TRAFFIC_IS_ERSPAN=1. More information can be found in Configuring Encapsulated Remote Switched Port Analyzer with Database Firewall.

Recovery Disk Group is Getting Full with Archive Logs

Problem

Archive logs are not deleting and are causing the disk group of the standby Audit Vault Server in a high availability pairing to get full.

Run the following commands to determine if the archivelog has been applied to the standby:

select a.thread#, a.sequence#, a.applied
from v$archived_log a, v$database d
where a.activation# = d.activation#
and a.applied='YES'
/

Make sure the standby is in sync with the primary.

Solution

Ensure that your retention policies are set as this can help free up space in the fast recovery area. For more information see Creating and Deleting Archive and Retention Policies.

If no files are eligible for deletion based on their retention policy, then manual intervention is required. For more information see Managing Archival and Retrieval in High Availability Environments.

Cannot View the Updated Maintenance Job Schedule After Making Changes

After changing the maintenance job schedule in AVS, the updated time may incorrectly display as 0:00 due to a display issue, but the changes are correctly applied.

Problem

After changing the start time of the maintenance job schedule in the Audit Vault Server, the schedule displays as 0:00 instead of the updated time upon re-login.

Cause

This is a display issue only found in Oracle AVDF 20.7 and later.

Solution

Although 0:00 is displayed, the schedule changes have been successfully applied. You can verify the updated schedule by running the following SQL query in the AVS repository database:

select next_run_date,STATE,enabled FROM dba_scheduler_jobs where job_name='AVS_MAINTENANCE_JOB';

Oracle DBSecCentral Does Not Failover When Primary Server Is Down

When the network connection to the Primary Oracle DBSecCentral server is down, the system does not failover to the Standby server. The current design only triggers a failover if the Oracle DBSecCentral Database or a critical process crashes.

Symptoms

When the network connection to the Primary setup is down and the Primary Oracle DBSecCentral server becomes inaccessible, Oracle DBSecCentral does not initiate a failover to the standby server. Once the network connection to the primary server is restored, Oracle DBSecCentral becomes accessible again through the primary site.

Cause

Oracle DBSecCentral is currently designed to failover only if the Oracle DBSecCentral Database or a critical process crashes and triggers a failover to the Standby site. However, if the network connection is disabled or down, the Observer cannot determine the status of the processes, and as a result, failover will not occur.

Solution

The current Oracle DBSecCentral failover mechanism does not guarantee High Availability, as failover only occurs during process crashes. In cases of network or system outages, the service may remain down. To maintain continuous availability, Oracle DBSecCentral should be accessible from the Standby site when the Primary server is inaccessible. Implementing a Load Balancer could help by directing traffic to the Standby site in such scenarios. See Handling a Failover Scenario for more information on Failover Scenarios.

Executing ‘ AVBACKUP BACKUP ‘ Command Fails

Problem

When executing the ‘/var/lib/oracle/dbfw/bin/avbackup backup’ command after running ‘/var/lib/oracle/dbfw/bin/avbackup config’ , the operation fails with the following error “info.txt: No such file or directory.”

Solution

The info.txt file should be located in the same directory. The user must ensure that the backup directory and its parent directories are owned by oracle:oinstall to prevent this error.

see Backup and Restore of Audit Vault Server for more information.

Error OAV-47411 “Export Path” Does Not Exist on Remote File System

Learn what to do when you receive the OAV-47411 error while registering a Network File System (NFS) export to Oracle DBSecCentral.

Problem

While registering an NFS export by executing the following command:

AVCLI> REGISTER REMOTE FILESYSTEM <Remote FS name> OF TYPE NFS ON HOST <Hostname> USING EXPORT <Export path> MOUNT;

This error might be encountered:

OAV-47411: Export <Export path> does not exist on remote filesystem.

Cause

To identify the root cause, complete the following steps:

  1. Run the following AVCLI commands and ensure the outputs are correct:

    • The output should display the export path:
    AVCLI> LIST EXPORT OF TYPE NFS ON HOST <Hostname>;
    • The output should display the Remote Filesystem name, along with export path:
    AVCLI> LIST REMOTE FILESYSTEM;
    • The output should be ACCESSIBLE:
    SHOW STATUS OF REMOTE FILESYSTEM "<Name of the Remote Filesystem>";

  2. On the NFS server, run the following command to check the existence and the permissions on the export path:

    ls -ld <Export path>

  3. Check if the entry for the NFS location is located in the /etc/fstab within the Audit Vault Server:

    cat /etc/fstab

  4. Check the output of following queries:

    select * from avsys.remote_filesystem;
select * from avsys.remote_location;
select * from avsys.archive_host;

  5. Follow the below steps on the AV Server:

    1. Login to the AV Server as support.

    2. Run the following commands:

      su root
su oracle
cd $ORACLE_HOME

    3. The output for the above command should display the export list for <IP address of the NFS server>, as shown below:

    [oracle@<AV host>]$ /usr/sbin/showmount --export <IP address of NFS server>
clnt_create: RPC: Port mapper failure - Unable to receive: errno 111 (Connection refused)
     This means the ports (NFS) and 111 (port map) are blocked by the firewalls on the NFS server OR they are not open.

Solution

To resolve this error, check if the NFS server is reachable and all the required ports are open (no firewall is blocking the request on specific ports).

  1. Turn off the firewall on the NFS machine.

  2. Register the remote filesystem.

  3. Mount and check the status.

Error OAV-47487 : Uploading a Certificate to DBSecCentral Fails

Learn what to do when you receive the OAV-47487 error while uploading the certificates to DBSecCentral.

Problem

Uploading a new certificate generated using a CSR from an external source (such as a third-party application) fails with the following error:

OAV-47487: Certificate is not compatible with server

Cause

Oracle DBSecCentral 12 does not support CSRs that originate outside of its own system.

Solution

The only supported process is to generate the CSR directly from the DBSecCentral application, signing it with a CA, and then uploading the signed certificate. Follow the steps below:

  1. Download the CSR from the DBSecCentral Server.

  2. Have the CSR signed by the Certificate Authority (CA).

  3. Then, upload only the newly signed certificate (excluding the CSR and any intermediate certificates). A certificate chain is not supported.

Troubleshooting Server Error 500 in DBSecCentral

Learn how to identify the cause of “Server Error 500” in the Oracle DBSecCentral environment.

Problem

When logging into the DBSecCentral Web Console, users may encounter “Server Error 500.” This error typically indicates a failure to connect to the repository database, a critical back-end required for the Web Console. Without this connection, login fails, and a Server Error 500 is triggered.

Cause

There are various possible reasons for the Server 500 Error:

  1. User account or password issues: Incorrect password, locked account, or unsupported characters in passphrase.

  2. Database unavailability: The repository database may be down or not correctly configured.

  3. Service startup issues: The database services may not have started correctly, or other dependencies may be unavailable.

  4. Database limitations: Connection may be restricted due to database limitations or session limits.

  5. Timeout issues: Long loading times for dashboard or console due to performance delays.

  6. File system iNode exhaustion: The iNode count on /var/lib/oracle is full, preventing login.

  7. Other configuration issues: Other system-level configurations may block access.

Solution

To troubleshoot and resolve Server Error 500 in DBSecCentral, follow these steps based on the potential causes listed above:

  1. User Account or Password Issues:

    1. Verify login credentials by attempting to connect with sqlplus:

      su - oracle
sqlplus <avadmin_user>/<password>

    2. If login fails, try changing the password or unlocking the account:

      su - oracle
sqlplus "/as sysdba"
SQL> ALTER USER <avadmin_user> IDENTIFIED BY <new_password> ACCOUNT UNLOCK;

    3. Ensure the passphrase does not contain special characters other than _, as unsupported characters may cause login issues.

  2. Database Unavailability:

    1. Check /var/log/messages for specific errors like ORA-01034 or ORA-27101 indicating that the database is not available.

    2. For persistent issues, inspect alert.log and diagnostic files. Rebooting the AVS server may help restart the repository database and resolve the error.

  3. Service Startup Issues:

    1. Verify that required services, including Grid Infrastructure (GI) resources, +ASM instance, and TNS listener, are running.

    2. Restart the AVS server if services are not initialized correctly, especially if running DBSecCentral version 12.2.0.4 or later.

  4. Database Limitations:

    1. Check for ORA-20 or ORA-18 errors, which indicate session limits. Reboot the AVS server if these limitations cause connectivity issues.

    2. If session limits continue to be problematic, contact Oracle Support for further investigation.

  5. Timeout Issues:

    1. Long loading times for the dashboard or Web Console may result in Server Error 500. This is often cause by performance issues or large alert volumes.

    2. For DBSecCentral versions before 12.2 BP#5, increase the TIMEOUT setting in /usr/local/dbfw/templates/template-httpd-httpd.conf and restart networking settings.

  6. File System iNode Exhaustion:

    • Run df -i to check if /var/lib/oracle has reached 100% inode usage. If so, remove excess audit files.

  7. Other Configuration Issues:

    • Review recent system or database changes. Incorrect configurations, such as manual host reboots without reconfiguration, may disrupt the database. Following these steps should help diagnose and resolve the underlying cause of Server Error 500. If issues persist, consult Oracle Support with DBSecCentral diagnostic files for additional assistance.

User Entitlement Retrieval Job Fails After Twelve Hours

Learn how to manage when a user entitlement job fails after running for twelve hours.

Problem

User Entitlement retrieval jobs consistently fail after running for an extended period. This issue may occur when retrieving data from databases with a large number of accounts, resulting in job termination before completion. The failure typically occurs with an error message indicating the inability to process privilege user data for the target database.

As a result, entitlement snapshots cannot be generated, which impacts reporting capabilities such as Privileged Users or other entitlement reports.

Cause

This issue is caused by a system-defined timeout setting that limits the maximum runtime for jobs. when the job exceeds this limit, it terminates prematurely. Logs may show errors such as: java.sql.SQLRecoverableException: IO Error: Socket read interrupted.

Solution

This issue is caused by a system-defined timeout setting that limits the maximum runtime for jobs. when the job exceeds this limit, it terminates prematurely. Logs may show errors such as: java.sql.SQLRecoverableException: IO Error: Socket read interrupted

To resolve this issue, follow the steps below:

  1. Modify the job configuration by executing the following commands:

    su - oracle
vi /var/lib/oracle/dbfw/bin/avjfwk

  2. Update the relevant Java process line to include the following:

    /usr/bin/java -ea -Dfile.encoding=UTF-8 -DNLS_LANG="$NLS_LANG"-DORACLE_HOME="$OH" -Doracle.jdbc.javaNetNio=false -mx1024m -classpath$CLASSPATH oracle.av.platform.server.javafwk.JfwkProcess "$@" >/dev/null 2>&1

  3. Restart the javafwk Service:

    restart javafwk
systemctl stop javafwk
systemctl status javafwk
systemctl start javafwk

  4. Increase Timeout Settings:

    AVCLI>alter system set JFWK.THREAD_TIMEOUT_MINUTES=1440;

  5. Resubmit the job. Once the changes are applied, resubmit the User Entitlement job from the AV UI to ensure it completes successfully.

Unable to Drop Audit Trail from Unreachable Host

If the error “OAV-46572: Agent is unreachable on host” appears during an audit trail drop operation, use the following steps to resolve it.

Symptoms

Attempts to drop an audit trail using the GUI or AVCLI fail with the error message:

OAV-46572: Agent is UNREACHABLE on host

The system reports that the audit trail from the secured target is still running, even though the host or agent has been removed.

Cause

This issue occurs if the agent and audit trails were removed without properly stopping them first. The Audit Vault server keeps the audit trail in an “UNREACHABLE” status, anticipating that the host or agent might recover. Since the agent is inactive, the audit trail cannot be fully stopped.

Solution

If the host and agent will not be restored, proceed with the following steps to manually update the audit trail status:

  1. Connect to Audit Vault Server as the support user, switch to root, then to the dvaccountmgr, and access SQL*Plus.

  2. Set a temporary password for the AVSYS user and unlock the account.

  3. Run the following SQL command to update the audit trail status to “STOPPED”:

    UPDATE AVSYS.AUDIT_TRAIL
SET COLLECTION_STATUS=0
WHERE COLLECTION_STATUS <> 0
AND ACTIVE='Y'
AND HOST_NAME = '&unreachable_host'
AND AUDIT_TRAIL_ID IN(
  SELECT AUDIT_TRAIL_ID
  FROM AVSYS.AUDIT_TRAIL
  WHERE HOST_NAME IN(
    SELECT HOST_NAME
    FROM AVSYS.AGENT_VIEW
    WHERE STATUS='UNREACHABLE'
  )
);
    COMMIT;

  4. After executing this update, attempt to drop the audit trail again using the GUI or AVCLI.

Note: In Oracle DBSecCentral 20.5 and later, you can use the command DROP HOST <hostname> FORCE to force the host and its audit trails to be dropped directly.

Error OAV-47746 : Sensitive Objects Data Upload Fails

Learn what to do when you receive the OAV-47411 error while uploading sensitive objects data in Oracle DBSecCentral.

Problem

When attempting to upload sensitive object data in DBSecCentral 20.8, users encounter the following error:

OAV-47746: "Input file with sensitive data is invalid format."

Cause

This error may occur when unsupported, invisible characters are present in the file. These characters can cause the file format to be unrecognized during the upload process.

Solution

To resolve this issue, follow the below steps:

  1. Open the file with sensitive data and re-saved it.

  2. Convert the file from DOS to Unix format using the following command:

    dos2unix <example.txt>

  3. After converting the file, attempt to upload it again through the DBSecCentral UI console.

For more information on converting files from DOS to Unix format, refer to “Convert DOS to Unix”.

Status “Certificate Validation Failed” Error Shown in Audit Vault Server GUI

If Database Firewall page shows a status of Certificate Validation Failed in the Audit Vault Server GUI with the error OAV-46981: Unable to connect to Database Firewall with IP <ipaddress>, follow these steps to resolve the issue.

Problem

The Audit Vault Server GUI displays the status “Certificate Validation Failed” for the Database Firewall. Moreover, the following errors appear in the Host Monitor log:

Cause

The Database Firewall is down due to failed certificate validation. This may be caused by issues with the existing certificates or keys used for SSL communication.

Solution

To resolve the issue, perform the following steps:

  1. Take a backup of /usr/local/dbfw/etc/avs/ folder.

  2. Remove the existing certificates and wallet files:

    On the Audit Vault Server, as the root user, run the following commands:

    rm -f /usr/local/dbfw/etc/avs/fwcerts/*
rm -f /usr/local/dbfw/etc/avs/avswallet/*

  3. Generate new SSL certificates:

    Run the following command to recreate the necessary certificates:

    /usr/local/bin/gensslcert create-certs

  4. Update certificates in the AV Console:

    • Log in to the AV console as an administrator.

    • Go to the Database Firewall.

    • Select each Database Firewall and select “Update Certificate” button.

    • Confirm that the Database Firewall status is now showing up as “Up.”

    This solution should resolve the certificate validation issue and restore the connection with the Database Firewall.

OAV-47804: Invalid Credentials for User While Registering AD With DBSecCentral

Problem:

While Registering AD with DBSecCentral, the following error appears:

Error OAV-47804: Invalid Credential for User

Cause:

This error arises when the AD user DN, such as cn=xyz,cn=users,dc=domain,dc=com is specified instead of the username xyz.

Solution:

To resolve this issue, replace the DN with the AD username xyz and retry the registration process.

“Check Health of Audit Vault Server” Is Seen as Failed in the Job Status

Problem

In Oracle DBSecCentral, the job status of the AVS health check job may fail with the following error message:

avsxxxxxxxxxxx oracle: [AVDF SYSTEM ALERT] <Severity:Critical> <Alert Category:Password> <Alert Subcategory:Audit Vault Server Administrator and Auditor users password expiry> <Description:Password has expired of Audit Vault Server user XXXXXXXXX, YYYYYYYYY, ZZZZZZZZ.> <Recommendation:Change the password for these users.>

Solution

To resolve the following error, reset the password for the users shown in the above log.

Note: User details may vary.

Agent Fails To Restart Automatically in Oracle DBSecCentral

Learn how to resolve issues when the Agent in Oracle DBSecCentral 20.9 does not restart automatically.

Problem

In Oracle DBSecCentral, Agent fails to start automatically.

Cause

The Agents fails to start automatically due to permission issue on /etc/cron.allow.

Solution

To resolve this issue, follow the steps below:

  1. Switch to the root user.

  2. Check the user list in cron.allow to see which users are allowed to access crontab:

    cat /etc/cron.allow

  3. If the OS user who owns the Agent is not listed, modify the cron.allow file to add their username.

    vi /etc/cron.allow

All Activity Scheduled Reports Fail with “ Unknown Report Type “ Error

Learn how to address the issue when all scheduled activity reports fail with ‘Unknown Report Type’.

Problem

All activity scheduled reports fail with ‘Unknown Report Type’ error message.

Cause

The directory /usr/local/dbfw/tmp may have run out of available space.

Solution

Ensure that sufficient space is available in the /usr/local/dbfw/tmp directory. Delete unnecessary files from this location and rerun the Scheduled Report.