Oracle® Audit Vault Administrator's Guide Release 10.2.3.1 Part Number E13841-02 |
|
|
View PDF |
This appendix contains:
Table A-1 describes the Audit Vault Server log and error files. These files are located in the Audit Vault Server $ORACLE_HOME/av/log
directory. They contain important information about the return status of commands and operations. Use this information to diagnose problems.
Table A-1 Name and Description of Audit Vault Server Log and Error Files
If you need to troubleshoot the Audit Vault Console, enable Oracle Enterprise Manager logging. To do so, modify the emomslogging.properties
file (located in the ORACLE_HOME
/sysman/config
directory) in the Audit Vault Server home. Add the following lines to this file:
log4j.appender.avAppender=org.apache.log4j.RollingFileAppender log4j.appender.avAppender.File=$ORACLE_HOME/oc4j/j2ee/OC4J_DBConsole__/log/av-application.log log4j.appender.avAppender.Append=true log4j.appender.avAppender.MaxFileSize =20000000 log4j.appender.avAppender.Threshold = DEBUG log4j.appender.avAppender.layout=org.apache.log4j.PatternLayout log4j.appender.avAppender.layout.ConversionPattern=%d [%t] %-5p %c{2} %M.%L - %m\n log4j.category.oracle =DEBUG, avAppender
You can use this information to debug communication problems between the server and the collection agents.
Table A-2 lists the names and a description of the Audit Vault collection agent log and error files located in the $ORACLE_HOME/av/log
directory. These files contain important information about the return status of commands and operations. Use this information to diagnose problems.
Table A-2 Name and Description of Audit Vault Collection Agent Log and Error Files
File Name | Description |
---|---|
|
Contains a log of all errors encountered in collection agent initialization and operation. You can delete this file at any time. |
|
Contains a log of all primary collection agent-related operations and activity. You can delete this file at any time. |
|
Contains a log of all |
|
Contains a log of all |
|
Contains a log of collection operations for the Oracle Database DBAUD and OSAUD collectors. This file has no maximum size limit. To delete this log, shut down the collector, delete the log, and then restart the collector. |
|
Contains a log of collection operations for the MSSQLDB, SYBDB, and DB2DB collectors. The You can only delete this file after you have shut down OC4J. For example, to delete the log where To increase the log level, restart OC4J on the collection agent side with the appropriate debug level, as in the following example: avctl stop_oc4j avctl start_oc4j -loglevel error See Section 7.12 and Section 7.16 for information about these commands, including the available log levels for OC4J. |
|
Contains a log of the collection agent operations and any errors returned from those operations. The |
|
Contains a log of all avctl stop_oc4j avctl start_oc4j -loglevel debug |
|
Contains a log of all avctl stop_oc4j avctl start_oc4j -loglevel debug |
|
Contains a log of all avctl stop_oc4j avctl start_oc4j -loglevel debug |
|
The Oracle Audit Vault collection agent $ORACLE_HOME/oc4j/j2ee/home/log
contains the logs generated by the collection agent OC4J. In this directory, the file AVAgent-access.log
contains a log of requests that the collection agent receives from the Audit Vault Server. Use this information to debug communication problems between the server and the collection agent.
Failed configuration commands are located in the Audit Vault collection agent $ORACLE_HOME/cfgtoollogs
directory, which includes the file, configToolFailedCommands
. This file contains only the name of the failed command. See the avca.log
or avorcldb.log
file for additional information, including any associated errors and error messages.
This section contains:
Checking Trace Files for Detailed Information About Oracle Database Errors
Troubleshooting Oracle Audit Vault in an Oracle Real Application Clusters Environment
For detailed information about the cause of an Oracle Database error message, check the trace files. The trace files also indicate problems that may have occurred with the Audit Vault data warehouse, alert, and some configuration issues. The USER_DUMP_DEST
initialization parameter specifies the current location of the trace files. You can find the value of this parameter by issuing SHOW PARAMETER USER_DUMP_DEST
in SQL*Plus. See Oracle Database Performance Tuning Guide for more information about trace files
Problem: Need to find the best way to tune the Audit Vault Server performance for the REDO collector.
The Audit Vault Server installation process sets the STREAMS_POOL_SIZE
initialization parameter to 150 MB. If you plan to use the REDO collector, you must tune this parameter to maximize REDO collector performance. In an Oracle Real Application Clusters (Oracle RAC) environment, tune this parameter on all nodes, because it is uncertain where the queue will be after a database instance starts.
Solution:
Typically, after you have configured and started the REDO collector, let it run for a while. This enables the Oracle Database autotuning feature to allocate memory for the best database performance for the STREAMS_POOL_SIZE
parameter. Using Automatic Workload Repository (AWR), check if Streams AQ has a flow control problem, such as enqueue
being blocked. If the performance is slow (for example, only 500 records are applied per second), you may need to tune the STREAMS_POOL_SIZE
parameter.
If you have at least 1 GB of physical memory in your Audit Vault Server system, set the STREAMS_POOL_SIZE
parameter to 200 MB using the ALTER SYSTEM
SQL statement, as follows:
ALTER SYSTEM SET STREAMS_POOL_SIZE=200;
The record apply rate should be 2000 records per second, which is a typical maximum rate for the REDO collector. Usually, setting the value to 200 MB is sufficient. If you are using Oracle Audit Vault in an Oracle RAC environment, set this parameter value accordingly on all nodes in the cluster, as follows:
ALTER SYSTEM SET STREAMS_POOL_SIZE=200 SID=avn;
Replace avn
with the SID for each node in the cluster.
Problem: Audit Vault agent status is blank on the Windows Services Panel.
After installing Audit Vault Agent for Microsoft Windows (32-bit), configuring a source and collectors, then starting the agent on the Audit Vault Server side, you notice that the Services Panel on the Windows system where the Audit Vault collection agent resides shows that the status is blank, rather than Started.
Solution:
This is normal behavior for the Audit Vault collection agent on Microsoft Windows systems because the service is a short-lived process. Once the Agent service process completes its task, it exits, so the status of the service will not show as Started. However, the Audit Vault collection agent is running without problems.
Run the avctl
show_agent_status
command to check the status of the Audit Vault Agent, as follows:
C:\ORACLE_HOME\agent_dir\bin\avctl show_agent_status -agentname agent1 AVCTL started Getting agent metrics... -------------------------------- Agent is running -------------------------------- Metrics retrieved successfully.
Problem: Need to debug a collection agent problem.
You want to enable debug logging while trying to diagnose an Audit Vault collection agent problem.
Solution:
Run the following set of AVCTL
commands on the command line:
avctl stop_oc4j avctl start_oc4j -loglevel debug
Check the log output in the Audit Vault collection agent $ORACLE_HOME
/av/log
directory.
Because debugging creates more logging and writing overhead, remember to disable it when you no longer need it, as follows:
avctl stop_oc4j avctl start_oc4j
See Section 7.12 and Section 7.16 for more information about the avctl stop_oc4j
and avctl start_oc4j
commands.
Problem: The Agent OC4J or Audit Vault Console OC4J fails to start.
After you run the avctl
start_oc4j
command, an avctl show_oc4j_status
command shows that OC4J is not running. Or, after you issue avctl start_av
command, an avctl show_av_status
command shows that OC4J is not running.
Solution:
Go to $ORACLE_HOME/av/log/agent.err
log file and check the error message that appears in the log.
Or, go to $ORACLE_HOME/oc4j/j2ee/home
and issue the following command to find the error message that appears on the console:
java -jar oc4j.jar
This problem is most likely caused by a port conflict. For example, if the problem is caused by an RMI port conflict, you would see a message similar to the following:
C:\oracle\product\10.2\avagentrc3_01\oc4j\j2ee\home>java -jar oc4j.jar 08/05/16 10:39:51 Error starting ORMI-Server. Unable to bind socket: Address already in use: JVM_Bind
The RMI, JMS, and HTTP ports are necessary for starting OC4J or the Audit Vault Console OC4J. The agent OC4J and Audit Vault Console OC4J can fail to start or the agent service of the Audit Vault Console can become unavailable if these ports have a conflict. If there is a port conflict, you can modify the port settings in the following files at $ORACLE_HOME/oc4j/j2ee/config
by selecting a port number not in use:
rmi.xml
jms.xml
http-web-site.xml or (av-agent-web-site.xml)
Problem: The setup command returned an error message that the connection to the source database using the credentials in the wallet was not successful.
This problem is most likely due to entering an incorrect user name or password or both when issuing the setup
command using the AVORCLDB
, AVMSSQLDB
, AVSYBDB
, or AVDB2DB
command-line utility.
Solution:
Reissue the setup
command again using the correct credentials.
Problem: Cannot start the DBAUD collector and the log file shows an error.
The DBAUD collector log file (in the Audit Vault collection agent home directory) shows the following entry:
INFO @ '17/08/2008 15:05:48 02:00': Could not call Listener, NS error 12541, 12560, 511, 2, 0
Solution:
Ensure that you completed the last step for configuring the source database and collectors, as described in Section 2.3.6, which describes how to run the avorcldb setup
command in the Audit Vault collection agent home. See also Section 2.2.3 and Section 2.2.4.
Follow these steps:
Change directories to the network/admin
directory:
$ cd $ORACLE_HOME/network/admin
Perform the cat
command on your tnsnames.ora
file.
There should be an entry similar to SRCDB1
. If there is no SRCDB1
entry in your tnsnames.ora
file, then run the avorcldb setup
command as shown in Section 2.3.6.
Try to connect to the source database with the following command, assuming your tnsnames.ora file has an SRCDB2
entry.
For example:
$ sqlplus /@SRCDB1
If the connection is successful, then your source database is set up correctly.
Try starting the DBAUD collector using the avctl
start_colletor
command.
For example:
$ avctl start_collector -collname REDO_Collector -srcname ORCLSRC1.EXAMPLE.COM
See Section 7.11 for more information about the avctl start_collector
command.
Problem: Not sure if the DBAUD and OSAUD collectors are collecting from the AUD$ table and the OS file, respectively.
After you set up both the DBAUD and OSAUD collectors, you want to check that they are collecting from the AUD$
table and OS file, respectively.
Solution:
To determine if the DBAUD collector is collecting from the AUD$
table, check the contents of the DBAUD log file, located in the $ORACLE_HOME/av/log
directory.
To determine if the OSAUD collector is collecting from the OS file, check the contents of the osaud_
collector-name_source-name_source-id
.log
file in the Audit Vault collection agent- home $ORACLE_HOME/av/log
directory.
Check each file for entries that indicate that the collector is collecting audit records.
For example, the DBAUD collector log file would have entries similar to the following:
***** Started logging for 'AUD$ Audit Collector' ***** . . . INFO @ '25/10/2008 19:08:42 -8:00': ***** SRC connected OK INFO @ '25/10/2008 19:08:53 -8:00': ***** SRC data retireved OK . . .
The OSAUD collector log file could have an entry as follows:
File opened for logging source "DBS1.REGRESS.RDBMS.DEV.US.ORACLE.COM" INFO @ '24/10/2008 18:16:18 -8:00': ***** Started logging for 'OS Audit Collector' *****
If the log files look correct, then refresh the data warehouse using the avctl refresh_warehouse
command in the Audit Vault Server shell. When this operation completes, log in to the Audit Vault Console as the Audit Vault auditor. Examine the graphical summary named Activity by Audit Event Category on the Overview page for the appearance of additional audit records in the various event categories. Increased counts for the various event categories indicate that these collectors are collecting audit records.
Problem: ORA-01017:invalid username/password; logon denied error when starting up the DBAUD_Collector or setting up the REDO_Collector.
When you try to start the DBAUD collector or configure the REDO collector, the following error message appears:
ORA-01017: invalid username/password; logon denied
Solution:
There may be a problem with your user name or password in the password file, or a problem with the wallet credentials. Try re-creating the user name and password. If the problem persists, re-create the password file. If this does not correct the problem, add the source user to the wallet again using the avorcldb setup
command. Ensure that this user is the same user name and password that you are using on the source database.
Problem: Collector log for the MSSQLDB, SYBDB, or DB2DB collector indicates that a jar file is missing.
If the following JDBC driver jar files are missing from the Audit Vault collection agent $ORACLE_HOME/jlib
library, this error appears in the collector log of the respective collector being used.
SQL Server: sqljdbc.jar
Sybase ASE: jconn3.jar
IBM DB2: db2jcc.jar
Under other circumstances, such as when you use either the AVMSSQLDB
, AVSYBDB
, or AVDB2DB
command-line utilities, the following error is appears when the JDBC driver is not in this directory:
JDBC Driver is missing. Please make sure that the JDBC jar exists in the location specified in Audit Vault documentation.
Solution:
See the following sections:
SQL Server: Section 2.4.1 for information about the sqljdbc.jar
file
Sybase ASE: Section 2.5.1 for information about the jconn3.jar
file
IBM DB2: Section 2.6.1 for information about the db2jcc.jar
files
After you download and copy these JDBC drivers in place, restart OC4J. See Section 7.16 and Section 7.12 for more information about stopping and starting the OC4J agent.
Problem: Unable to connect to source database.
When you try to verify the ORCLDB, MSSQLDB, SYBDB, or DB2DB collector using the verify
command, the following error message appears:
Unable to connect to source database
Solution:
This error appears if the source user that you specified in the verify
command for the source database does not have sufficient privileges to connect to the source database. Check if the source user has sufficient privileges to connect to the respective database. See the following sections for information about creating a source user with sufficient privileges:
Section 2.3.2 for Oracle databases
Section 2.4.2 for Microsoft SQL Server databases
Section 2.5.2 for Sybase ASE databases
Section 2.6.2 for IBM DB2 databases
Problem: Audit Vault Console does not appear in the Web browser.
When you try to access the Audit Vault Console in a Web browser, it appears to hang, or after a while it times out.
Solution:
This may be happening because the Audit Vault Console is down. To check the status of the Audit Vault Console, issue an avctl show_av_status
command in the Audit Vault Server shell. If the status indicates that the Audit Vault Console is down, issue the avctl start_av
command in the Audit Vault Server shell to restart it. Then start the Audit Vault Console in the Web browser. The Audit Vault Console should appear and let you log in to the management system of the Audit Vault auditor administrator.
Problem: Need to debug an Audit Vault Console problem.
You want to enable debug logging while trying to diagnose an Audit Vault Console problem.
Solution:
Run the following commands on the command-line:
avctl stop_av avctl start_av -loglevel debug
Then check the log output in the Audit Vault Server $ORACLE_HOME/av/log
directory.
Because debugging creates more logging and writing overhead, remember to disable it when you no longer need it, as follows:
avctl stop_av avctl start_av
See Section 7.10 and Section 7.14 for more information about these commands.
Problem: In an Oracle RAC environment, the avca drop_agent operation fails with an error when this command is issued from one of the Oracle RAC nodes.
When you try to run the avca
add_agent
command from one of the Oracle RAC nodes, the command fails.
Solution:
In an Oracle RAC environment, you must run the AVCA
commands from the node on which Oracle Enterprise Manager resides. This is the same node on which the av.ear
file is deployed.
To find where the av.ear
file is deployed, locate the $ORACLE_HOME/oc4j/j2ee/oc4j_applications/applications/av/av/WEB-INF/classes/av.properties
file is located.
Once you locate the node, run the AVCA
and AVCTL
commands from that node.
If the node on which the av.ear
file is deployed is down, deploy the av.ear
file to another node using the avca
deploy_av
command. See Section 6.4 for more information about this command.
When you run the avca
deploy_av
command, on the other node Oracle Database creates a wallet containing the default avadmin
entries. You must add the other entries, such as the source user credentials, to the wallet by using the setup
command for the appropriate utility (AVORCLDB
, AVMSSQLDB
, AVSYBDB
, or AVDB2DB
), depending on the collectors being used.
To access the Audit Vault Console from this other node, enter the following URL in the Web browser:
http://host:port/av
In this specification:
host
is the host name or IP address of the other Oracle RAC node
port
is the port number for the Oracle RAC node