Managing the Audit Vault Server and Database Firewalls

15 Managing the Audit Vault Server and Database Firewalls

Learn how to manage day-to-day Audit Vault Server and Database Firewall operations after the initial configuration is completed.

15.1 Managing Audit Vault Server Settings, Status, and Maintenance Operations

Learn how to manage the Audit Vault Server settings, status, and maintenance operations.

15.1.1 Checking Server Status and System Operation

This procedure enables you to check the server status and system operation.

Log in to the Audit Vault Server as an administrator.
Use one of the following methods to find the system status:
- Select the Home tab. The page displays overall system status information (some of which you can drill down into) for Agents, Audit Trails, Targets, Disk Space, Database Firewalls, and CPU Usage.
- Select the Settings tab, and then in the left navigational menu, select System. The status page displays detailed information for areas such as the overall server status, configuration status, and monitoring status.

15.1.2 Managing Diagnostics

You can generate diagnostic reports to find the source of errors, warnings, and other issues in Audit Vault and Database Firewall.

15.1.2.1 About Managing Diagnostics

You can run diagnostic tools to help you debug problems that may arise.

You can adjust the amount of diagnostics information gathered by setting the LOGLEVEL for different server components using the AVCLI ALTER SYSTEM SET command. When you perform the download operation, the process captures the log and trace file information, along with configuration information that is available at that time. Be aware that a change in the log level only affects those trace or log files that are generated after the change is made. For example, if you encounter a problem after you set the log level to DEBUG, then you must reproduce the issue before you run the procedure in this section to download the diagnostic report. Otherwise, the debug or trace is not captured in the report.

Be aware, however, that the DEBUG setting will generate many files, which can affect the performance of your system. Therefore, only use this setting on a temporary basis, when you are trying to diagnose problems. After you find and correct the problem, then set DEBUG to the original setting, such as ERROR.

Logging Levels

The logging levels determine the amount of information to record in the log files. The following logging levels are listed in the order of amount of information written to log files, with Debug providing the most information:

Error: Reports only critical information. This generates the least amount of log messages.
Warning: (Default) Reports warning and error messages (not supported for Web Console UI).
Info: Writes informational, warning, and error messages. This level is appropriate for testing environments but not for production.
Debug: Writes detailed messages for debugging purposes. This generates the most amount of log messages. Debug logs may contain sensitive information about the state of your system. Add the debug log level only when necessary, and remove it once debugging is complete.

System Components

You can set different logging levels for these system components:

Table 15-1 Components with Variable Logging Levels

Agent	Alert
Archive and Retrieve	Background Server Process
Data Repository	Database Firewall
Notification	Plug-in Management
Policy Management	Report Generation
SAN Storage	Transaction Log Trail
Web Console UI (has three logging levels only)	N/A

15.1.2.2 Running Diagnostics Checks for the Audit Vault Server

Follow this procedure to run diagnostics checks for Oracle Audit Vault Server.

You can run a diagnostics check for the Audit Vault Server that tracks activities such as whether necessary files exist, whether the HTTP server is running, whether the Oracle listener and other processes are running.

Log in to the Audit Vault Server console as a super administrator.
Click the Settings tab.
In the left navigation menu, click the System tab.
Under Monitoring section, click the Diagnostics link. The Diagnostics dialog is displayed as follows:

Description of the illustration diagnostics.png
In the Diagnostics dialog, do one of the following:
- Set the diagnostic category that you want to use for diagnostic (for example, for Agent, selecting Warning), and then click Save.
- Set the diagnostic categories that you want, and then click Run Diagnostics. A second window Test Diagnostics appears listing the diagnostics as they are being captured.
- To download the diagnostics report, click Download Diagnostics. A diagnostics log file (.zip) is downloaded to the location that you select. Be aware that the diagnostics zip file may contain sensitive data from your appliance. Take appropriate precautions when you transfer and store this file.
- Click Clear Diagnostic Logs to clear the current set of diagnostic logs on the Audit Vault Server.

15.1.2.3 Downloading Detailed Diagnostics Reports for Oracle Audit Vault Server

You can download diagnostics reports for Oracle Audit Vault Server to review activity and to assess other operations.

To download zip file for Audit Vault Server diagnostics:

Log in to the Audit Vault Server console as a super Administrator.
Click the Settings tab, and in the System menu, click Diagnostics.
Click the Download Diagnostics button.

A download window appears for the diagnostics zip file.
Select a file location and then click Save.
See Also:
- ALTER SYSTEM SET for details about setting the LOGLEVEL parameter.

15.1.2.4 Clearing Diagnostic Logs

Follow this process to empty your Oracle Audit Vault and Database Firewall diagnostic logs.

Log in to the Audit Vault Server console as a super administrator.
Click the Settings tab.
In the left navigation menu, select System.
Under Monitoring, click Diagnostics.
In the Diagnostics window, click Clear Diagnostic Logs, then in the confirmation window, click OK.

15.1.3 Accessing the Audit Vault Server Certificate and Public Key

You can use the Audit Vault Server console to access the Audit Vault server certificate and public key.

15.1.3.1 Accessing the Server Certificate

If you have deployed Database Firewalls, you must provide the Audit Vault Server certificate and IP address to each Database Firewall.

Log in to the Audit Vault Server console as an administrator.
Click the Settings tab.
In the left navigation menu, select Security.
In the status page, click Certificate.
The server's certificate is displayed. You can copy the certificate and provide it to each Database Firewall.
In the Security menu, click Certificate.

Related Topics

Specifying the Audit Vault Server Certificate and IP Address

15.1.3.2 Accessing the Server Public Key

You can access the Audit Vault Server public key from the Manage Archive Locations area.

You must provide the server's public key to another system in order to upload archive files from the Audit Vault Server to that system. This public key must be added to the authorized_keys file for that system. For a typical Linux installation, this file is in the user's home directory under .ssh, and its permissions must be set to 0600, or even 0400.

Log in to the Audit Vault Server console as an administrator.
Click the Settings tab.
In the left navigation menu, select Archiving.
In the page that appears, click Manage Archive Locations, and then click Create.
In the Create Archive Location window, in the Public Key field, copy this key and past it into the appropriate file on another system.

15.1.4 Changing the Keyboard Layout

You can change the keyboard layout for Audit Vault and Database Firewall depending your geographic location.

Log in to the Audit Vault Server console as a super administrator.
Click the Settings tab.
In the left navigation menu, select System.
In the page that appears, under Configuration, select Manage.
In the Manage window, select the keyboard you want from the Keyboard menu.
Click Save.
In the confirmation dialog box, select OK.
You may be logged out if your selection affects the server's time changes significantly.

15.1.5 Restarting or Powering Off the Audit Vault Server

You must be a super administrator to restart or power off the Audit Vault Server.

Log in to the Audit Vault Server as super administrator.
Click the Settings tab.
In the left navigation menu, select System.
Under Configuration, select Manage.
In the Manage window, click Reboot or Power Off.

15.2 Changing Oracle Audit Vault Server Network and Services Configurations

Use this procedure to change Oracle Audit Vault Server network and services configurations.

To set or change the network or services configuration, follow the relevant procedure below:

15.3 Managing Server Connectors for Email and Syslog

Follow this procedure to manage server connectors for email and syslog.

To set or change connector information, follow the relevant procedure below:

15.4 Configuring Remote Syslog Over TLS

Use this procedure to configure remote syslog over TLS.

Syslog provides a convenient mechanism to transfer logs from one device to another. The logs contain sensitive information. Therefore, it is important that you secure the logs during transfers. Do this by authenticating and encrypting the connection beween the client and the server. The remote feature in the syslog supports this functionality. Use this procedure to establish secure communications between the syslog clients and servers.

Prerequisites

Ensure that the normal or unencrypted remote syslog functionality works using TCP.
Complete the server side configuration. Load the imtcp module and specify the listener port.
Complete the client side configuration. Specify the remote machine to which the logs are sent.
Upon completion of the server side and client side configuration. Restart the syslog service.
Restart the syslog service in case any of the devices were added, modified, or activated.
Ensure the logs from the client are listed in the log file of the server. This is a confirmation and you can proceed to securing the communication channel.

To load the imtcp module and to specify the listener port, modify the /etc/rsyslog.conf file as follows:


# listen for tcp input
$ModLoad imtcp
# listening on port <port number>
$InputTCPServerRun <port number>
# Remote logs written to /var/log/messages.
*.* /var/log/messages

To specify the destination remote machine to which the logs will be sent, modify the /etc/rsyslog.conf file as follows:


# Forward messages to remotehost:port
*.* @@<Ip address of the remote host>:<port number>

Syslog contains modules, protective transport layer, and digital certificates to ensure mutual authentication. It covers many aspects. The syslog messages are encrypted in transit. The syslog sender authenticates to the syslog receiver. The receiver is able to identify and in return authenticates to the syslog sender. The receiver performs few checks to validate if it is the valid recipient of the messages. This kind of mutual authentication and hand shake prevents any kind of attacks.

The syslog mutual authentication system makes use of CA certificate and peer certificates. In case there is no signed certificate available, the user can create a self signed certificate using OpenSSL. The server must have the CA (certificate authority) certificate and it’s own digital certificate. These certificates enable SSL operation that provides the necessary crypto keys used to secure the connection.

Syslog makes use of GTLS module as the network stream driver. Syslog has TLS protected transport security feature and ensures messages are encrypted. It makes use of digital certificates to ensure mutual authentication.

Configuring the server

The server configuration involves specifying the location of the certificates, the GTLS driver to be used, and starting of the listener. The following example is applicable to Linux based remote server only. It is different for other platforms. To make any changes to the server's syslog configuration, refer to the documentation of the specific syslog server. Modify the /etc/rsyslog.conf file as follows:


# listen for tcp input
$ModLoad imtcp
# make gtls driver the default
$DefaultNetstreamDriver gtls
# certificate files
$DefaultNetstreamDriverCAFile /path/to/cacert.pem
$DefaultNetstreamDriverCertFile /path/to/servercert.pem
$DefaultNetstreamDriverKeyFile /path/to/serverkey.pem
# Auth mode and permitted peers
$InputTCPServerStreamDriverAuthMode x509/name
$InputTCPServerStreamDriverPermittedPeer <permittedHost>
$InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode
# Listening on port <port number>
$InputTCPServerRun <port number>
# Generic log file watching
$WorkDirectory /var/cache/rsyslog
# Remote logs
*.* /var/log/messages

Configuring the client

Syslog sends messages to a remote system from the client (Audit Vault Server or the Database Firewall). The client configuration involves specifying the location of the certificates, the GTLS driver to be used, and specifying the destination of the messages. Modify the /etc/rsyslog.conf as follows:


# make gtls driver the default
$DefaultNetstreamDriver gtls
# certificate files
$DefaultNetstreamDriverCAFile /usr/local/dbfw/syslog/certs/cacert.pem
$DefaultNetstreamDriverCertFile /usr/local/dbfw/syslog/certs/clientcert.pem
$DefaultNetstreamDriverKeyFile /usr/local/dbfw/syslog/certs/clientkey.pem
# Auth modes and permitted peers
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer avs08002719479d
$ActionSendStreamDriverMode 1 # run driver in TLS-only mode
# Send to remote system
*.* @@<Ip address>:<port number>

Note:

The rsylog.conf is generated from the template file:

/usr/local/dbfw/templates/template-rsyslog-conf

The client settings must also be made to the template file. Any changes made to this template is persistent and preserved even after the reboot of the appliance.

Creating and Using SSL Certificates

Execute the following command to create CA certificates:

openssl req -new -x509 -keyout private/cakey.pem -out cacert.pem -days 365 -config openssl.cnf
Execute the following commands to create server certificates using the CA certificate:

openssl req -nodes -new -x509 -keyout serverkey.pem -out serverreq.pem -days 365 -config openssl.cnf

openssl x509 -x509toreq -in serverreq.pem -signkey serverkey.pem -out tmp.pem

openssl ca -config openssl.cnf -policy policy_anything -out servercert.pem -infiles tmp.pem
Execute the following commands to create client certificates using CA certificate:

openssl req -nodes -new -x509 -keyout clientkey.pem -out clientreq.pem -days 365 -config openssl.cnf

openssl x509 -x509toreq -in clientreq.pem -signkey clientkey.pem -out tmp.pem

openssl ca -config openssl.cnf -policy policy_anything -out clientcert.pem -infiles tmp.pem
Transfer the following certificates to the specific location on the log server.

cacerts.pem,servercert.pem,serverkey.pem
Transfer the following certificates to the specific location on the client.

cacerts.pem,clientcert.pem,clientkey.pem

15.5 Archiving and Retrieving Audit Data

Learn how to archive and retrieve audit data.

15.5.1 Enabling Automatic Archival

Oracle Audit Vault and Database Firewall 20.1 supports automatic archival to an NFS configured location.

When the online period of the data on the tablespace expires, it is automatically archived without user's intervention. The data is removed from the online location and is available in the archive location. The data cannot be deleted online manually.

Oracle recommends enabling Automatic Remote Archiving. To enable it, you must configure at least one NFS archive location.

Oracle AVDF Release 20.1-20.8
Oracle AVDF Release 20.9 and later

Configure a list of NFS archive locations. Ensure that NFS is configured on the Audit Vault Server as the archive location.
Log in to the Audit Vault Server console as an administrator.
Click Settings tab.
In the left navigation menu, select Archiving.
On the main page, click Data Retention.
Click Enable button, against Auto Archive Status.

Log in to the Audit Vault Server console as an administrator.
Click the Data Retention tab.
Click Remote Archiving in the left navigation menu.
Click Enable Automatic Remote Archiving at the top.

Related Topics

15.5.2 Starting an Archive Job Manually

To start an archive job, you must have configured at least one archive location.

Oracle recommends that you use NFS to transfer data to an archive location. If you use Secure Copy (SCP) or Windows File Sharing (SMB) to transfer data to an archive location, then your data files are first copied to a staging area in the Audit Vault Server. Therefore, you must ensure that there is additional space in the file system. Else copying the data file may fail. Transferring large files using SCP or SMB may take long.

You can register a remote file system by using the AVCLI command REGISTER_REMOTE_FILESYSTEM.

Oracle AVDF Release 20.1-20.8
Oracle AVDF Release 20.9 and later

Log in to the Audit Vault Server console as an administrator.
Click Settings tab.
In the left navigation menu, select Archiving.
On the main page, click Data Retention.
Under Job Name, enter a name for the archive job.
Under Archive Location, select the archive location from the list.
From the list, select the files you want to archive. The files listed are those for which the Months Online period has expired according to the target's retention policy.
Click Archive.
Tip:
If the archive job fails and you receive error OAV-46599, check your RMAN configuration as autobackup in the controlfile should be set to off.
```
rman /RMAN> configure controlfile autobackup off;
```

Log in to the Audit Vault Server console as an administrator.
Click the Data Status tab.
Click Remote Archiving in the left navigation menu.
Click the Archived Data tab.
Select one or more archived data files from the list by clicking the box to the right of the target name.
Click Move to Remote.
In the dialog box that appears enter the job name.
Select a remote archive location from the drop down list. The selected archived data files will be moved to the remote archive location selection.
Click Save.
Tip:
If the archive job fails and you receive error OAV-46599, check your RMAN configuration as autobackup in the controlfile should be set to off.
```
rman /RMAN> configure controlfile autobackup off;
```

Related Topics

15.5.3 Retrieving Oracle Audit Vault and Database Firewall Audit Data

You can retrieve data files for a specific target and time range.

The Months Archived value in a targets retention (archiving) policy determines how long the target's data is available to retrieve to the Audit Vault Server. When the Months Archived period expires, the data is no longer available to retrieve, however, it continues to reside in the archive location.

Log in to the Audit Vault Server as an administrator.
Click the Settings tab, and from the left navigation menu, click Archiving.
Select Retrieve sub tab on the main page.
Under Retrieve Request, enter the following:
- Target menu: Select the target.
- Start Date field: Enter the start date, optionally using the date icon to select from a calendar. The start and end dates are associated with the event time (the time the event occurred).
- End Date field: Enter the end date, optionally using the date icon to select from a calendar.
Click the Retrieve button.
Note:
- You can check the status of the retrieve job in the Jobs dialog that can be accessed from the System tab in the left navigation menu.
- When the retrieved data files are available, they are listed in the Retrieved Datafiles section of the Retrieve tab, and the data will be visible in reports.
- Starting Oracle AVDF 20.4, the datafiles archived in NFS locations are deleted from the location after the retrieve job completes.
To purge retrieved files when no longer needed, from the Retrieved Datafiles section. Select the files you want to unload from the system, and then click the Release button. Once the release is successful, the data is not visible in reports.
After the retrieved data files are released, they are now eligible to be archived again. If they are not needed anytime soon, then they should be archived to release disk space to the system.
Note:

Alternately, you can view or get the tablespaces archived by following these steps:
1. Connect to the primary Audit Vault Server using SSH.
2. Connect to SQL*Plus as administrator.
3. Run the following commands:
```
set linesize 100
```
```
column TABLESPACE_NAME format a30
```
```
column EVENT_MONTH format a15
```
```
SELECT * FROM TABLE(avsys.ilm.get_target_eventmonth_for_tablespaces);
```
4. The above query displays the results with TABLESPACE_NAME, SECURED_TARGET_ID and EVENT_MONTH indicating the month for which the data is stored for the respective target ID for each tablespace. This information can be used to retrieve data.

Related Topics

15.6 Managing Repository Encryption

Managing the repository encryption key includes tasks such as rotating the master encryption key or changing the keystore password.

15.6.1 About Oracle Audit Vault Server Repository Encryption

Learn about repository encryption.

Encryption of the Oracle Audit Vault Server's event repository is enabled on new installations of Oracle Audit Vault and Database Firewall. This feature uses Oracle Database's Transparent Data Encryption (TDE) to encrypt all audit event data stored in the Audit Vault Server, data stored in external SAN storage, and data stored in archive locations.

15.6.2 Rotating the Master Key for Repository Encryption

Rotating encryption keys adds a layer of security to your encrypted data.

You should rotate the master encryption key for the Audit Vault Server's event repository on a regular basis, according to your organization's guidelines. It is also a good practice to rotate the encryption key as needed. For example, when a person who had access to your master key leaves your organization.

Note:

If you restore the Audit Vault Server from a backup, the restore operation restores the system to a point in time. Therefore, restoring the system may reinstate an older encryption key.

Log in to the Audit Vault Server console as a super administrator.
Click the Settings tab.
In the left navigation menu, select Storage tab.
In the page that appears, click Repository Encryption tab.
In the Rotate Master Key section, in the Keystore Password field, enter the keystore password.

This password is originally set as a required post-installation step.
Click the Re-key button.

15.6.3 Changing the Keystore Password

For better security, periodically change the keystore password.

The keystore password for repository encryption is originally set as a required post-installation step. It is the same as the Event Repository Encryption password. You only need this password for restore operations, not backup operations. Thereafter, you can change this password in the Audit Vault Server console.

Log in to the Audit Vault Server console as a super administrator.
Select the Settings tab.
In the left navigation menu, select Storage.
In the page that appears, click Repository Encryption tab.
Under Change Keystore Password section, enter the following:
- Old password
- New password
- Re-enter new password
Click Change Password button.

See Also:

Backup and Restore of Audit Vault Server for more information on using the keystore password to restore the Audit Vault Server from backup files.

15.6.4 Backing Up TDE Wallets

You can back up TDE wallets to preserve information.

It is important to perform regular backups of Oracle Audit Vault Server, which include the TDE wallet. However, if you cannot back up Oracle Audit Vault Server, then you should, at a minimum, do regular backups of the TDE wallet at this location:

/usr/local/dbfw/etc/wallets/dbfwdb_wallet

Oracle Audit Vault and Database Firewall does not provide the ability to back up wallets. You should securely back the wallet up in a remote location.

15.6.5 Data Encryption on Upgraded Instances

Learn about data encryption on upgraded instances in Oracle Audit Vault Server.

Phases of Data Encryption

This topic contains a detailed procedure that can be used to start data encryption process.

WARNING:

Do not run data encryption processes on a newly installed Oracle Audit Vault Server or on a system that has been upgraded from fresh install of release Oracle Database 12.2.x. With versions of Oracle Database 12.2.0 and above, all of the new installations have encryption enabled automatically. Thus, all of the table spaces are encrypted by default.

The data encryption process happens in two phases:

Enabling Data Encryption:

This phase is automatic and data encryption is enabled while performing an Audit Vault Server upgrade. The upgrade process prompts for a keystore password on standalone and primary systems. Upon successful upgrade, data encryption is automatically enabled. The newly created table spaces thereafter are automatically encrypted. However, table spaces created before upgrade continue to be in clear text.
Encrypting existing clear text table spaces:

This phase is triggered by the user. To encrypt the existing clear text table spaces, the user must initiate the data encryption process. This process is triggered by running the /usr/local/dbfw/bin/avdf_data_encryption.sh script. The detailed steps for encrypting existing clear text table spaces triggered by the user are available in this topic.

Before you begin

The rate of encryption is approximately 20 to 50 seconds to encrypt 1 GiB of data, depending on the hardware profile of the system.
To begin the process of encrypting the table spaces, the user must execute the /usr/local/dbfw/bin/avdf_data_encryption.sh script as root.
Ensure to take AVDF backup prior to the encryption process.
The user must have root operating system user privileges to execute this procedure. Ensure the proper privileges are obtained.
The encryption process script must be executed on standalone system or on the primary in a HA set up. Ensure that the standby system is also up and running before running the encryption script. The script may result in an error if the standby system is down. The script encrypts table spaces on both the primary and standby system.
Ensure that the database is up and running prior to executing the encryption process. To verify the status of the database, log in as root user and execute the command /etc/init.d/dbfwdb status
The encryption process script stops all the jobs running in the background. Ensure there is no critical process running that may be impacted.

Note:

Data encryption is not completely enabled on HA system, until the primary is successfully upgraded. After a successful upgrade, all clear text table spaces are in one of the following states:

online
offline local (offline but the data file resides on the AVS)
offline remote (offline but the data files reside on the remote archive location)
online retrieved by user
online retrieved by a trail

To start Data Encryption process:

Log in to the system as root user.
Execute the following command to start encryption:
/usr/local/dbfw/bin/avdf_data_encryption.sh start

The following message is displayed on the screen:

**************************************************************
* This script will encrypt all online tablespaces and create *
* a background job to encrypt offline tablespaces.           *
* Encrypting online tablespaces could potentially take long  *
* time depending on the size of the online data collected.   *
* Note that during this time                                 *
*   - There will be no access to Web UI console.             *
*   - Event collection will be shutdown.                     *
*   - AV agents will not be able to connect.                 *
*   - AVCLI will not be able to connect.                     *
*                                                            *
*  NOTE: It is recommended to take backup before continuing. *
**************************************************************
Do you want to continue (Y/N):

Type Y to continue with encryption.

The following message is displayed:

************************************************************
* Do not interrupt this script execution or reboot.        *
* To stop the script execution use                         *
* 'avdf_data_encryption stop' command.                     *
* Check /root/avdf_data_encryption.log to track progress   *
************************************************************

Note:

At this point, it is recommended to move the process to background by executing Ctrl+z followed by bg. Alternately to keep the session alive, the user can execute the command ssh -o ServerAliveInterval 20

The following messages are displayed on the screen:

Successfully encrypted online table spaces. 
System is ready for use.
Offline table space encryption can be managed on the AVS GUI.

Note:

Contact My Oracle Support with the printed output in the event of a failure.

The following message is displayed in the /var/log/avdf_data_encryption.log file:
```
Encrypting <tbsp name> Tablespace : % done
```
Once the encryption process is successfully completed, another job to encrypt offline table spaces is created and enabled in the background. All the services appear online and the following message is displayed:
System is ready for use
In case the encryption process fails, the /var/log/avdf_data_encryption.log file displays the following error message.
Failed to encrypt table spaces: Please contact Oracle Support
Execute the following command to stop encryption:
/usr/local/dbfw/bin/avdf_data_encryption.sh stop

Note:

Ensure to execute the stop command only after you see the following message in the /var/log/avdf_data_encryption.log:

You may issue stop command to gracefully stop the encrypting process

Note:

Once the stop encryption command is executed, the encryption process exits only after encrypting the current table space that is being encrypted. It is always recommended to run the script again to complete the encryption process.
In case the user decides to perform a reboot of the system during the encryption process, it stops at the current table space that encryption last accessed. The user can decide to run the script again to complete the encryption process.
In case the dbfwdb service terminates unexpectedly, contact Oracle Support. The encryption script will not run if this service is down.
The encryption process collects all the logs to /var/log/avdf_data_encryption.log file securely.
After all online table spaces are encrypted, a background job ENCRYPT_OFFLINE_TBSP is enabled to perform encryption of offline table spaces. This job encrypts all table spaces for those data files that reside locally on the system. In case the data file is located on the remote location nfs/scp/smb, the data file is copied to the local system, encrypted, and setup for re archival. The user must manually perform the re archival process to ensure that the data file in the remote location scp/smb is encrypted. The user can navigate to Settings and Repository Encryption page to view a list of offline table spaces that are not encrypted. If the data file is not available, the message displayed indicates the same.

The process of encrypting offline table spaces can be in one of the following states.

Message	Description
`NOT YET STARTED`	The user has not executed the script to encrypt table spaces.
`COMPLETED`	All online and offline table spaces are encrypted. Any new table spaces created will also be encrypted. This is the final state.
`IN PROGRESS`	The background job is currently encrypting offline table spaces.
`USER`	The background job is waiting for user input. User must visit the Repository Encryption page and take appropriate action.
`ERROR`	There was an error in encrypting one or more table spaces. The user must download the diagnostics and provide that to Oracle Support.
`TRAIL`	The table space has been retrieved by a trail as it is collecting old data. Wait for the trail to release the table space.

In the ERROR state the background job is disabled and hence the user, after fixing the cause of the error must re-enable the job from the Repository Encryption page.
In the event of system reboot, power failure, switch over, or fail over the user can execute the encryption process again.

15.7 Backup and Restore of Audit Vault Server

Learn about backup and restore of Audit Vault Server.

15.7.1 About Backup and Restore of Audit Vault Server

Learn about the details of backing up and restoring Audit Vault Server.

Audit and network event data that is collected by Oracle AVDF is stored in the embedded Audit Vault Server repository (Oracle Database). This repository also contains reports and configuration data. Audit Vault Sever administrator must take backup of Audit Vault Server on a regular basis. In case of an Audit Vault Server outage, the administrator can install a new Audit Vault Server and restore from the backup taken earlier. This minimizes data loss. For example, if the Audit Vault Server machine faces hardware failure, a new Audit Vault Server can be restored using the most recent backup. This process brings the entire data back to the point when the last backup was taken.

Audit Vault Server backup contains the collected data and the configuration data. The configuration data includes data associated with Agent registration, target registration, wallets, etc. The configuration data is located in the OS files and the embedded repository. Audit Vault Server backup does not take backup of the archived data files. As long as the restored Audit Vault Server has access to these archived files, the data can be retrieved and viewed.

The size of the backup is dependent on the size of the data collected and stored in the embedded repository.

Database Firewall does not require a backup. Audit Vault Server can reapply all existing configuration of the monitoring points to the Database Firewall during the post restore steps.

Types of Backup Supported by Audit Vault Server

Audit Vault Server supports full backups and incremental backups. An incremental backup contains only the new data and configuration changes since the previous backup. For example, a full backup is taken on Sunday, and then incremental backups are taken on Monday, Wednesday, and Friday. A full backup and the subsequent incremental backups together make a backup set. Oracle recommends to test and validate the backup set regularly to determine whether the backup set can be restored. Backup validation can be included as part of the backup cron job. An example is mentioned in the later topics.

Audit Vault Server backup functionality supports both hot (online) and cold (offline) backups. A hot backup is a backup that is taken when the Audit Vault Server is up and running and when the embedded repository is online. This option ensures that the deployment continues to monitor the targets when backup is in progress. The administrator can access reports and make changes to the configuration in this case. Oracle recommends to setup hot backup as a best practice.

A cold backup requires the Audit Vault Server repository to be offline. In this case, the targets are not protected and the Audit Vault Server console is unavailable until the backup is completed.

Types of Backup Locations Supported by Audit Vault Server

A backup location is a user defined directory path to store the backup files. The location can be on a local disk or on NFS (Network File System). Oracle recommends using NFS for the backup location as a best practice. The location of the backup files should be accessible by the Audit Vault Server as root and oracle OS user.

Redundancy of Audit Vault Server Backups

Redundancy settings control as to when a backup set is marked as obsolete. For example, if redundancy is set to 2 and the third full backup is taken, the files in the first backup set are marked as obsolete. It is recommended to take more backups than the redundancy setting within a period of 30 days to ensure the obsolete backup files are purged properly. Redundancy is an important setting in managing storage space in the backup location.

Encryption of Audit Vault Server Backup

The Audit Vault Server backup contains information of the OS files and the embedded repository. The embedded repository is encrypted using Transparent Data Encryption and the pertaining backup files are also automatically encrypted. However, the OS configuration files are not encrypted and the backup of these files are also not automatically encrypted. Audit Vault Server backup provides a setting to encrypt the backup of the OS configuration files. This allows to restrict access to configuration like wallets present in the backup location.

Tasks Involved in Audit Vault Server Backup and Restore

The following are the backup tasks:

Planning for Audit Vault Server backup
Configuring Audit Vault Server for backup
Performing Audit Vault Sever backup
Monitoring Audit Vault Server backup

The following are the restore tasks:

Planning for restoring Audit Vault Sever
Performing necessary checks for restore
Restoring Audit Vault Server from backup taken earlier

15.7.2 Audit Vault Server Backup and Restore in High Availability Environment

Learn about backup and restore of Audit Vault Server in a high availability environment.

In a high availability environment, there are two Audit Vault Servers paired (primary and standby). The primary Audit Vault Server is the active server that provides the Audit Vault Server functionality. The standby is automatically synchronized (audit data and network event data) and has a consistent copy of the primary. The backup operation must be performed on the primary Audit Vault Server and not on the standby Audit Vault Server. When Audit Vault Server is restored from a backup taken on primary Audit Vault Server, the restored Audit Vault Server is configured in standalone mode. It must be paired again with another Audit Vault Server to achieve high availability.

The following diagram illustrates different aspects involved in backup and restore functionality of Audit Vault Server configuration in a high availability environment.

Figure 15-1 Audit Vault Server Backup and Restore in High Availability Environment

Description of "Figure 15-1 Audit Vault Server Backup and Restore in High Availability Environment"

15.7.3 About Audit Vault Server Backup and Restore Utility

Learn how the Audit Vault Server backup and restore utility works.

Audit Vault Server backup and restore utility provides all the necessary functionality to perform backup and restore related operations on the Audit Vault Server. This utility avbackup must be run as the root OS user. It is located in /var/lib/oracle/dbfw/bin.

The backup utility supports:

full backup and incremental backup
hot and cold backup
local and NFS backup locations
restore of data and configuration to a new Audit Vault Server from backup taken earlier
validation of the backup taken earlier

Audit Vault Server Backup and Restore Commands

The following are the commands used to backup and restore Audit Vault Server:

Command Task

avbackup config

To configure Audit Vault Server backup settings.

avbackup backup

To perform backup of Audit Vault Server.

avbackup validate

To validate the backup file set.

avbackup restore

To restore Audit Vault Server using backup file taken earlier.

Audit Vault Server Backup Settings

Configure the Audit Vault Server backup and restore functionality by running the avbackup config command as OS root user.

Note:

The following settings must be the same for both backup and restore operations.

Field / Category	Description
`MAXPIECESIZE`	This setting controls the maximum size of a backup file. A backup can contain multiple backup files, otherwise known as backup pieces. The valid maximum file size depends on the actual file system. Setting `MAXPIECESIZE` is required only if `CHANNEL_PARALLELISM` is set to 1. Valid values: 1 - 2048 Kbs/Mbs/Gbs Default value: `2G` (2 GB) For example: `MAXPIECESIZE[2G]:`
`BACKUP_DIR`	The directory that stores all the backup files. The directory path is limited to 200 characters. Default value: `/backup` After this directory is specified, do not change this directory path as Oracle Recovery Manager (RMAN) tracks the backup files in this directory. The files are written to this directory and access to this directory is managed by oracle user (`UID 503`) who is part of the `oinstall` group (`GID 504`). Oracle AVDF backup utility automatically uses this directory path during the restore operation. When the `BACKUP_DIR` setting is changed, performing a full backup is a must. This is to ensure the backup set is complete and located in the new `BACKUP_DIR` location. Note: Do not change this setting between full backup and incremental backup, as the incremental backup may fail. Change this setting only to put the new full backup files into a different location. If this rule is not followed, it may cause failure of restore operation. The `BACKUP_DIR` location value must be different for online backup and offline backup. If this rule is not followed, it may corrupt the backup file and failure of restore operation. Do not change this setting between full backups, as the redundancy setting may not apply correctly. In case `REDUNDANCY` is set to 2 and `BACKUP_DIR` is changed before the third full backup is taken, then the first backup set is not purged after taking the third full backup. If `BACKUP_DIR` is not changed before the third full backup is taken, the first backup set is purged after taking the third full backup. The backup directory must be a mounted file system with enough free space to hold the backup files. This can be NFS (Network File System). Tape storage is not supported as a backup location. All the backup files are saved in `BACKUP_DIR` location, except when `CHANNEL_PARALLELISM` is specified to a value greater than 1. When `CHANNEL_PARALLELISM` is greater than 1, `CHANNEL_LOCATION` must be specified for each channel, and the specified locations are used as backup locations. The `BACKUP_DIR` location must have enough free space to hold the backup files. The space requirement depends on `REDUNDANCY` setting. See Also: Backup Location Storage Requirements in Backup of Audit Vault Server
`BACKUP_TYPE`	Specifies the type of backup to be performed. Enter `HOT` or `COLD`. A `HOT` backup is an online backup when the Audit Vault Server is fully operational. A `COLD` backup is an offline backup that requires the embedded repository (Oracle Database) in the Audit Vault Server to be shut down. Default value: `HOT` Archive log mode must be enabled for hot (or online) backup. Enabling archive log mode is quick and requires a restart of the embedded repository. For a cold (offline) backup, the embedded repository in Audit Vault Server is shut down during the backup process. Shutting down the embedded repository results in downtime of monitoring until the backup operation is completed. See Also: Backup Strategy in Backup of Audit Vault Server
`PASSWD`	The password used to encrypt the configuration data stored in the OS files. If this is omitted, then the backup OS files are not encrypted. However, the data backed up from embedded repository is always encrypted by TDE (Transparent Data Encryption). If this password is specified during backup, it must be provided for restore operation later. There is no way to recover if the password is lost for restore operation. Hence, it is recommended to keep the password stored in a safe location. Default value: Not set. Password must be at least 8 characters long and must contain a number, uppercase letter, lowercase letter, and a special character `. , _ + :`
`CHANNEL_PARALLELISM`	Specifies the number of channels (processes) used for running the commands. In case there are multiple backup locations and each backup location is mounted on separate physical disks, then this setting can be used to increase the speed of backup operations. Set this parameter to match the number of physical disks used for backup locations. If this parameter is set to 1, then the value for `MAXPIECESIZE` must be specified. If this setting is greater than 1, then the locations and section size (`CHANNEL_LOCATION` and `SECTION_SIZE`) must be specified. Default value: `1` For example: `CHANNEL_PARALLELISM[1]:4`
`CHANNEL_LOCATION`	Specifies the location for each channel and is required when `CHANNEL_PARALLELISM` is greater than 1. Based on the value of the `CHANNEL_PARALLELISM`, specify the same number of channel locations. If `CHANNEL_PARALLELISM` is 2, then provide values for `CHANNEL_LOCATION_1` and `CHANNEL_LOCATION_2`. Default value: None As shown in the example below, it is recommended to specify each location in a different physical disk. For example: `CHANNEL_LOCATION_1[]:/disk_1` `CHANNEL_LOCATION_2[]:/disk_2` `CHANNEL_LOCATION_3[]:/disk_3` `CHANNEL_LOCATION_4[]:/disk_4` Note: The `CHANNEL_LOCATION` takes precedence of the `BACKUP_DIR` for backup in case of the embedded repository. However, the backup of the OS files continue to be located in `BACKUP_DIR`.
`SECTION_SIZE`	This determines the section size for each channel to backup. This setting is required only if `CHANNEL_PARALLELISM` is set to greater than 1. If `CHANNEL_PARALLELISM` is set to more than 1, then the Audit Vault Server backup is performed in parallel. The datafiles in the embedded repository are split into logical sections for which the size is determined by this setting. Each channel performs a backup of one section at a time. Specify this size considering the maximum file size of the channel location and the biggest datafile in the embedded repository. A small value could result in an increase in the number of files created in the channel location and a much larger value may be beyond the maximum file size that the channel location can support. Default value: None For example: `SECTION_SIZE[]:32G`

Note:

Use CHANNEL_PARALLELISM, CHANNEL_LOCATION, and SECTION_SIZE configuration parameters for a database with a size of 1 TB or more.

The following settings can be different for each backup, validate, or restore operations:

Field / Category	Description
`TMP_DIR`	It is a temporary working parent directory that stores all temporary files and logs. It must have at least 100 MB of free space. The oracle user must have `read-write` access to this directory. Default value (Oracle AVDF release 20.7 and earlier): Directory path `/tmp` For example: `TMP_DIR[/tmp]: /tmp/BCKTMP` Default value (Oracle AVDF release 20.8 and later): Directory path `/usr/local/dbfw/tmp` For example: `TMP_DIR[/usr/local/dbfw/tmp]: /usr/local/dbfw/tmp/BCKTMP`
`KEEP_LOGS`	It determines whether the log files are stored after a successful backup operation. Logs are always kept after a failure. Enter `YES` to retain logs after backup, validate, or restore operations. Enter `NO` to automatically delete logs after successful backup or restore operations. The logs are always saved if the backup or restore operation fails irrespective of the `KEEP_LOGS` setting. Default value: `NO` For example: `KEEP_LOGS[NO]:yes`
`INCREMENTAL`	This setting provides an option to choose from full or incremental backup. Enter `0` for a full backup. Enter `1` for incremental backup. This setting applies only to backup. Default value: `0` (full backup) For example: `INCREMENTAL[0]:0`
`USE_NEW_IP`	Specifies whether to use a new or an existing IP address for the restored system. This setting applies only to restore operation. This setting is ignored if the Audit Vault Server is restored on an OCI image. The allowed values are `Y` or `N`. Default value: `N` If `USE_NEW_IP` is set to `N`, the restored Audit Vault Server automatically switches to the IP address of the Audit Vault Server when the backup was taken. If `USE_NEW_IP` is set to `Y`, the restored Audit Vault Server retains the IP address set during fresh installation of this Audit Vault Server. Case 1 Backup of Audit Vault Server with IP1 Audit Vault Server with IP1 goes offline `USE_NEW_IP` set to `N` for restore operation Restore of Audit Vault Server with IP2 The IP address of the final restored Audit Vault Server is switched from IP2 to IP1 Case 2 Backup of Audit Vault Server with IP1 Audit Vault Server with IP1 goes offline `USE_NEW_IP` set to `Y` for restore operation Restore of Audit Vault Server with IP2 The IP address of the final restored Audit Vault Server remains IP2 For example: `USE_NEW_IP[N]:Y`
`REDUNDANCY`	Specifies the number of full backups to be kept before purging obsolete backup sets. This setting applies only to backup. This setting impacts the backup storage specified in the `BACKUP_DIR` parameter. For example: If this value is set to `2`, the first backup set is purged when the third full backup has completed successfully. Default value: `1` See Also: Backup Location Storage Requirements in Backup of Audit Vault Server

Note:

Audit Vault Server backup and restore operations may take a long time. When you use SSH to connect to Audit Vault Server to perform these operations, ensure you have configured SSH properly to avoid SSH timeouts.
In this table and throughout this document, 1 GB represents 2 to the 30th power (2³⁰) bytes or in decimal notation 1,073,741,824 bytes.

15.7.4 Setting Up NFS for Audit Vault Server Backup and Restore

Oracle recommends using Network File System (NFS) for the Audit Vault Server backup location. This location must be the same for backup and restore operations..

For example, if BACKUP_DIR for the Audit Vault Server backup is /var/lib/oracle/avs_backup, then configure the same BACKUP_DIR (/var/lib/oracle/avs_backup) for the Audit Vault Sever restore operation. This location (for example, /var/lib/oracle/avs_backup) should be owned by oracle:oinstall with read-write permission.

For example, to mount avs_backup to the NFS server, follow these steps:

Log in to the Audit Vault Server through SSH and switch to the root user.

See Logging In to Oracle AVDF Appliances Through SSH.

Run the following command:

mount -t nfs <NFS_IP>:<export_path> /var/lib/oracle/avs_backup

Note:

Configure the same mount point on the Audit Vault Server before backup and restore.
The exact mount command may vary.
Make sure that the oracle user has read, write, and execute permissions for the directory that you created as the mount point.
Ensure that BACKUP_DIR is set to /var/lib/oracle/avs_backup for both backup and restore.
If you updated /etc/fstab to add the mount point, it reverts to the original state when the system is restarted.

15.7.5 Backup of Audit Vault Server

Learn how to perform Audit Vault Server backup.

These are the steps involved in the backup of Audit Vault Server:

Planning for Audit Vault Server backup.
Configuring the Audit Vault Server backup.
Performing or automating Audit Vault Server backup.
Monitoring the Audit Vault Server backup.

Planning for Backup

Planning for backup of Audit Vault Server involves the following aspects:

Aspects	Description
Backup strategy	Backup strategy involves setting up a backup schedule that meets corporate requirements or guidelines. Frequent backups can minimize data loss but impact system performance. A full backup can take longer than an incremental backup. A good backup strategy aims to minimize both the data loss and the impact on system performance. Best Practice: A full online backup once a week and multiple incremental backups during the week is optimal. Consider backup optimization mentioned in this table below to reduce backup times. Note: Do not keep both online and offline backup files in the same directory. In such a case, make sure they are stored in two separate backup locations.
Backup location storage requirements	Ensure there is sufficient disk space in the backup location to store the backup files. The backup location contains the OS configuration files (`.tar` file) and the related backup of the repository. There may be multiple backup sets in the backup location depending on the redundancy setting. For example, if redundancy is configured to 3, the backup location keeps up to 4 sets of backups before it begins to purge the obsolete backup files. Determining the space requirement for the backup location, depends on the size of the Audit Vault Server repository (Oracle Database). Run the following SQL query as sysdba user for an approximate calculation: `sqlplus / as sysdba Enter password: password SELECT SUM (BYTES)/1024/1024/1024\|\|' GB' FROM DBA_DATA_FILES` This calculation is a simple estimation of a full backup file. For each incremental backup, add more disk space in addition to the above specified amount. It is not possible to calculate the specific size for incremental backup in a live system. Use this as a guideline after Audit Vault Server is deployed and is stable. To calculate a simple estimate of incremental backup file, find the difference between two full backup files. Divide this by the number of incremental backup specified. This provides an average size of an incremental backup file. Note: Ensure oracle user and group oinstall users have `read`, `write`, and `execute` permission to the backup directory.
Backup type	Oracle AVDF supports online and offline backup. Offline backup requires Audit Vault Server downtime. Offline backup does not have data loss up to the time the backup is taken. Online backup allows taking backup when Audit Vault Server is online. There is a potential loss of data involved in online backup. Online backup requires archive log mode to be enabled for the database. Oracle recommends taking online backup.
Retention	The retention of the backup depends on the `REDUNDANCY` setting. Specify an appropriate value based on the organization's policy. In most situations, `REDUNDANCY` is set to a value greater than 1 to keep more than one backup set. For obsolete backup files to be purged properly, schedule more full backups than the retention configuration within a period of 30 days.
Channel parallelism	Setting higher parallelism (`CHANNEL_PARALLELISM`), can improve backup performance. However, it only improves performance if it matches the actual physical number of disks available. If there is only one physical disk, it does not improve the backup performance even if `CHANNEL_PARALLELISM` is set to greater than 1.
Backup optimization	It is recommended to increase the channel parallelism to match the physical number of disks. This can improve the backup performance. When channel parallelism is set to a value greater than 1, then set the section size too. Section size defines how the datafile is handled by each channel during backup operation. To improve performance specify different `CHANNEL_LOCATION` on a different physical hard disk. Specifying all the channel locations to the same path, does not utilize the benefits of parallelism. Increasing the maximum piece size can also improve the performance if channel parallelism is set to 1. The maximum piece size depends on the file size supported by the filesystem.

Space Required for Backup Files

Determine the amount of space needed for backup files. The amount of space needed for backup files is determined by the size of Audit Vault Server repository. You can obtain an upper estimate of the backup file size for the database by running the following SQL query on the Audit Vault Server:


sqlplus system
Enter password: password
SELECT SUM (BYTES)/1024/1024/1024||' GB' FROM DBA_DATA_FILES

Note:

Ensure the RAM size and disk size in the new system is equal or greater than the original system. This ensures out of memory error is not observed while performing the backup and restore tasks.
The backup process does not include the SAN configuration. Ensure the new system has sufficient disk space before performing restore. For more information on the disk space needed, refer to the info.txt file available in the backup directory.
The restore system requires at least the same amount of memory and disk space as the backup system. Otherwise, the restore operation fails.

15.7.6 Configuring Audit Vault Server Backup

Learn how to configure the backup utility for Audit Vault Server.

Audit Vault Server must be configured before performing the backup operations. This configuration includes different settings and values that are saved in a backup settings file /var/lib/oracle/dbfw/av/backup/.backup_restore_config. This file is used to configure the backup process.

The backup of the OS files can be encrypted with a password that must be specified when configuring the Audit Vault Server backup. The backup of the repository is encrypted using TDE (Transparent Data Encryption). Make sure to store both the password (if specified) for backup of the OS files and the TDE keystore password for repository, in a safe location. These passwords are needed during the restore operation.

Run the following command as root user to configure the backup settings, and follow the prompt:

/var/lib/oracle/dbfw/bin/avbackup config

15.7.7 Performing Audit Vault Server Backup

Learn how to run Audit Vault Server backup tasks.

Prerequisite: Configure the backup utility for Audit Vault Server.

For online (hot) backup, follow these steps before initiating backup:

Choose the backup type HOT when configuring the backup utility.
Ensure that the Audit Vault Server is in archive log mode by following these steps:
1. Connect through SQL*Plus as the oracle OS user, by running the command:
```
sqlplus / as sysdba
```
2. Run the following command:
```
archive log list;
```
3. If the output from the above command displays No Archive mode, then Audit Vault Server is not in archive log mode. If the output displayed is Archive mode, then skip the next step.
To enable archive log mode follow these steps:
- Oracle AVDF 20.12 and later
- Oracle AVDF 20.1 - 20.11
Log in to the Audit Vault Server through SSH and switch to the root user.

See Logging In to Oracle AVDF Appliances Through SSH.

Execute the following:
/var/lib/oracle/dbfw/bin/avbackup enable_archinvelog

Enter Y to confirm you want to continue with the process which includes restarting the database.
The process may take several minutes to finish.
Log in to the Audit Vault Server through SSH and switch to the root user.

See Logging In to Oracle AVDF Appliances Through SSH.

Run the following command to stop the monitor process:

systemctl stop monitor

Run the following command to shut down the Audit Vault Server repository (Oracle Database):

systemctl stop dbfwdb

Run the following command to ensure that the Audit Vault Server repository is shut down:

/usr/local/dbfw/bin/dbfwdb status

The output is ORACLE instance shut down.

Switch to the oracle user.

su - oracle

Start SQL*Plus as sysdba.

sqlplus / as sysdba

Run the following commands at the SQL*Plus prompt to enable archive log mode:

startup mount

alter database archivelog;

alter database open;

shutdown immediate;

Exit SQL*Plus.

exit
Switch to root OS user.

Run the following commands:

systemctl start dbfwdb

systemctl start monitor

As the root OS user, run the following command to initiate the backup:
```
/var/lib/oracle/dbfw/bin/avbackup backup
```
Enter the required information by following the prompt.
A list of files (similar to the following example) appear in the backup location when the backup is complete.
```
DBID_1440353975_09Q7EF7L_1_1
DBID_1440353975_C-1440353975-20150520-00
```

Offline (Cold) Backup

Choose the backup type as COLD when configuring the backup utility.
As root OS user, run the following command to initiate backup:
```
/var/lib/oracle/dbfw/bin/avbackup backup
```
Enter the required information by following the prompt.

Note:

For an offline backup the Audit Vault Server repository is shutdown for the entire duration of the backup.
Oracle recommends to reboot the system in case there is a failure while performing a cold (offline) backup operation.

15.7.8 Monitoring and Validating the Audit Vault Server Backup

Learn how to validate, monitor, and troubleshoot the Audit Vault Server backup process.

The backup configuration file is release specific. It works on the same release. It is advisable to run the avbackup config command and create a new configuration file before performing the backup operation after Oracle AVDF upgrade.

Follow these steps to validate the backup last created:

Run the following command:

/var/lib/oracle/dbfw/bin/avbackup validate

The backup status is displayed, similar to the following:


Backup Restore exit status: 0
Status 0 = Success. Status 1 = Failure.

The backup process records the logs in the /var/lib/oracle/dbfw/av/log/av.backup*.log file. Check the log files for any errors. In case there is any issue in the backup process, then the backup process also records the logs in TMP_DIR/av_backup* directory. The following logs contain detailed information:
- /TMP_DIR/av_backup_<timestamp>/<logs>
- /var/lib/oracle/dbfw/av/log/av.backup_restore-pid-0.log
- /var/lib/oracle/dbfw/av/log/av.backup_restore_error-pid-0.log

Oracle AVDF administrator must monitor the disk space usage to make sure there is enough space for the new backup files and obsolete backup files that are purged properly based on the retention setting.

Note:

Important aspects involved in troubleshooting of Audit Vault Server backup process:

The backup directory must be owned by oracle:oinstall with permission 770.
Make sure to take more backups than REDUNDANCY setting within 30 days and obsolete backup files are purged properly.
Check /var/lib/oracle/dbfw/av/log/av.backup* log files for any errors. Check TMP_DIR/av_backup* for more detailed logs if there are any issues.
Check available disk space for backup.
The location of offline backup and online backup must be different. Do not use the same BACKUP_DIR location. Once this location is specified, it is advisable not to change the directory path until the next full backup.
Ensure the BACKUP_DIR and CHANNEL_LOCATION_x disk has enough space for the backup files.

15.7.9 Automating the Backup Schedule

Learn how to automate the backup schedule of Audit Vault Server.

The backup process can be scheduled and automated by setting up a cron job. For example, set up a cron job to take a full online backup once a week. Set up another cron job to take incremental online backup thrice a week.

The backup operation takes the configuration details from the /var/lib/oracle/dbfw/av/backup/.backup_restore_config file. Full backup and incremental backup require different configuration details, and hence need to have two separate configuration files.

Follow these steps to automate both full and incremental backups:

Create a settings file for full backup. Run the following command and specify INCREMENTAL value to 0:
```
avbackup config
```
Move the /var/lib/oracle/dbfw/av/backup/.backup_restore_config file to another location /var/lib/oracle/dbfw/av/backup/backup_restore_config_full.
Create a settings file for incremental backup. Run the following command and specify INCREMENTAL value to 1:
```
avbackup config
```
Move /var/lib/oracle/dbfw/av/backup/.backup_restore_config file to another location /var/lib/oracle/dbfw/av/backup/backup_restore_config_incremental.
Create a full backup script (full_backup_script), that copies the /var/lib/oracle/dbfw/av/backup/backup_restore_config_full to /var/lib/oracle/dbfw/av/backup/.backup_restore_config before running the avbackup backup command.
Set up a cron job to run full backup script as root user at a specific time on a specific day of the week. Follow these steps:
1. Run the following command as root user:
```
crontab -e
```
2. Add a line similar to the following example in the editor. The example time specified is for midnight on every Saturday.
```
0 0 * * 6 /<some_directory_path>/<full_backup_script>
```
3. Save the file and exit.
4. Check the cron job setup by running the following command:
```
crontab -l
```
Create incremental backup script (incremental_backup_script), that copies /var/lib/oracle/dbfw/av/backup/backup_restore_config_incremental file to /var/lib/oracle/dbfw/av/backup/.backup_restore_config before running the avbackup backup command.
Set up a cron job to run incremental backup script as root user at a specific time thrice a week on specific days. Follow these steps:
1. Run the following command as root user:
```
crontab -e
```
2. Add the lines in the editor similar to the following example for running backup at midnight every Monday, Wednesday, and Friday:
```
0 0 * * 1 /<some_directory_path>/<incremental_backup_script>
0 0 * * 3 /<some_directory_path>/<incremental_backup_script>
0 0 * * 5 /<some_directory_path>/<incremental_backup_script>
```
3. Save and exit.
4. Check the cron job setup by running the following command:
```
crontab -l
```
Note:
Use this as a guideline to automate scheduled backups. It is recommended to test out the full_backup_script, incremental_backup_script, and the cronjob setting before deploying in production. Change the cron job configuration as per your requirement and policy.

15.7.10 Performing Audit Vault Server Backup in High Availability

Learn how to run the Audit Vault Server backup task in a high availability environment.

In a high availability environment, there are two Audit Vault Servers (primary and standby). Backup operation must be performed on the primary Audit Vault Server.

Follow these steps:

Disable automatic failover before performing backup operation.
Run the backup operation.
Enable automatic failover.

See Also:
Disabling or Enabling Failover of the Audit Vault Server

15.7.11 Restoring from Audit Vault Server Backup

Learn how to restore Audit Vault Server from backup taken earlier.

In case of outage or contingency, the Audit Vault Server can be restored from backup taken earlier.

Important aspects involved in restoring of Audit Vault Server:

Aspects	Description
Planning and strategy	The restore operation can only be performed on the same version of Audit Vault Server. The new system must be a freshly installed system without any data. For example, restore of Oracle AVDF release 20.3 backup can be performed on a newly installed 20.3 Audit Vault Server, but not on a 20.4 Audit Vault Server. Choose to restore with the original IP address or restore to a new IP address. Restore using the original IP address requires the new system to be on the same subnet as the backup system. Audit Vault Server can be restored on a new system with a new IP address. The system on which the Audit Vault Server is being restored must have equal (or more) memory and disk space. Audit Vault Server cannot be restored on a system with less memory or disk space. After the restore operation is initiated, all the information in the restore system is wiped out and replaced by the information from the backup system. After restore operation, the Audit Vault Server contains data of the backup Audit Vault Server and until the time of the backup taken. Note: To perform restore on Audit Vault Server, the administrator must provide: the repository encryption password of the backup system the encryption password for backup if that is configured for backup
Space	Ensure the new system has sufficient disk space before performing the restore operation. For more information on the disk space needed, refer to the `info.txt` file available in the backup directory.
Memory	The new system on which the Audit Vault Server is being restored must have equal or more memory available than the backup system.
Storage	The backup files must be in the same backup location or path, as the backup system. In case the local file system is being used, then copy the backup files to the same path similar to the backup system. In case of NFS (Network File System), mount the backup location to the same path as of the backup system.
Configuration	To restore the new Audit Vault Server from backup file taken earlier, configure the backup utility on the new Audit Vault Server. The configuration of the restore system must match the configuration of the backup system. For example, if the backup files are for online backup, you must configure the restore system for online backup. A new IP address can be used if required on the restore system. If a new IP address is used to configure during restore, the new Audit Vault Server stays on the new IP address and does not switch to the original backup Audit Vault Server IP. In this case all the existing Audit Vault Agents and Database Firewall instances must be updated with the new IP address. In case the restore operation is performed with the existing IP address, then make sure the original IP address is available. After the completion of restore operation, the Audit Vault Server is set to the original IP if `NEW_IP` is configured to `N`.

Follow these steps to restore the Audit Vault Server:

Verify and make sure the backup files are owned by oracle:oinstall.
Ensure the OS user root on the restore system has access to BACKUP_DIR directory.
Copy the backup files to the new Audit Vault Server, or mount the NFS placing the files in the BACKUP_DIR directory that was specified earlier.
Log in to the Audit Vault Server as root.

Run the following command:

/var/lib/oracle/dbfw/bin/avbackup restore

When prompted, enter the keystore password. This password is the same keystore password used for the original system.
When restore operation is completed, check the following log files for errors:
- /TMP_DIR/av_backup_*_timestamp
- /var/lib/oracle/dbfw/av/log/av.backup_restore-pid-0.log
- /var/lib/oracle/dbfw/av/log/av.backup_restore_error-pid-0.log
Note:
- Do not initiate the restore process inside the backup directory.
- Ensure there is no terminal accessing the backup directory. If any terminal is accessing the backup directory, it results in device busy error during restore process.
- Audit Vault Server must be restored from the most recent backup to minimize data loss.

15.7.12 Post Restore Tasks

Perform these tasks after restoring the Audit Vault Server.

Update the Audit Vault Agents or the agentless collection service, depending on what you've deployed.
Update the Database Firewall instances.
Start audit data collection on the restored Audit Vault Server.
Configure secondary Network Interface Cards (NICs), if required.

Update Audit Vault Agents

Manually update Audit Vault Agents when restoring the Audit Vault Server with a new IP address so that they connect using the new IP address.

Follow these steps for Oracle AVDF release 20.4 and earlier:

Log in to the Agent machine.
In case of Audit Vault Agent, update the IP addresses in the Agent_Home/av/conf/bootstrap.prop file. In case of Host Monitor Agent, update the IP addresses in the Agent_Home/hm/bootstrap.prop file. Replace all the old IP addresses with the new IP addresses.
Restart the Audit Vault Agent. The restart downloads the new agent.jar file from the Audit Vault Server with the new IP address. Refer to Stopping, Starting, and Other Agent Operations for more information.

Note:

Perform this operation on all the Audit Vault Agents and restart them.

Follow these steps for Oracle AVDF release 20.5 and later:

Log in to the Agent machine.
Stop the Audit Vault Agent.

Run the following command on the Agent machine:

Platform Command

Windows

agentctl.bat update_agent_configuration -ip [new ip address of AVS] -port [new TCP port of AVS ]

Linux/Unix/AIX/Solaris

agentctl update_agent_configuration -ip [new ip address of AVS] -port [new TCP port of AVS ]

Note:

In case the Audit Vault Server is in high availability configuration, enter the new IP address and port number of the primary Audit Vault Server.
In case of multiple network interface cards on Audit Vault Server, enter the new IP address corresponding to the card which is reachable from the Agent machine.

Restart the Audit Vault Agent. The restart downloads the new agent.jar file from the Audit Vault Server with the new IP address. Refer to Stopping, Starting, and Other Agent Operations for more information.

Update the Agentless Collection Service

If you're using agentless collection (Oracle AVDF 20.9 and later), the agentless collection service will not run on the restored machine if the backup Audit Vault Server and the restored Audit Vault Server are two different machines with different IP address.

If the backup Audit Vault Server and the restored Audit Vault Server are two different machines with different IP address, run the following commands to stop the agentless collection service on the backup Audit Vault Server and deploy and start agentless collection on the restored Audit Vault Server.

Enter the following commands to stop the agentless collection service on the backup Audit Vault Server:

su root

/usr/bin/systemctl stop monitor_default_agent.service

/usr/bin/systemctl disable monitor_default_agent.service

/usr/bin/systemctl stop monitor_default_agent.timer

/usr/bin/systemctl disable monitor_default_agent.timer

/usr/bin/systemctl stop default_agent.service

/usr/bin/systemctl disable default_agent.service

Enter the following commands to deploy and start agentless collection on the restored Audit Vault Server:
```
su root
```
```
/usr/local/dbfw/bin/deploy_default_agent.py
```

Update Database Firewall Instances

After restoring the Audit Vault Server on a new IP address is completed, the Audit Vault Server console certificate is invalid. The certificate details are pertaining to the backup system which is no longer valid. A new certificate must be generated and uploaded.

Copy the existing certificate to the Database Firewall server to reconnect. Refer to Specifying the Audit Vault Server Certificate and IP Address for more information.

Start Audit Data Collection on Restored Audit Vault Server

The audit trails must be started on the newly restored Audit Vault Server. The trails start collecting audit records from the time stamp when the Audit Vault Server was backed up.

Note:

If audit trail cleanup is configured on the targets, then the audit data collected after the backup may be purged on the target. This data is not available for collection on a restored Audit Vault Server.
If audit trail cleanup is not configured on the targets, then the audit data collected after the backup is still available on the target. This data is available for collection on a restored Audit Vault Server.

Audit Vault Server Networking

The restore process is only restoring the management interface. All the secondary NICs and their associated network configuration has to be manually reconfigured and tested after restoring the Audit Vault Server.

Ensure the network interfaces are connected to the correct networks. Also ensure the new IP addresses and the network masks are correct. After restoring the Audit Vault Server appliance, ensure the following have been reconfigured correctly:

Secondary NIC IP configuration
Static routes
Agent connectivity for secondary NICs
Access for services (SSH, NTP, etc) previously configured for secondary NICs

See Also:

Multiple Network Interface Cards

15.7.13 Monitor the Restore Process

Learn about monitoring the Audit Vault Server restore process.

The restore process usually takes long time. The output can be monitored by checking the log files in the following locations:

/var/lib/oracle/dbfw/av/log/av.backup*
TMP_DIR/av.backup_<timestamp>/*

Note:

Since the restore operation takes a long time depending on the size of the backup, ensure the session used to run the command does not abruptly terminate. Oracle recommends to use commands like /usr/bin/screen to run restore commands.

Additional Information for Restoring Audit Vault Server

Here are some additional pointers for troubleshooting the Audit Vault Server restore process:

Ensure the access to BACKUP_DIR directory is owned by oracle:oinstall.
Ensure the OS user root on the restore system has access to BACKUP_DIR directory.
In case of incorrect password, run the avbackup config command again to set the password right. This is applicable to password used to encrypt configuration data and not the keystore password.
The script must be run as root user.
The backup directory path on the restore system must be the same as the backup system. In case of NFS, it must be mounted on the same path as the backup system.
In case device busy error is observed during restore, then contact Oracle Support for detailed steps for a workaround on the issue.

15.7.14 Restoring Audit Vault Server in High Availability

Learn how to restore Audit Vault Server in a high availability environment.

After restoring Audit Vault Server in a high availability environment, the system is a standalone. The restored system is not automatically configured for high availability. To set up high availability, pair the restored system with another new Audit Vault Server.

15.8 Backing Up and Restoring the Database Firewall

You can back up and restore the Database Firewall instance.

The Database Firewall configuration is backed up automatically when the Audit Vault Server backup is taken. This does not require any additional action.

Restoring the Database Firewall Configuration on an Existing Database Firewall Instance

A copy of the Database Firewall configuration is stored on the Audit Vault Server. If required, it can overwrite the existing settings of the Database Firewall and maintain a copy of the same on the Audit Vault Server.

To restore the Database Firewall configuration from the Audit Vault Server, follow these steps:

Ensure the Audit Vault Server's certificate is installed on the Database Firewall. See Specifying the Audit Vault Server Certificate and IP Address for more information.
Ensure the IP address of the Database Firewall remains the same.
Follow the steps mentioned in section Resetting Database Firewall.

Restoring the Database Firewall Configuration on a New Database Firewall Instance

In case the Database Firewall instance has failed, it is possible to restore the Database Firewall settings on a newly installed system.

To restore the Database Firewall configuration on a new system, follow these steps:

Install the Database Firewall. See Configuring Database Firewall for complete information.
Configure the same IP address which was used during the previous configuration.
Install the Audit Vault Server's certificate on the Database Firewall. See Specifying the Audit Vault Server Certificate and IP Address for more information.
Update the Audit Vault Server with the new Database Firewall's certificate by following the instructions mentioned in section Fetching an Updated Certificate from Database Firewall.
To restore the Database Firewall configuration, follow the steps mentioned in section Resetting Database Firewall.

15.9 Enabling Oracle Database In-Memory for the Audit Vault Server

After you enable Oracle Database In-Memory, you can monitor it.

15.9.1 About Enabling Oracle Database In-Memory for Oracle Audit Vault Server

You can enable Oracle Database In-Memory for Oracle Audit Vault Server.

You can improve the performance of Oracle Audit Vault and Database Firewall reports and dashboards by enabling Oracle Database In-Memory in Oracle Audit Vault Server. This feature lets you allocate a certain amount of system memory for audit data for a specified period of time. The audit data residing in-memory then becomes available more quickly for use in dashboards and reports.

Based on the amount of system memory you allocate for Oracle Database In-Memory, and the average amount of data collected per day in your environment, Oracle Audit Vault and Database Firewall calculates the number of days of audit data that will fit into that allocated memory. From this calculation, the system displays the in-memory date range to Oracle Audit Vault and Database Firewall auditors, letting them know the time ranges for which they can obtain faster reports. For example, if 1 gigabyte can accommodate 2 days of data, and you have provided 1 gigabyte of memory for Oracle Database In-Memory, then 2 days of the latest data will be put in Oracle Database In-Memory. If you provide 2 gigabytes of memory to Oracle Database In-Memory, then 4 days of data will go to Oracle Database In-Memory.

Before enabling Oracle Database In-Memory, be sure to estimate the amount of memory needed for your current and future targets and Database Firewall monitoring points. You can find some guidelines for calculating RAM requirements in the Oracle Audit Vault and Database Firewall Sizing Advice (My Oracle Support Doc ID 2092683.1). This document can be obtained from Oracle Support. After estimating your normal RAM requirements, if you want to use the Oracle Database In-Memory feature, estimate how much RAM you want to use for in-memory database and add that to your RAM requirement. If you enable this feature, you must allocate at least 1 GB for Oracle Database In-Memory.

15.9.2 Enabling and Allocating Memory for Oracle Database In-Memory

You can enable and allocate memory for Oracle Database In-Memory from the Audit Vault Server console.

Log in to the Audit Vault Server console as a super administrator.
Click the Settings tab.
In the left navigation menu, select select System.
Under Configuration, select Oracle Database In-Memory.
In the Oracle Database In-Memory dialog, select the Enable Oracle Database In-Memory check box.
The Oracle Database In-Memory window expands to show the total system RAM.
If you have sufficient memory on your system, configure the following settings:
- Allocated In-Memory field: Enter (or change) the amount of RAM to allocate in Gigabytes. You must enter a minimum of 1 (default), and up to Maximum available for Database In-Memory indicated on this dialog.
- Keep latest data option: Select this option to retain the data that has just been collected. This setting enables the system to automatically select the most recent dates, based on the in-memory size that was configured.
- Select a date range option: Select this option if you want the memory to be available for a specific period of time.
Click Save.

After you enable or disable Oracle Database In-Memory, the Audit Vault Server database, Audit Vault Agents, and audit trails shut down for a few minutes, and then restart automatically.

15.9.3 Disabling Oracle Database In-Memory

You can disable Oracle Database In-Memory from the Audit Vault Server console.

Log in to the Audit Vault Server console as a super administrator.
Click the Settings tab.
In the left navigation menu, select System.
Under Configuration section, select Oracle Database In-Memory.
In the Oracle Database In-Memory window, select the Enable Oracle Database In-Memory check box to clear it. If enabled, the disable option is available in the dialog.
Click Save.

After you enable or disable Oracle Database In-Memory, the Audit Vault Server database, Audit Vault Agents, and audit trails go down for a few minutes, and then restart automatically.

15.9.4 Monitoring Oracle Database In-Memory Usage

You can monitor the Oracle Database In-Memory usage from the Audit Vault Server console.

To see in-memory usage in the Audit Vault Server dashboard:

Click the Settings tab.
In the left navigation menu, select select System.
Under Configuration, select Oracle Database In-Memory.
The Oracle Database In-Memory dialog contains all the details.

15.10 Managing Plug-ins

You can use plug-ins to deploy additional target types in Oracle Audit Vault and Database Firewall environments.

You can deploy additional plug-ins to support more types of targets, or un-deploy plug-ins that are no longer needed.

15.11 Monitoring and Adding Server Tablespace Space Usage

You can monitor and add server table space usage in Oracle Audit Vault Server.

Oracle Audit Vault Server contains the following tablespaces:

SYSTEM
SYSAUX
TEMP
USERS
UNDOTBS1

By default, these tablespaces have one datafile.The tablespaces are locally managed with automatic segment space management.

You should monitor the space usage for the tablespace and create additional data files for storage as needed.

Starting with Oracle AVDF 20.9 the Audit Vault Server console will monitor the space remaining in the SYSTEM, SYSAUX, and TEMP tablespaces. You will receive system alerts when the remaining space is low and additional datafiles need to be added.

To add a datafile to any of the above tablespaces:

Log in to the Audit Vault Server through SSH and switch to the root user.

See Logging In to Oracle AVDF Appliances Through SSH.
Switch to the oracle user.
```
su - oracle
```
Connect as a super administrator user.
```
sqlplus superadmin/superadmin_password
```
Run the following to add a 100MB datafile to the provided tablespace.
```
execute avsys.datafile_management.add_datafile(tablespace_name);
```

Related Topics

See Also:

System Alerts for more information on how to receive notifications when tablespace is low in the Audit Vault Server.
Creating Data Files and Adding Data Files to a Tablespace in the Oracle Database Administrator's Guide for more information about the ALTER TABLESPACE SQL statement, which you can use to add more storage data files.
Altering a SQL Profile in the Oracle Database SQL Tuning Guide for more information about optimizing a tablespace.

15.12 Monitoring Server Archive Log Disk Space Use

You can monitor archive log disk space use to manage your system.

By default, ARCHIVELOG mode is disabled in Oracle Audit Vault Server. The ARCHIVELOG mode once enabled, copies the filled online redo logs to disk. This enables you to back up the database while it is open and being accessed by users, and to recover the database to any desired point in time. You should monitor the disk space usage for the redo logs.

To change from No Archive Mode to Archive Mode, follow these steps:

Log in as oracle user.
Connect to SQL*Plus as sysdba user.
Run the command:
```
SQL> archive log list;
```
Observe the following output:

Database log mode No Archive Mode

Automatic archival Disabled

Archive destination USE_DB_RECOVERY_FILE_DEST
In case the Database log mode is No Archive Mode, then run the following commands:
```
SQL> shutdown immediate;
```
```
SQL> startup mount;
```
```
SQL> alter database archivelog;
```
```
SQL> archive log list;
```
Observe the following output:

Database log mode Archive Mode

Automatic archival Enabled

Archive destination USE_DB_RECOVERY_FILE_DEST
Run the command:
```
SQL> alter database open;
```

Note:

If you change the ARCHIVELOG mode during the backup configuration process, after the database restarts, then ensure the Java Framework internal tool is running on the Audit Vault Server.
Archivelog mode is required for hot backup.

See Also:

Oracle Database Administrator’s Guide for more information to set up archive log mode and other general information about Archive logs.
Method 1: Using the LOG_ARCHIVE_DEST_n Parameter for more information about changing the LOG_ARCHIVE_DEST_n location to relocate these archive log files to larger disks.
Oracle Database Backup and Recovery User’s Guide for information about backing up the archive logs.

15.13 Monitoring Server Flash Recovery Area

Monitoring server flash recovery area is advisable to ensure you have enough space for backups.

By default, Oracle Audit Vault Server has the following initialization parameter settings:

The DB_RECOVERY_FILE_DEST_SIZE initialization parameter is set to 2 GB.
The DB_RECOVERY_FILE_DEST initialization parameter is set to the default flash recovery area, typically the ORACLE_HOME/flash_recovery_area directory.

Ensure that the size of your flash recovery area is large enough to hold a copy of all data files, all incremental backups, online redo logs, archived redo logs not yet backed up on tape, control files, and control file auto backups. This space can fill quickly, depending on the number of audit trails configured, the scope of the audit record collection being administered, and the backup and archive plans that you have in place.

You can use Oracle Enterprise Manager Database Control to monitor the available space in the flash recovery area. Monitor the percent space that is usable in the Usable Flash Recovery Area field under the High Availability section on the Home page. Check the alert log in the Database Console for messages. When the used space in the flash recovery area reaches 85 percent, a warning message is sent to the alert log. When the used space in the flash recovery area reaches 97 percent, a critical warning message is sent to the alert log.

You can manage space in the flash recovery area by increasing the value of the DB_RECOVERY_FILE_DEST_SIZE initialization parameter to accommodate these files and to set the DB_RECOVERY_FILE_DEST initialization parameter to a value where more disk space is available.

See Also:

15.14 Monitoring Jobs

You can see the status of various jobs that run on the Audit Vault Server, such as report generation, and user entitlement or audit policy retrieval from targets.

Log in to the Audit Vault Server as an administrator.
Click the Settings tab.
In the left navigation menu, select System.
In the Status page, under Monitoring section, click Jobs.

The Jobs window appears, listing all the jobs that have been configured. It shows the job type, current status (such as Starting), when last updated, when started, who created the job, and any messages that may result from the job.

Description of the illustration jobs.png
To see details for an individual job, click the Job Details icon to the extreme left of a specific job.

15.15 Schedule Maintenance Jobs

Oracle Audit Vault and Database Firewall (Oracle AVDF) runs some jobs on the Audit Vault Server for proper and effective functioning of the system.

Oracle recommends that you run these jobs during a period when the Audit Vault Server usage is low, such as at night. You can schedule these jobs based on your time zone.

Log in to the Audit Vault Server console as an administrator.
Click the Settings tab.
Click System in the left navigation menu.

In the Configuration section, click one of the following links, depending on your release:

Oracle AVDF Release	Link
20.1 and 20.2	Manage
20.3 and later	Maintenance

To schedule a new maintenance job, enter the start time in hours and minutes.
The time that you specify here is the time on the browser.
In the Time Out (In hours) field, enter the duration of the maintenance job in hours.

If the job doesn't complete in the specified duration, it times out.

Note:
The job runs at the specified start time daily. You can't change the repeat frequency.
Click Save.

15.16 Downloading and Using the AVCLI Command Line Interface

You can download the AVCLI command line interface from the Audit Vault Server console.

15.16.1 About the AVCLI Command-Line Interface

Learn about the AVCLI command-line interface.

As an alternative to using the Oracle Audit Vault Server console (Web) UI, you can use the AVCLI command line interface to manage Oracle Audit Vault and Database Firewall, including registering and configuring targets and their connections to the Audit Vault Server.

You can run AVCLI from the Audit Vault Server, or download the AVCLI utility from the Audit Vault Server and install and run the utility on another computer.

The syntax used for AVCLI is similar to SQL*Plus. For example, from within AVCLI, you can use the CONNECT command to log in as another user. In addition, the AVCLI commands are not case sensitive. In this manual, the commands are entered in upper case.

Note:

Set the JAVA_HOME environment variable to point to JDK installation directory. On Windows, add %JAVA_HOME%\bin to the PATH environment variable.

See Also:

AVCLI Commands Reference for details of the available AVCLI commands.

15.16.2 Downloading the AVCLI Command Line Utility and Setting JAVA_HOME

The AVCLI utility is already installed on the Audit Vault Server. If you want to run AVCLI on a different computer, then you must download it from the Audit Vault Server console and install it on the other computer.

Log in to the Audit Vault Server console as an administrator.
Click the Settings tab.
In the left navigation menu, select Audit Vault CLI.
Click the Download AVCLI button. Save the avcli.jar file.
Copy the avcli.jar file to the computer from which you want to run AVCLI. Run the command:

java -jar avcli.jar

The AVCLI utility is installed in the current directory with the necessary permissions. To install in a different directory, use the command:

java -jar avcli.jar -d directory_name
Set the JAVA_HOME environment variable to point to the JDK installation directory. On Windows, add %JAVA_HOME%\bin to the PATH environment variable.

15.16.3 Logging in to AVCLI

You can log in to the Audit Vault command line interface by using different methods.

15.16.3.1 About Logging in to AVCLI

You can log in to AVCLI interactively with or without a user name, and with stored credentials.

Before users log in, ensure that the JAVA_HOME environment variable in the server points to JDK installation directory. The user who logs in to AVCLI must be granted the granted the AV_ADMIN role, which you can grant by using the Audit Vault Server console.

The ways to log in are as follows:

By supplying a user name and password by executing the avcli command
Without supplying a user name and password, but you will be prompted for these credentials after you execute avcli
By using a stored credential, which is useful for situations in which you must run scripts.

15.16.3.2 Logging in to AVCLI Interactively

You can start AVCLI interactively at the command line with or without a user name.

Except for a few commands where it is optional, all AVCLI commands must end in a semi-colon (;). For simplicity, in this guide we use a semi-colon for all AVCLI commands.

Log in to the server where AVCLI is installed as a user who has been granted the AV_ADMIN role.
Go to the directory where AVCLI has been installed, and open /bin.
cd ../directory_name/bin/

At the command line, use one of the following methods to log in to AVCLI:

Logging in with a user name: Use the following syntax:

avcli -u username
Enter password: password

For example:

avcli -u psmith
AVCLI : Release 20.1.0.0.0 - Production on timestamp
Copyright (c) 1996, 2020 Oracle.  All Rights Reserved.
Enter password for 'psmith': password

Connected to:
Oracle Audit Vault Server 20.1.0.0.0

AVCLI>

Logging in without a user name: Use the following syntax:
```
avcli
AVCLI> CONNECT [username];
```
For example:
```
avcli

AVCLI : Release 20.1.0.0.0 - Production on timestamp
Copyright (c) 1996, 2020 Oracle.  All Rights Reserved.

AVCLI> CONNECT psmith
Enter password: password;
Connected.
```
If you do not enter a user name, then you will be prompted for one.

15.16.3.3 Storing or Overwriting Administrative Credentials

If you are the AVCLI owner (that is, you installed the AVCLI utility) you can store the credentials of one Oracle AVDF administrator in the AVCLI wallet.

Thereafter, that administrator can invoke AVCLI without providing credentials, and can also run scripts without intervention.

As a prerequisite for an administrator to be able to invoke AVCLI without credentials (non-interactively), the AVCLI owner must store that administrator's credentials. As the AVCLI owner, you can store credentials for only one administrator.

As the AVCLI owner, run avcli without connecting to the Audit Vault Server.

For example:

avcli

AVCLI : Release Release 20.1.0.0.0 - Production on timestamp
Copyright (c) 1996, 2020 Oracle.  All Rights Reserved.

AVCLI>

Run the command STORE CREDENTIALS and provide the administrator's credentials when prompted.
For example:
```
AVCLI> STORE CREDENTIALS;
Enter user name: username
Enter password:password
Re-enter password:password
```
Any previously stored credentials will be overwritten. If this administrator's password changes, follow this procedure again to store the new credentials.

15.16.3.4 Logging in to AVCLI Using Stored Credentials

To start AVCLI without having to enter credentials, your credentials must be stored in the Audit Vault Server.

Log in to the server where AVCLI is installed as a user who has been granted the AV_ADMIN role.
Use one of the following methods to log in to AVCLI using stored credentials:
- From the shell: In the Audit Vault Server console, enter the following command, which logs you in to AVCLI and connects to the Audit Vault Server:
```
avcli /@
```
- From within AVCLI: If you have invoked AVCLI from the shell without credentials (by typing avcli), connect to the Audit Vault Server by entering:
```
AVCLI> CONNECT /@;
```
  For example:
```
avcli

AVCLI : Release 20.1.0.0.0 - Production on timestamp
Copyright (c) 1996, 2020 Oracle.  All Rights Reserved.

AVCLI> CONNECT /@;
Connected.
```

Related Topics

Running AVCLI Scripts

15.16.4 Running AVCLI Scripts

You can run AVCLI scripts without user intervention or putting credentials inside the script.

An AVCLI script contains a series of AVCLI commands. You can run an AVCLI script from the shell. Valid AVCLI script names have a .av extension.

Here is an example AVCLI script:

#Here is an AVCLI command
start collection for secured target sample_target1 using host sample_host1 from        table SYS.AUD$;
#More AVCLI commands
#Quit command
quit;

Log in to the server where AVCLI is installed as a user who has been granted the AV_ADMIN role.
Use the following syntax to run the script:
```
avcli -u username -f scriptname.av
```
For example:
```
avcli -u psmith -f myscript.av
AVCLI : Release 20.1.0.0.0 - Production on timestamp
Copyright (c) 1996, 2020 Oracle.  All Rights Reserved.
Enter password for 'psmith': password

Connected to:
Oracle Audit Vault Server 20.1.0.0.0

AVCLI> the script myscript.av executes
```
If you have stored administrator credentials, to run an AVCLI script, use the appropriate command below:
- avcli /@ -f sample_script1.av
  
  This command uses the stored credentials, connects to the Audit Vault Server, and runs the script.
- avcli -f sample_script2.av
  
  You can use the above command if you include the following command at the beginning of your script:
  
  connect /@
  
  Then the script runs using the stored credentials, and connecting to the Audit Vault Server.

Related Topics

Logging in to AVCLI Using Stored Credentials

15.16.5 Specifying Log Levels for AVCLI

When you run AVCLI, you can specify log levels to capture different categories of information or errors.

Oracle Audit Vault and Database Firewall writes the logs to the Audit Vault Server $ORACLE_HOME/av/log directory.

info: Logs informational and error messages
warning: Logs both warning and error messages
error: Logs only error messages (default)
debug: Logs debug, error, warning, and informational messages

To specify a log level, enter the L option. For example, to invoke AVCLI as user psmith with the log level set to warning:

avcli -l warning -u psmith
AVCLI : Release 20.1.0.0.0 - Production on timestamp
Copyright (c) 1996, 2020 Oracle.  All Rights Reserved.
Enter password for 'psmith': password

Connected to:
Oracle Audit Vault Server 20.1.0.0.0

AVCLI>

To invoke AVCLI using a script and with the debug warning level:

avcli -l debug -f myscript.av

AVCLI : Release 20.1.0.0.0 - Production on timestamp
Copyright (c) 1996, 2020 Oracle.  All Rights Reserved.

AVCLI> Connected.

AVCLI> the script myscript.av executes

Note: You must be connected as a valid user who has been granted the AV_ADMIN role. You can do so using the CONNECT username/password directive.

15.16.6 Displaying Help and the Version Number of AVCLI

You can display help information for various AVCLI commands and find the AVCLI version number from the command line.

To display the AVCLI help information and version number:

avcli -h

If you only want to find the version number, then use the V argument:

avcli -v

15.17 Downloading the Oracle Audit Vault and Database Firewall SDK

An SDK is available for developing custom Oracle Audit Vault and Database Firewall plug-ins.

Log in to the Audit Vault Server console as an administrator.
Click the Settings tab.
In the left navigation menu, click System.
In the Status page that appears, under Monitoring, click Plug-ins.
In the Plug-ins window, do not select any of the plug-ins.
Click Download SDK.
Select Save File and then specify a location.

Related Topics

Oracle Audit Vault and Database Firewall Developer's Guide

15.18 Managing Database Firewalls

Management tasks for Database Firewalls include tasks such as changing the network or services configuration.

15.18.1 Changing the Database Firewall Network or Services Configuration

Learn how to change the Database Firewall network or services configuration.

See one of the topics below if you need to change a Database Firewall's network, traffic sources, or services configuration:

15.18.2 Viewing Network Traffic for a Database Firewall

You can capture and view network traffic in a .pcap file that you can download and analyze for debugging.

Log in to the Audit Vault Server console as an administrator.
Click the Database Firewalls tab.
In the left navigation menu, click Database Firewalls.
Click the link for the Database Firewall instance for which you want to capture network traffic.
Under Diagnostics, click Network Traffic Capture.
In the Network Traffic Capture dialog box, select the network traffic source in the Network Interface field.
For Duration (min), set the number of minutes for which you want to capture traffic.
Click the Capture button.

After the specified duration, a message appears saying that the network files were successfully captured and the captured traffic file appears in the table.

Note:
The maximum file size of the captured network traffic is 1 MB. As soon as the file reaches that size, traffic capture stops, regardless of the specified duration. To capture traffic for longer durations, you can use a network protocol analyzer like Wireshark. For more details, see My Oracle Support Doc ID 2085200.1 and Doc ID 1141588.1.
Select the network traffic file, and click Download.
Specify the location and download the traffic file in .pcap format.

15.18.3 Restarting or Powering Off Database Firewall

Use this procedure to restart or power off Database Firewall.

To restart or power off a Database Firewall:

Log in to the Audit Vault Server as an administrator.
Click the Database Firewalls tab.
Select the specific Database Firewall you want to reboot or power off.
Click the Reboot or Power Off button.

See Also:

Using Audit Vault Server Console

15.18.4 Removing Database Firewall from Audit Vault Server

You can remove Database Firewall from Audit Vault Server.

To remove Database Firewall from Audit Vault Server:

Log in to the Audit Vault Server as an administrator.
Click the Database Firewalls tab.
Select the specific Database Firewall you want to remove.
Click the Delete button.

See Also:

Using Audit Vault Server Console

15.18.5 Fetching an Updated Certificate from Database Firewall

Learn how to obtain updated certificates from Database Firewall.

You can update the Database Firewall certificate stored in the Audit Vault Server using the Audit Vault Server console. You must update this certificate when you upgrade the Database Firewall to maintain communication between the Database Firewall and the Audit Vault Server.

To update the Database Firewall certificate stored in the Audit Vault Server:

After upgrading the Database Firewall, log in to the Audit Vault Server console as an administrator.
Select the Database Firewalls tab.
In the left navigation menu, select Database Firewalls tab.
Select the specific Database Firewall instance from the list.
If the Database Firewall instance is down due to certificate validation error, then the Update Certificate button appears on the page. Click this button to update the certificate.

See Also:

Using Audit Vault Server Console

15.18.6 Viewing Diagnostics for Database Firewall

See Also:

Viewing the Status and Diagnostics Report for Database Firewall for viewing Database Firewall diagnostics.

15.18.7 Resetting Database Firewall

Learn how to reset the Database Firewall instance.

This block contains information about the Database Firewall settings and the details of resetting a Database Firewall instance. The Reset Firewall button is available in the page that contains details of the specific Database Firewall instance. It performs a reset of the Firewall ID. The Firewall ID is a unique identification number of the Database Firewall. It is derived from the Management Interface card.

Once the reset is performed, it removes the existing monitoring point instances and creates new ones using the configuration information stored in Audit Vault Server. The monitoring point instances not listed on the Audit Vault Server are removed once the reset is performed. The captured data which is not processed is also deleted. The network setting (Management Interface) of the Database Firewall is not altered. This operation restores the network interface card settings other than the Management Interface. It also restores the proxy ports information that was stored in the Audit Vault Server.

Note:

Whenever the Network Interface Card is replaced, the Database Firewall ID must be reset.
The network setting (Management Interface) of the Database Firewall is not altered. Ensure the Database Firewall network is configured appropriately before attempting to reset Firewall ID.

The user must reset the Firewall ID in the following scenarios:

After replacing the Management Interface card on the Database Firewall.
After replacing an existing and configured Database Firewall instance with a newly installed Database Firewall instance.

15.18.8 Restoring Database Firewall Monitoring Points

Learn how to restore Database Firewall monitoring points.

When you restore the Audit Vault Server from a backup, you must restore the status of the Database Firewall monitoring points that are registered with the Database Firewall.

See Also:

Resetting Database Firewall for more information.

15.19 System Alerts

System Alerts allow administrators to be notified about important system states and possible issues, including, the status of standby servers for high availability, storage availability, certificate expiration, and password expiration through the admin dashboard.

15.19.1 About System Alerts

System alerts allow administrators to be notified of important system statuses and possible issues through the Oracle Audit Vault Server console in Oracle Audit Vault and Database Firewall (Oracle AVDF) 20.9 and later.

System Alerts provide Oracle AVDF administrator users with proactive information necessary for maintaining important Oracle AVDF components. They can help you identify system failures before they happen and improve the stability and reliability of your Oracle AVDF system. The status of different components and processes are checked every six hours. If there are any issues a system alert will be generated for one of the following:

The status of high availability of Audit Vault Server.
The utilization of the Audit Vault Server Fast Recovery Area in a high availability environment.
The Apply Lag on Audit Vault Server. Apply lag is the degree to which the data in Standby Server lags behind the data in the Primary Server.
The storage availability in the Audit Vault Server file systems, directories, tablespaces, and disk groups.
The expiration of Audit Vault Server, Audit Vault Agent, and Database Firewall certificates.
The expiration of Audit Vault Server passwords for administrators, auditors, and operating system users (support and root).

To learn more about the specific system alerts and the recommended resolution, see System Alerts and Recommendations.

15.19.2 Configuring or Modifying System Alert Email Notifications

Starting in Oracle AVDF 20.10, you can configure email notifications for system alerts. This allows you to receive an email whenever a critical or high severity system alert occurs. The email you receive will contain the system alert category and severity in the email subject. The system will check if any email notifications need to be sent out every six hours.

Configuring or Modifying System Alert Email Notifications

Prerequisites:

The connection between your email and Oracle AVDF needs to be configured. See Configuring the Email Notification Service for more information.

To configure or modify email notifications for system alerts:

Log in to the Audit Vault Server Console as a super administrator.
Click the Settings tab.
Click System in the left navigation menu.
In the System Alerts section,
- If configuring for the first time, click Click here.
- If modifying, click Email Notification.
A dialog box will appear.
In the To field, enter the email(s) you would like to receive system alert notifications. Each email address can be comma or semicolon separated.
Optionally, in the Subject field, enter the Email Subject you want to receive for system alerts. By default, "AVDF System Alerts" will be used as part of the subject. In addition, the email subject will contain the system alert category and severity.
Click Save.

Example 15-1 Email Notification for One Alert

If the Subject field was left blank and there is one system alert with high severity under Storage category, the email subject will be "AVDF System Alerts: Storage - High".

Example 15-2 Email Notification for Multiple Alerts

If the Subject was set to “Oracle AVDF System Alerts”, and there are multiple system alerts in the system such as:

A critical severity alert in the High Availability category
A critical severity alert in the Password category
A high severity alert in the Storage category

The email subject will be “Oracle AVDF System Alerts: High Availability - Critical, Password - Critical, Storage - High”.

Adjusting the Frequency of the Background Job For System Alert Email Notifications

By default, email notifications are sent out every six hours. This frequency can be adjusting by performing the following steps:

Unlock the avsys user.

See Unlocking the AVSYS User.

Note:
Remember to relock the avsys account when you've completed this task.

Run the following command on SQL*Plus as the avsys user:

exec dbms_scheduler.set_attribute('avsys.avs_email_notification_job','repeat_interval','FREQ=<YEARLY
      | MONTHLY | WEEKLY | DAILY | HOURLY | MINUTELY | SECONDLY>;INTERVAL=<1-99>');

Lock the avsys user.

See Locking the AVSYS User.

Example 15-3 Adjust the Email Notification Schedule to Daily

exec dbms_scheduler.set_attribute('avsys.avs_email_notification_job','repeat_interval','FREQ=DAILY;INTERVAL=1');

Example 15-4 Adjust the Email Notification Schedule to Every 30 Minutes

exec dbms_scheduler.set_attribute('avsys.avs_email_notification_job','repeat_interval','FREQ=MINUTELY;INTERVAL=30');

15.19.3 Viewing System Alerts

System Alerts can be viewed from the admin dashboard, directly from the System tab in the Audit Vault Server console, or in the syslog.

Viewing alerts from the dashboard:

Log in to the Audit Vault Server Console as an administrator.
Click on the system alerts chart from the admin dashboard. This will bring you to the System page.
View the system alerts in the System Alerts section at the bottom of the page.
Click on a system alert to view the history of that alert. A pop-up will show a list of alerts based on the selected alert. The alert history will show a maximum history of three months.

Viewing alerts from the System tab:

Log in to the Audit Vault Server Console as an administrator.
Click the Settings tab.
Click System in the left navigation menu.
View the system alerts in the System Alerts section at the bottom of the page.
Click on a system alert to view the history of that alert. A pop-up will show a list of alerts based on the selected alert. The alert history will show a maximum history of three months.

Viewing alerts from the syslog.

Log in to the Audit Vault Server through SSH and switch to the root user.

See Logging In to Oracle AVDF Appliances Through SSH.
View system alerts in the syslog (/var/log/messages) by filtering for the AVDF SYSTEM ALERT tag.

For descriptions of the severity levels, see System Alerts Severity Levels.

For a list of possible alerts and recommendations, see System Alerts and Recommendations.

15.19.4 Closing System Alerts

Once an error condition is fixed, a super administrator user can close a system alert.

Log in to the Audit Vault Server Console as a super administrator.
Click the Settings tab.
Click System in the left navigation menu.
In the System Alerts section, select one or more system alerts that you'd like to close.
Click Close System Alerts.

Once an alert is closed, the same alert will not be displayed within 24 hours, even if the error condition is met again. After 24 hours a new system alert will be generated if the error condition is met.

For descriptions of the severity levels, see System Alerts Severity Levels.

For a list of possible alerts and recommendations, see System Alerts and Recommendations.

15.19.5 System Alerts Severity Levels

System alerts include the following severity levels:

Critical: This severity means that some functionality of the system is not working or will stop working soon. For example, in case of high availability, Fast Recovery Area is more than 80% full or agent certificate is going to expire in five days etc.
High: This severity means that some functionality of the system will stop working in some time. For example, in case of high availability Fast Recovery Area is more than 70% full but less than 80% or agent certificate is going to expire in six weeks.
Medium: This severity means that some functionality of the system may stop in the future or is not performing as expected.
Low: This severity is for information that needs some user attention, but there is no functionality failure expected in near future.