1 What is Oracle Audit Vault and Database Firewall

Learn about Oracle Audit Vault and Database Firewall software (Oracle AVDF), and about collection plug-ins.

1.1 Overview of Oracle Audit Vault and Database Firewall

Learn about Oracle Audit Vault and Database Firewall components, and what each component does.

Oracle Audit Vault and Database Firewall (Oracle AVDF) supports native database audit data collection and network-based SQL monitoring to deliver a comprehensive Database Activity Monitoring solution. It is comprised of these components:

• Audit Vault Server: A server that contains an embedded Oracle Database and other software components that manage the activities of Oracle Audit Vault and Database Firewall.
• Audit Vault Agent: A Java component that runs on a remote host and manages the collection of audit information based on commands from the Audit Vault server. The agent interfaces with the collection plug-ins under its control to gather audit records and sends it to the Audit Vault Server.
• Database Firewall: The Database Firewall is a dedicated server that runs the Database Firewall software. Each Database Firewall monitors SQL traffic on the network from database clients to target databases. The Database Firewall then sends the SQL data to the Audit Vault Server to be analyzed in reports.

Oracle Audit Vault and Database Firewall ships with several prepackaged collection plug-ins, which are software programs that know how to access and interpret audit data from target systems of various types. Collection plug-ins collect audit data from an audit trail generated by a target system and store it in an Audit Vault Server repository. Each collection plug-in is specific to a particular type of trail from a particular type of target. These collection plug-ins collect data from databases such as Oracle, SQL Server, Sybase ASE, and DB2.

See Also:

Oracle Audit Vault and Database Firewall Administrator's Guide

1.2 How Oracle Audit Vault Server and Agent Work

Audit Collection Plug-ins retrieve audit data in the form of audit trails, which are sequences of audit records.

Audit Collection Plug-ins retrieve audit data in the form of audit trails, which are sequences of audit records. Audit trails are generated by different target types and stored in database tables or XML audit records.

A target can write one or more audit trails; each audit trail is stored in a separate location, and can have its own format.

To elaborate a little on these terms:

• Target

  A target is a software component which generates an audit trail. A target is an instance of a target type and has specific properties such as connection credentials and trail types.

• Target Type

  A target type represents a collection of a particular type of target that generates the same type of audit data. Oracle Database, for example, is a target type which can have many instances. However, all Oracle Databases generate the same audit data and record the same fields.

• Audit Trail

  An Audit Trail identifies a location and format where audit data resides. Each audit trail is generated by one and only one target. Examples of audit trails are:

  • For targets that write data into files, the trail is the directory path plus the file mask.

  • For targets that write audit data into a database table, the name of the table is the trail for that target. Unified_Audit_Trail is an example of a database table audit trail in an Oracle database.

1.3 Types of Audit Collection Plug-Ins

Learn what audit collection plug-ins are, which audit collection plug-ins you should use for your audit trails, and what Java-based collection plug-ins you can use with Oracle Database Vault.

1.3.1 What Are Audit Collection Plug-ins?

Learn about audit collection plug-ins.

Audit collection is supported from many database types. See Product Compatibility Matrix for a list of supported database types and versions.

In case audit collection is not supported out of the box from a specific database type, then you can build custom audit collection plug-in to retrieve audit data stored in the audit trails.

A collection plug-in provides functionality similar to the prepackaged collection plug-ins shipped with Oracle Audit Vault and Database Firewall, by retrieving audit data stored in audit trails.

Oracle Audit Vault and Database Firewall allows developers and third-party vendors to build custom collection plug-ins. These custom plug-ins are capable of collecting audit data from a new target type.

You can write collection plug-ins that collect audit trails stored in database tables and XML files, or that are accessible in any other way.

You can support targets, such as relational databases, operating systems, mid-tier systems, or enterprise applications.

To obtain more individualized audit data, you can create custom collection plug-ins, and deploy them into existing Oracle Audit Vault and Database Firewall installations.

1.3.2 About Oracle AVDF Plug-In Types

You can create two types of collection plug-ins for Oracle AVDF. The actual type that you need to create depends on the properties of the audit trail that you want to collect.

To describe the audit data being collected, you create an XML file, called a mapper file, for the collection plug-in to use. Oracle Audit Vault Server uses this file to access and interpret the audit records being collected. You do not need to write code for this type of plug-in.

There is also a Java-based type of collection plug-in, which uses a Java API. You can design your own Java-based collection plug-in, or you can use one that is prepackaged with Oracle Audit Vault and Database Firewall.

1.3.3 Determining Which Audit Collection Plug-in Type to Create

The audit collection plug-in that you use depends on the type of audit trail that you are collecting for Oracle Audit Vault and Database Firewall.

You can easily define a mapper file (template) and a collection plug-in if the audit trails you wish to collect are stored in either of the following:

• Database Tables: Stored in database tables that conform to specific constraints

• XML/JSON/CSV Files: Stored in XML/JSON/CSV files based on the Oracle AVDF XML Audit File format

• REST: REST data source that generates data in JSON format.

1.3.4 Java-Based Collection Plug-ins

When the audit trail you need to collect is not in a format that a Collection plug-in can easily read, you write Java-based collection plug-ins in Java code.

Using the Java API provided, you can write code to collect these more complex audit trails and send them to the Audit Vault Server repository.

1.4 Audit Vault Server Events and their Attributes

Oracle AVDF monitors the stream of events that occur in target systems.

1.4.1 About Audit Vault Server Events and Attributes

Learn about Audit Vault Server events, fields, and audit records.

Monitoring the activity, the stream of events, that occur in a target system is the essence of Oracle Audit Vault and Database Firewall. These events are described by fields. A collection of fields describing a single event that occurred on the target system is an audit record.

The following applies for Oracle Audit Vault and Database Firewall:

• Each target logs events as audit events that occur on that target. Audit records capture information about audit events.

• Audit records typically have a target type event name that describes what happened to what type of object. They also contain the target of the action that happened. In addition, they must contain a time when the action occurred, the subject, or actor, who caused the action to happen, and may also contain additional data.

  Audit Vault Server organizes the fields of an audit record into these groups: core fields, extension fields, large fields, and marker fields.

1.4.2 Understanding Core Fields

Learn what core fields are, and what their purpose is with Oracle Audit Vault and Database Firewall actions.

Core fields are the fundamental fields that describe an event, and most audit records contain some or all of these fields. However, not all core fields are required in every audit record.

Starting with Oracle Audit Vault and Database Firewall release 12.1.1, the core fields which describe the actions occurred are:

• CommandClass field: The action that caused the audit record to be generated.

• UserName and OsUserName fields: The subject or user who performed the action.

• EventTime field: When, what time, the action occurred.

• ClientHostName, ClientIp, and other related fields: Where, what location, of the action.

• TargetType, TargetOwner, and TargetObject fields: The object type, object owner, or target of the action.

See Also:

Core Fields for a complete list of core fields.

1.4.3 CommandClass and Target Types

Learn about the core fields CommandClass and TargetType in Oracle Audit Vault and Database Firewall.

The CommandClass and TargetType fields have well-known values, which cover a set of general-purpose events that occur in targets belonging to various domains, such as databases or operating systems.

Some examples of the CommandClass values are Logon, Select, Update, and Shutdown.

1.4.4 Other Oracle Audit Vault Fields

In addition to core fields, Oracle Audit Vault Server can interpret Large Fields, Marker fields, and Extension Fields

1.4.4.1 About Large Fields

In Oracle Audit Vault and Database Firewall, large fields are fields that contain arbitrarily large amount of data.

Large fields are fields that contain arbitrarily large amount of data.

1.4.4.2 About Extension Field

In Oracle Audit Vault and Database Firewall, Extension fields provide a way to make target fields that do not have a semantically equivalent Oracle Audit Vault field, and do not map to Core or Large fields.

As a developer, you can determine the format used to store extension fields.

1.4.4.3 About Marker Fields

In Oracle Audit Vault and Database Firewall, Marker fields provide unique identifiers of a record in an audit trail.

A marker field is constructed out of one or more fields in an audit record.

1.4.5 Storing Audit Records in Audit Vault

When you develop plug-ins for Oracle Audit Vault and Database Firewall, Oracle recommends that you follow Oracle guidelines for storing audit records.

As a plug-in developer, you must map the various events that occur within targets, and their fields, to the various fields allowed by Oracle Audit Vault. If a field in the audit record maps to one of the named fields (core, large, or marker fields) in Audit Vault, then you should map it as such.If a field in the audit record does not map to one of the named fields, then you can map it to an extension field of your choosing.

For the Action and TargetType Oracle Audit Vault Server fields, see the list of field values. If your audit record maps to one of these values semantically, then Oracle strongly encourages you to use that value. However, you are free to use other values than the Oracle Audit Vault Server fields.

Oracle strongly encourages you to follow these basic guidelines when you store values in Oracle Audit Vault:

• Do not store IDs that reference objects in the target database. Oracle Audit Vault does not have access to these objects. Consequently, values that refer to objects in the target database are meaningless. Instead, store literal names of objects, so that they can be understood by the auditors.
• Follow defined Audit Vault conventions. For example, all the ACTION fields and TARGET TYPE fields in Oracle Audit Vault have uppercase values. |Oracle recommends that you follow this convention, unless this convention is not applicable to your target type, and would cause the data stored in Oracle Audit Vault to be interpreted incorrectly.
• Map to the values if possible. For example, if TABLE exists in the list as a TargetType, do not add an audit record with the TargetType of DATABASE TABLE.

Finally, if you think a field in the audit record of a target merits becoming a core field, then Oracle recommends that you contact Oracle, so that this field can be reviewed and added to the model appropriately.

1.5 The Collection Process

Learn how the Oracle Audit Vault collection process works.

1.5.1 Flow of Collection: User

When you develop a collection plug-in for Oracle Audit Vault and Database Firewall, your process of development and deployment has a consistent flow.

Collection plug-ins proceed through this lifecycle:

Flow of Development

1. You, the developer, create a collection plug-in and provide it to the user.

2. Your user deploys the plug-in into the Oracle Audit Vault Server. The act of deploying a plug-in into the server creates a new version of the Oracle Audit Vault Agent. This new agent contains collector code from the collection plug-in.

3. Your user then deploys the new Agent onto the host where it needs to run.

   From then on, the user can start collecting audit trails supported by the collector code.

4. Your User starts collecting audit trails supported by the collector code.

Flow of Collection

After the user starts collection, the flow of collection proceeds as shown in this diagram.

Figure 1-1 Flow of Collection for Oracle Audit Vault Collection Agents

Description of "Figure 1-1 Flow of Collection for Oracle Audit Vault Collection Agents"

1.5.2 Flow of Control Inside the Audit Collection Plug-in

After you develop a collection plug-in for Oracle Audit Vault and Database Firewall, the collection plug-in follow this flow as it collects audit trails.

A collection plug-in accesses an audit trail, and extracts an audit record and its related fields from the audit trail. Next, it maps the audit record to an Oracle Audit Vault event, and all the fields to Oracle Audit Vault fields. The collection plug-in then passes the Oracle Audit Vault event and fields to the Audit Vault Agent, which sends the information to the Oracle Audit Vault Server.

Flow of Collection

1. The Oracle Audit Vault Server commands the Agent Framework to create a thread to collect a specific audit trail.
2. The new thread created by the agent collects a specific audit trail, and then turns over control of the thread to the Collection Framework.
3. Within the thread, the Collection Framework connects to the Oracle Audit Vault Server, and queries for configuration information for the audit trail being collected. Additionally, it requests information for the last checkpoint set for that trail. The checkpoint is stored in the Oracle Audit Vault Server in a checkpoint table.
4. With the information it now has, the Collection Framework refers to the plug-in manifest file for the correct Java class to start within the correct collection plug-in. It passes the configuration information to this class, and submits a request to initialize itself.
5. After the the collector has initialized itself, the Collection Framework loops repeatedly. Within each loop, the Collection Framework does the following:

   • Asks the collector for any additional audit records in the audit trail.

     The collector transforms (by mapping) any further audit records into the form of audit records as specified in the mapper file, and hands them to the Collection Framework through the Collection API.

   • The Agent sends the checkpoint information and other metric data received from the collection plug-in to the Audit Vault Server. The Audit Vault Server stores this information in a checkpoint table.

6. If the Audit Vault Server sends commands to the Collection Framework, such as a shutdown command, the Collection Framework passes them to the collector to act on. If the Collection Framework receives a STOP command from the Oracle Audit Vault Server, it notifies the collector to stop sending records. It then exits the collection thread, and shuts itself down.

1.5.3 Collection Concepts

To use Oracle Audit Vault, review basic Oracle Audit Valut basic concepts.

1.5.3.1 Collection Thread

Learn about how Oracle Audit Vault collection threads are run.

The Agent starts collection threads. Within each thread, the Audit Vault Collection Framework executes code provided by the collection plug-in. The collection Framework is the run time infrastructure that exposes the collection API with which the collection plug-in interfaces. The collection plug-in also uses utility APIs if required.

1.5.3.2 Collection Phase

The collection phase is the phase in which Audit Vault collects audit trail records.

During the collection phase, the collection plug-in accesses the audit trail to extract new records. The exact mechanism of how audit trails are accessed depends on the audit trail. After a target audit record is retrieved from the trail, the collection plug-in transforms (maps) it into an audit record that can be sent to the Audit Vault Server.

The collection plug-in must also acquire information about the character set of the target records, the encoding used, and issues related to the time stamps. This is done to make these things coordinate with Audit Vault Server requirements.

1.5.3.3 Mapping

The mappings required from targets to Audit Vault Server depends on the fields in the target records.

These types of mapping are required for the Audit Vault Server:

• Event Mapping: Maps a target specific event to an Audit Vault Server event.
• Field Mapping: Maps the various field of the target records to Audit Vault Server fields.
• Value Mapping: Maps various field values collected into a set of normalized values for each field (for example, 0 and 1 may be mapped to FALSE and TRUE for a specific field).

• Complex Mapping: Complex mappings are used when there are no simple mappings from one target field to an Audit Vault Server field, or one target audit event to one Audit Vault Server event.

  Complex mappings can require additional data from the target, or require additional programming code in the collection plug-in.

See Also:

Configuring Quick JSON Target Type to Collect Audit Data from MongoDB

1.5.3.4 Checkpoint of a Trail

A Checkpoint, or a checkpoint of a trail, is the point up to which audit records were committed to the Oracle Audit Vault Server.

The collection plug-in sets a checkpoint periodically so that it can resume from the last checkpoint when restarted.

1.5.3.5 Recovery Phase Of Data Collection

Learn how Oracle AVDF manages recovery of audit records through checkpoints and recovery mechanisms.

Recovery happens when the Agent stops in the middle of collection and has to be restarted. At this point, the recovery process ensures there are no duplicate records in the Audit Vault Server.

Audit Vault Server ensures that every audit record is archived once and only once. For this purpose, Audit Vault Server implements a checkpoint and recovery mechanism.

In the recovery phase of data collection, a collection plug-in has stopped and restarted, resuming collection. The collection plug-in resumes collection from the checkpoint at which it previously stopped

If the collection plug-in has not collected any records from the audit trail, then the checkpoint occurs before the first record. If the collection plug-in has started collecting records and then stopped, then the checkpoint occurs immediately after the last record that it collected.

Resuming collection immediately after the checkpoint ensures that the collection plug-in does not miss any records. To avoid collecting duplicate records during recovery, the collection plug-in checks the Marker field of each record.

The collection plug-in should not collect and pass on to the agent any records that occurred before the last checkpoint. However, the Agent automatically filters out records committed after the last checkpoint, and recollected when the collection plug-in restarts. Collection plug-ins built using Oracle Audit Vault and Database Firewall SDK write the EventTimeUTC field into a file with the extension .atc. A script can subsequently read this file, and delete audit records as appropriate.

1.5.3.6 Audit Trail Clean Up

Audit Vault Server collection plug-ins can clean up archived audit trail data for targets.

Audit trail clean up is a feature that some targets provide to clean up audit records after they have been archived. If this type of feature exists in the target, an Audit Vault Server collection plug-in can integrate with it, to tell the target to what extent the audit trail has been archived. Identifying which portion of the audit trail is archived enables the target to clean up the audit trail (remove the original audit data) to that point, because cleaning up data that has been archived results in no loss of data. The collection plug-in gives the clean up utility information about the checkpoint, the point up to which data has been collected.

The collection plug-in can notify the clean-up feature of the target system by invoking the appropriate interface of the feature. For instance, the system may read a timestamp from a file in the file system and clean up the audit trail up to that timestamp. If that is the case, the plug-in can write that file periodically.

For example, Oracle Database targets provides this type of utility in the DMBS_AUDIT_MGMT package, and the Oracle Database prepackaged collection plug-ins integrate with it.

1.6 General Procedure for Writing Audit Collection Plug-ins

To ensure that you develop audit collection plug-ins correctly, follow this procedure.

The general procedure for writing collection plug-ins is:

1. Know what capability you want to add to Oracle Audit Vault and Database Firewall, a new target type or a new audit trail for an existing target type.
2. Check if Oracle provides a plug-in that does what you want. If so, then use it.

   Continue only if the plug-in you need does not exist.

   Note:

   Do not create version-dependent target types. Version-dependent target types are different target types written to use for different versions of the same software. If you create version-dependent target types, then Oracle Audit Vault and Database Firewall is unable to collect from the target after it is upgraded to a different version.

   For example, suppose that you create the version-dependent target types SQL Server 2000 and SQL Server 2005 and a collection plug-in that collects the audit trail from a SQL Server 2000 target. If you upgrade that target to SQL Server 2005, then Oracle Audit Vault agent cannot collect its audit trail.

3. Understand the events that your target type writes and their fields.

   Use appropriate existing events and fields when you write your plug-in (for examples of existing events and fields. If the events or fields you need are not available, you can use extension fields. Oracle, from time to time, evaluates the set of fields that Audit Vault supports, and may add new fields if they apply to a broad set of target types. If you believe your fields satisfy this criterion, please contact Oracle Support.

4. Decide which type of collection plug-in to write.
5. Set up the development environment.
6. Learn more about the type of plug-in you are creating.
   Refer to "Java-Based Audit Trail Collection Plug-ins" for information about Java-based plug-ins
7. Determine the following for your collection plug-in:

   • How to connect to the target.

   • How to interrogate the target to learn what you must know.

   • Which platforms your plug-in will support.

8. Decide whether your plug-in will support audit trail cleanup.
9. Set up the collection plug-in parameters.

   For Java-based plug-ins, write the relevant code.

10. Create a plug-in manifest file to describe the collection plug-in.
11. Run the avpack utility to package the plug-in.
12. Test the plug-in the staging environment.
13. If the plug-in works, then make it available to the Oracle Audit Vault administrator to deploy in the development environment, using the command-line commands.

    See Also:

    • Oracle Audit Vault and Database Firewall Administrator's Guide to see the existing plug-ins offered.

    • About Extension Field

    • Types of Audit Collection Plug-Ins

    • Audit Vault Server Fields

    • Setting Up Your Development Environment

    • Audit Collection Plug-ins

    • Audit Trail Clean Up

    • Packaging Audit Collection Plug-ins

    • Packaging Audit Collection Plug-ins

    • Testing Audit Collection Plug-ins

    • Oracle Audit Vault and Database Firewall Administrator's Guide for information on command-line commands.