1 Introduction

This chapter introduces the Audit Collection Plug-in for Audit Vault and Database Firewall. The chapter begins by briefly describing Oracle Audit Vault and Database Firewall software (AVDF) and then explains the collection plug-in. Later chapters describe how you can create custom collection plug-ins to collect audit information written to database tables, XML files, or other locations.

This chapter contains:

• What Is Oracle Audit Vault and Database Firewall?

• What Are Audit Collection Plug-ins?

• Audit Vault Events and Fields

• The Collection Process

• General Procedure for Writing Audit Collection Plug-ins

1.1 What Is Oracle Audit Vault and Database Firewall?

Oracle Audit Vault and Database Firewall is a software system that collects, consolidates, protects, and analyzes audit data from multiple, distributed, heterogeneous systems. It is comprised of these components:

• Oracle Audit Vault Server: A server that contains an embedded Oracle database and other software components that manage the activities of Oracle Audit Vault and Database Firewall.

• Oracle Audit Vault Agent: A Java component that runs on a remote host and manages the collection of audit information based on commands from the Audit Vault server. The agent interfaces with the collection plug-ins under its control to gather audit records and sends it to the Audit Vault Server.

• Oracle Database Firewall: The Database Firewall is a dedicated server that runs the Database Firewall software. Each Database Firewall monitors SQL traffic on the network from database clients to secured target databases. The Database Firewall then sends the SQL data to the Audit Vault Server to be analyzed in reports.

Oracle Audit Vault and Database Firewall ships with several prepackaged collection plug-ins, which are software programs that know how to access and interpret audit data from secured target systems of various types. Collection plug-ins collect audit data from an audit trail generated by a secured target system and store it in an Audit Vault Server repository. Each collection plug-in is specific to a particular type of trail from a particular type of secured target. These collection plug-ins collect data from databases such as Oracle, SQL Server, Sybase ASE, and DB2 as described in Oracle Audit Vault and Database Firewall Administrator's Guide.

1.1.1 How Oracle Audit Vault Server and Agent Work

Audit Collection Plug-ins retrieve audit data in the form of audit trails, which are sequences of audit records. Audit trails are generated by different secured target types such as database tables or XML audit records.

A secured target can write one or more audit trails; each audit trail is stored in a separate location, and can have its own format.

To elaborate a little on these terms:

• Secured Target

  A secured target is a software or hardware system performing a specific function. As part of performing that function, the secured target system generates an audit trail. A secured target is an instance of a secured target type and has specific properties such as connection credentials and trail types.

• Secured Target Type

  A secured target type represents a collection of a particular type of secured target that generates the same type of audit data. Oracle Database, for example, is a secured target type which can have many instances. However, all Oracle Databases generate the same audit data and record the same fields.

• Audit Trail

  An Audit Trail identifies a location and format where audit data resides. Each audit trail is generated by one and only one secured target. Examples of audit trails are:

  • For secured targets that write data into files, the trail is the directory path plus the file mask.

  • For secured targets that write audit data into a database table, the name of the table is the trail for that secured target. SYS.AUD$ is an example of a database table audit trail in an Oracle database.

1.2 What Are Audit Collection Plug-ins?

A collection plug-in provides functionality similar to the prepackaged collection plug-ins shipped with Oracle Audit Vault and Database Firewall, by retrieving audit data stored in audit trails. See "What Is Oracle Audit Vault and Database Firewall?".

Starting with Oracle Audit Vault and Database Firewall release 12.1.1, developers, as well as third-party vendors, can build custom collection plug-ins. These collection plug-ins can collect audit data from a new secured target type or support new audit trails written by a secured target type that AVDF already knows about, and make them available to users.

You can write collection plug-ins that collect audit trails stored in database tables and XML files, or are accessible in another way.

You can support secured targets such as relational databases, operating systems, mid-tier systems, or enterprise applications.

This guide describes how you can create these custom collection plug-ins and deploy them into existing Oracle Audit Vault and Database Firewall installations.

1.2.1 Types of Audit Collection Plug-ins

You can create two types of collection plug-ins. The actual type you need to create depends on the properties of the audit trail being collected.

A collection plug-in uses an XML file, called a mapper file, which you create, to describe the audit data being collected. Audit Vault Server uses this file to access and interpret the audit records being collected. You do not need to write code for this

1.2.1.1 Determining Which Audit Collection Plug-in Type to Create

This section describes the two types of collection plug-ins and explains the properties that are required to use each type.

You can easily define a mapper file (template) and a collection plug-in if the audit trails you wish to collect are stored in either of the following:

• Database Tables: Stored in database tables that conform to specific constraints

• XML Files: Stored in XML files based on the Audit Vault XML Audit File format

See Also:

• "Database Table Collection Plug-in Example"

• "XML File Collection Plug-in Example"

1.3 Audit Vault Events and Fields

Monitoring the activity, the stream of events, that occur in a secured target system is the essence of Oracle Audit Vault and Database Firewall. These events are described by fields. A collection of fields describing a single event that occurred on the secured target system is an audit record.

Oracle Audit Vault and Database Firewall release 12.1.1 greatly simplifies the representation of events and event structure as compared to Audit Vault 10.3.

In Oracle Audit Vault and Database Firewall release 12.1.1 the following applies:

• Each secured target logs events as audit events that occur on that secured target. Audit records capture information about audit events.

• Audit records typically have a secured target type event name that describes what happened to what type of object. They also contain the target of the action that happened. In addition, they must contain a time when the action occurred, the subject, or actor, who caused the action to happen, and may also contain additional data.

  Audit Vault Server organizes the fields of an audit record into these groups: core fields, extension fields, large fields, and marker fields.

1.3.1 Core Fields

Core fields are the fundamental fields that describe an event, and most audit records contain some or all of these fields. However, not all core fields are required in every audit record.

In Oracle Audit Vault and Database Firewall release 12.1.1, the core fields, which describe the actions that occurred are:

• CommandClass field: The action that caused the audit record to be generated.

• UserName and OsUserName fields: The subject or user who performed the action.

• EventTime field: When, what time, the action occurred.

• ClientHostName, ClientIp, and other related fields: Where, the location, of the action.

• TargetType, TargetOwner, and TargetObject fields: The object type, object owner, or target of the action.

For a complete list of core fields, see core fields listed in Appendix A, Section A.1.1, "Core Fields".

1.3.1.1 CommandClass and Target Types

The CommandClass and TargetType fields have well-known values, which cover a set of general-purpose events that occur in secured targets belonging to various domains, such as databases or operating systems.

Examples of CommandClasses are Logon, Select, Update, and Shutdown.

These are listed in Appendix A, Section A.2, "Actions and Target Types".

1.3.2 Other Audit Vault Fields

In addition to core fields, Audit Vault Server understands these categories:

• Large Fields

• Marker Field

• Extension Field

1.3.2.1 Large Fields

Large fields are fields that can contain arbitrarily large amounts of data. See Section A.1.2, "Large Fields" for further information.

1.3.2.2 Extension Field

The Extension field provides a way to make available secured target fields that do not have a semantically equivalent Audit Vault field and do not map to Core or Large fields.

The developer determines the format used to store the fields.

See Section A.1.4, "Extension Field" for further information.

1.3.2.3 Marker Field

The Marker field is a unique identifier of a record in a trail. It is constructed out of one or more fields in an audit record.

See Section A.1.3, "Marker Field" for further information.

1.3.3 Storing Audit Records in Audit Vault

As a plug-in developer, you must map the various events that occur within secured targets, and their fields, to the various fields allowed by Audit Vault.

If a field in the audit record maps to one of the named fields (core, large, or marker fields) in Audit Vault, you should map it as such.

If a field in the audit record does not map to one of the named fields, you can map it to an extension field of your choosing.

For the Action and TargetType Audit Vault Server fields, see the list of field values in the "Actions and Target Types". If your audit record maps to one of these values semantically, we strongly encourage you to use that value. You are, however, free to use values not listed in the appendix.

You are strongly encouraged to follow these basic guidelines when you store values in Audit Vault.

• Do not store IDs that reference objects in the secured target database. Audit Vault does not have access to these objects, and the stored values will be meaningless. Store literal names of objects instead, so they can be understood by the auditors,

• Follow defined Audit Vault conventions. For example, all the ACTION fields and TARGET TYPE fields in Audit Vault have uppercase values. Please follow this convention, unless this is not applicable to your secured target type, and would cause the data stored in Audit Vault to be interpreted incorrectly,

• Map to the values documented in "Actions and Target Types" if possible. For example, if TABLE exists in the list as a TargetType, do not add an audit record with the TargetType of DATABASE TABLE.

Finally, if a field in the audit record of a secured target merits becoming a core field, please contact Oracle so that it can be added to the model appropriately.

1.4 The Collection Process

This section covers the following topics:

• Flow of Collection: User

• Flow of Control Inside the Audit Collection Plug-in

• Collection Concepts

1.4.1 Flow of Collection: User

1. You, the developer, create a collection plug-in and provide it to the user.

2. The user deploys the plug-in into the Audit Vault Server. The act of deploying a plug-in into the server creates a new version of the Audit Vault Agent. This new agent contains collector code from the collection plug-in.

3. The user then deploys the new Agent onto the host where it needs to run.

4. From then on, the user can start collecting audit trails supported by the collector code.

5. User starts collecting audit trails supported by the collector code.

The next section describes what happens inside the collection plug-in after the user starts collection.

Figure 1-1 Flow of Collection

Description of ''Figure 1-1 Flow of Collection''

1.4.2 Flow of Control Inside the Audit Collection Plug-in

Once a collection plug-in accesses an audit trail, and extracts an audit record and its related fields from the audit trail, then it maps the audit record to an Audit Vault event, and all the fields to Audit Vault fields. The collection plug-in then passes the Audit Vault event and fields to the Audit Vault Agent, which sends the information to the Audit Vault Server.

This section describes this process in detail.

1. The Audit Vault Server commands the Agent Framework to create a thread to collect a specific audit trail. See "Collection Thread"l

2. The new thread, just created by the agent, collects a specific audit trail.

   At this point control is handed to the Collection Framework. See "Collection Phase".

3. Within the thread, the Collection Framework connects to the Audit Vault Server and queries for configuration information for the audit trail being collected.

   Additionally, it requests information for the last checkpoint set for that trail. See "Checkpoint of a Trail".

4. With the information it now has, the Collection Framework determines, through the plug-in manifest file, the correct Java class to invoke within the correct collection plug-in. It passes the configuration information to this class, and asks it to initialize itself.

5. Once the collector has initialized itself, the Collection Framework loops repeatedly. Within each loop, the Collection Framework does the following:

   • Asks the collector for any additional audit records in the audit trail.

     The collector transforms (by mapping) any further audit records into the form of audit records that Audit Vault Server expects, and hands them to the Collection Framework through the Collection API. See "Mapping"

   • Sends the Audit Vault Server any checkpoint information and other metric data received from the collection plug-in.

6. If the Audit Vault Server sends commands to the Collection Framework, such as a shutdown command, the Collection Framework passes them to the collector to act on. If the Collection Framework receives a STOP command from the Audit Vault Server, it notifies the collector to stop sending records, then exits the collection thread, shuts itself down.

1.4.3 Collection Concepts

This section explains some of the concepts related to the collection process.

1.4.3.1 Collection Thread

The agent starts threads of collection. Within each thread, the Audit Vault Collection Framework executes code provided by the collection plug-in. The collection Framework is the run-time infrastructure that exposes the collection API which the collection plug-in interfaces with. The collection plug-in also uses utility APIS (not shown) if it needs to.

1.4.3.2 Collection Phase

During the collection phase, the collection plug-in accesses the audit trail to extract new records. The exact mechanism of how audit trails are accessed depend on the audit trail. Once a secured target audit record is retrieved from the trail, the collection plug-in must be able to transform (map) it into an audit record hat can be sent to the Audit Vault Server. See "Mapping".

The collection plug-in must also acquire information about the character set of the secured target records, the encoding used, and issues related to the time stamps, in order to make these things coordinate with Audit Vault Server requirements.

1.4.3.3 Mapping

The mappings required depend on the secured target records.

These types of mapping are required:

• Event Mapping: Maps a secured target-specific event to an Audit Vault Event.

• Field Mapping: Maps the various field of the secured target records to Audit Vault Fields.

• Value Mapping: Maps various field values collected into a set of normalized values for each field (for example, 0 and 1 may be mapped to FALSE and TRUE for a specific field).

• Complex Mapping: Complex mappings are used when there are no simple mappings from one secured target field to an Audit Vault Field, or one secured target audit event to one Audit Vault Event.

1.4.3.4 Checkpoint of a Trail

The checkpoint or the checkpoint of a trail is the point (as a timestamp) up to which the audit records were committed to the Audit Vault Server. The collection plug-in sets a checkpoint periodically so that it can resume from the last checkpoint when restarted.

1.4.3.5 Recovery Phase Of Data Collection

Audit Vault provides a once and only once guarantee to its users, stating that each audit record is archived in the Audit Vault Server once and only once. It implements a checkpoint and recovery mechanism for this purpose.

In the recovery phase of data collection, a collection plug-in has stopped and restarted, resuming collection. The collection plug-in resumes collection from the checkpoint at which it had previously stopped

If the collection plug-in has not collected any records from the audit trail, then the checkpoint occurs before the first record. If the collection plug-in has started collecting records and then stopped, then the checkpoint occurs immediately after the last record that it collected.

Resuming collection immediately after the checkpoint ensures that the collection plug-in does not miss any records. To avoid collecting duplicate records during recovery, the collection plug-in checks the Marker field of each record.

The collection plug-in should not collect and pass on to the agent any records that occurred before the last checkpoint. The agent, however, automatically filters out records that were committed after the last checkpoint and re-collected when the collection plug-in restarts. In fact, collection plug-ins built using this SDK (see Chapter 3) write the EventTimeUTC field into a file with the extension.atc. A script can subsequently read this file and delete audit records as appropriate.

1.4.3.6 Audit Trail Clean-Up

Audit Trail clean-up is a feature that some secured targets provide to clean up audit records once they've been archived. If this type of feature exists in the secured target, an Audit Vault collection plug-in can integrate with it, to tell the secured target to what extent the audit trail has been archived. This enables the secured target to clean up the audit trail (remove the original audit data) to that point, knowing this will not result in loss of data. The collection plug-in gives the clean-up utility information about the checkpoint, the point up to which data has been collected.

The collection plug-in can notify the clean-up feature of the secured target system by invoking the appropriate interface of the feature. For instance, the system may read a timestamp from a file in the filesystem and clean up the audit trail up to that timestamp. If that is the case, the plug-in can write that file periodically.

For example, Oracle Database secured targets provides this type of utility in the DMBS_AUDIT_MGMT package, and the Oracle Database prepackaged collection plug-ins integrate with it.

1.5 General Procedure for Writing Audit Collection Plug-ins

The general procedure for writing collection plug-ins is:

1. Know what capability you want to add to Oracle Audit Vault and Database Firewall, a new secured target type or a new audit trail for an existing secured target type.

2. Check Oracle Audit Vault and Database Firewall Administrator's Guide to see if Oracle provides a plug-in that does what you want. If so, use it; do not reinvent the wheel.

   Continue only if the plug-in you need does not exist.

   Note:

   Do not create version-dependent secured target types, that is, different secured target types for different versions of the same software. If you do, then AVDF cannot collect from the secured target after it is upgraded to a different version.

   For example, suppose that you create the version-dependent secured target types SQL Server 2000 and SQL Server 2005 and a collection plug-in that collects the audit trail from a SQL Server 2000 secured target. If you upgrade that secured target to SQL Server 2005, then Oracle Audit Vault agent cannot collect its audit trail.

3. Understand the events that your secured target type writes and their fields.

   Use appropriate existing events and fields when you write your plug-in (for examples of existing events and fields (see Appendix A, "Audit Vault Server Fields"). If the events or fields you need are not available, you can use extension fields (see "Extension Field"). Oracle, from time to time, evaluates the set of fields that Audit Vault supports, and may add new fields if they apply to a broad set of secured target types. If you believe your fields satisfy this criterion, please contact Oracle Support.

4. Decide which type of collection plug-in to write, as discussed in "Types of Audit Collection Plug-ins".

5. Set up the development environment. See Chapter 2, "Setting Up Your Development Environment".

6. Learn more about the type of plug-in you are creating in Chapter 3, "Audit Collection Plug-ins".

7. Determine the following for your collection plug-in:

   • How to connect to the secured target.

   • How to interrogate the secured target to learn what you must know.

   • Which platforms your plug-in will support.

8. Decide whether your plug-in will support audit trail cleanup as described in "Audit Trail Clean-Up".

9. Set up the collection plug-in parameters.

10. Create a plug-in manifest file to describe the collection plug-in. See Chapter 4, "Packaging Audit Collection Plug-ins".

11. Run the avpack utility to package the plug-in (see Chapter 4, "Packaging Audit Collection Plug-ins").

12. Test the plug-in the staging environment (see Chapter 5, "Testing Audit Collection Plug-ins").

13. If the plug-in works, make it available to the Audit Vault administrator to deploy in the development environment, using the command-line commands described in Oracle Audit Vault and Database Firewall Administrator's Guide.