SunSHIELD Basic Security Module Guide

Chapter 3 Audit Trail Analysis

Using the tools described in this chapter, you can develop shell scripts to manage and report on the audit files and then run these scripts periodically. Audit management tasks might include compressing files, combining multiple audit files into one, moving files to different locations on disks in the distributed system, or archiving old files to tape. The scripts can also monitor storage usage, although the audit daemon does some of that automatically.

Another auditing task is to examine the audit trail, which is the logical combination of all the audit files. You can use the auditing tools to interactively query the audit data files for specific information.

Auditing Features

The following features of Solaris BSM auditing are provided to interpret the audit records:

Because each audit record contains an audit ID that identifies the user who generated the event, and because full path names are recorded in audit records, you can look at individual audit records and get meaningful information without looking back through the audit trail.

Audit User ID

Solaris BSM processes have an additional user identification attribute not associated with processes in the standard Solaris release: the audit ID. A process acquires its audit ID at login time, and this audit ID is inherited by all child processes.

Audit Session ID

Solaris BSM processes have an audit session ID assigned at login time. The ID is inherited by all child processes.

Self-Contained Audit Records

The Solaris BSM audit records contain all the relevant information about an event and do not require you to refer to other audit records to interpret what occurred. For example, an audit record describing a file event contains the file's full path name starting at the root directory and a time and date stamp of the file's opening or closing.

Tools for Merging, Selecting, Viewing, and Interpreting Audit Records

Solaris BSM provides two tools that allow you to merge, select, view, and interpret audit records. The tools can be used directly or in conjunction with third-party application programs.

The following sections describe the audit record format, the praudit, and auditreduce commands in more detail, and provide some hints and procedures for using the tools.

Audit Record Format

A Solaris BSM audit record consists of a sequence of audit tokens, each of which describes an attribute of the system.

Appendix A, Audit Record Descriptions gives a detailed description of each audit token. The appendix also lists all the audit records generated by Solaris BSM auditing. The definitions are sorted in order of the short descriptions, and a cross-reference table translates event names to event descriptions.

Binary Format

Audit records are stored and manipulated in binary form; however, the byte order and size of data is predetermined to simplify compatibility between different machines.

Audit Event Type

Each auditable event in the system generates a particular type of audit record. The audit record for each event has certain tokens within the record that describe the event. An audit record does not describe the audit event class to which the event belongs; that mapping is determined by an external table, the /etc/security/audit_event file.

Audit Token Types

Each token starts with a one-byte token type, followed by one or more data elements in an order determined by the type. The different audit records are distinguished by event type and different sets of tokens within the record. Some tokens, such as the text token, contain only a single data element, while others, such as the process token, contain several (including the audit user ID, real user ID, and effective user ID).

Order of Audit Tokens

Each audit record begins with a header token and ends (optionally) with a trailer token. One or more tokens between the header and trailer describe the event. For user-level and kernel events, the tokens describe the process that performed the event, the objects on which it was performed, and the objects' tokens, such as the owner or mode.

Each user-level and kernel event typically has at least the following tokens:

Many events also include a trailer token, but it is optional.

Human-Readable Audit Record Format

This section shows each audit record format as it appears in the output produced by the praudit command. This section also gives a short description of each audit token. For a complete description of each field in each token, see Appendix A, Audit Record Descriptions.

The following token examples show the form that praudit produces by default. Examples are also provided of raw (-r) and short (-s) options. When praudit displays an audit token, it begins with the token type, followed by the data from the token. Each data field from the token is separated from other fields by a comma. However, if a field (such as a path name) contains a comma, this cannot be distinguished from a field-separating comma. Use a different field separator or the output will contain commas. The token type is displayed by default as a name, like header, or in -r format as a decimal number.

The individual tokens are described in the following order:

header Token

Every audit record begins with a header token. The header token gives information common to all audit records. The fields are:

When displayed by praudit in default format, a header token looks like the following example from ioctl:

header,240,1,ioctl(2),es,Tue Sept  1 16:11:44 1992, + 270000 msec

Using praudit -s, the event description (ioctl(2) in the default praudit example above) is replaced with the event name (AUE_IOCTL), like this:

header,240,1,AUE_IOCTL,es,Tue Sept 1 16:11:44 1992, + 270000 msec

Using praudit -r, all fields are displayed as numbers (that can be decimal, octal, or hex), where 158 is the event number for this event.

20,240,1,158,0003,699754304, + 270000 msec

Notice that praudit displays the time to millisecond resolution.

trailer Token

This token marks the end of an audit record and allows backward seeks of the audit trail. The fields are:

A trailer token is displayed by praudit as follows:


arbitrary Token

This token encapsulates data for the audit trail. The item array can contain a number of items. The fields are:

An arbitrary token is displayed by praudit as follows:


arg Token

This token contains system call argument information. A 32-bit integer system call argument is allowed in an audit record. The fields are:

An arg token is displayed by praudit as follows:


attr Token

This token contains information from the file vnode. The attr token is usually produced during path searches and accompanies a path token, but is not included in the event of a path-search error. The fields are:

An attr token is displayed by praudit as follows:


exit Token

An exit token records the exit status of a program. The fields are:

An exit token is displayed by praudit as follows:

exit,Error 0,0

file Token

This token is generated by the audit daemon to mark the beginning of a new audit trail file and the end of an old file as the old file becomes deactivated. The audit record containing this token links successive audit files into one audit trail. The fields are:

A file token is displayed by praudit as follows:

file,Tue Sep  1 13:32:42 1992, + 79249 msec,

groups Token

A groups token records the groups entries from a process's credential. The fields are:

A groups token is displayed by praudit as follows:


in_addr Token

An in_addr token gives a machine Internet Protocol address. The fields are:

An in_addr token is displayed by praudit as follows:

ip addr,

ip Token

The ip token contains a copy of an Internet Protocol header. The fields are:

An ip token is displayed by praudit as follows:

ip address,

ipc Token

This token contains the System V IPC message/semaphore/shared-memory handle used by a caller to identify a particular IPC object. The fields are:

An ipc token is displayed by praudit as follows:


ipc_perm Token

An ipc_perm token contains a copy of the System V IPC access information. Audit records for shared memory, semaphore, and message IPCs have this token added. The fields are:

An ipc_perm token is displayed by praudit as follows:

IPC perm,root,wheel,root,wheel,0,0,0x00000000

iport Token

This token contains a TCP (or UDP) address. The fields are:

An iport token is displayed by praudit as follows:

ip port,0xf6d6

opaque Token

The opaque token contains unformatted data as a sequence of bytes. The fields are:

An opaque token is displayed by praudit as follows:


path Token

A path token contains access path information for an object. The fields are:

A path token is displayed by praudit as follows:


process Token

The process token contains information describing a process. The fields are:

A process token is displayed by praudit as follows:


return Token

A return token gives the return status of the system call and the process return value. This token is always returned as part of kernel-generated audit records for system calls. The fields are:

A return token is displayed by praudit as follows:


seq Token

This token is optional and contains an increasing sequence number used for debugging. The token is added to each audit record when the seq policy is active. The fields are:

A seq token is displayed by praudit as follows:


socket Token

A socket token describes an Internet socket. The fields are:

A socket token is displayed by praudit as follows:


subject Token

This token describes a subject (process). The fields are:

A subject token is displayed by praudit as follows:

subject,cjc,cjc,staff,cjc,staff,424,223,0 0 quisp

text Token

A text token contains a text string. The fields are:

A text token is displayed by praudit as follows:


Using the auditreduce Command

The auditreduce command merges audit records from one or more input audit files. You would usually enter this command from the machine on which all the audit trail files for the entire distributed system are mounted.

Without options, auditreduce merges the entire audit trail (all of the audit files in all of the subdirectories in the audit /etc/security/audit directory) and sends the merged file to standard output.

The praudit command, described in "Using praudit" makes the records human-readable.

These are some of the capabilities provided by options to the auditreduce command:

How auditreduce Helps in a Distributed System

When multiple machines running Solaris BSM are administered as part of a distributed system, each machine performs auditable events, and each machine writes audit records to its own machine-specific audit file. This simplifies software and is robust in the face of machine failures.

The auditreduce command makes the job of maintaining the whole audit trail practical. Using auditreduce (or shell scripts you write yourself to provide a higher-level interface), you can read the logical combination of all audit files in the system as a single audit trail, without regard to how the records were generated or where they are stored.

The auditreduce program operates on the audit records produced by the audit daemon. Records from one or more audit files are selected and merged into a single, chronologically ordered output file. The merging and selecting functions of auditreduce are logically independent. auditreduce selects messages from the input files as the records are read, before the files are merged and written to disk. Refer to the auditreduce(1M) man page.

Using auditreduce

This section describes a few common uses of auditreduce to analyze and manage data.

How to Display the Whole Audit Log

To display the whole audit trail at once, pipe the output of auditreduce into praudit.

#auditreduce | praudit

How to Print the Whole Audit Log

With a pipe to lp, the output goes to the printer.

# auditreduce | praudit | lp

How to Display User Activity from a Selected Date

In the following example, the system administrator checks to see when a user named fred logged in and logged out on April 13, 1990, by requesting the lo event class. The short-form date is in the form yymmdd. (The long form is described in the auditreduce(1M) man page.)

# auditreduce -d 900413 -u fred -c lo | praudit

How to Copy Login/Logout Messages to a Single File

In this example, login/logout messages for a particular day are summarized in a file. The target file is written in a directory other than the normal audit root.

# auditreduce -c lo -d 870413 -O /usr/audit_summary/logins 

The -O option creates an audit file with 14-character timestamps for both start-time and end-time, and the suffix logins:


How to Clean Up a not_terminated Audit File

Occasionally, if an audit daemon dies while its audit file is still open, or a server becomes inaccessible and forces the machine to switch to a new server, an audit file remains in which the end-time in the file name is the string not_terminated, even though the file is no longer used for audit records. When such a file is found, you can manually verify that the file is no longer in use and clean it up by specifying the name of the file with the correct options.

# auditreduce -O machine 19870413120429.not_terminated.machine

This creates a new audit file with the correct name (both time stamps), the correct suffix (machine, explicitly specified), and copies all the messages into it.

Other Useful auditreduce Options

auditreduce has many additional options described in the man page. Notice that the uppercase options select operations or parameters for files, and the lowercase options select parameters for records. This subsection shows how to utilize other useful options.

The date-time options -b and -a allow you to specify records before or after a particular day and time. A day begins at yyyymmdd00:00:00 and ends at yyyymmdd23:59:59. The six parameters of a day are: year, month, day, hour, minute, and second. The digits (19) of the year are assumed and need not be specified.

If -a is not specified, auditreduce defaults to 00:00:00, January 1, 1970. If -b is not specified, auditreduce defaults to the current time of day (GMT). The -d option selects a particular 24-hour period, as shown in "How to Copy Login/Logout Messages to a Single File".

The auditreduce -a command with the date shown in the following screen example sends all audit records created after midnight on July 15, 1991, to praudit.

# auditreduce -a 91071500:00:00 | praudit 

The auditreduce -b command with the same date shown above sends all audit records created before midnight on July 15, 1991 to praudit.

# auditreduce -b 91071500:00:00 | praudit 

The message type selection for auditreduce (-m option) accepts either numeric message identifiers or AUE_xxxxx codes. auditreduce rejects an incorrect format, but does not describe the correct format.

Using praudit

The praudit command reads audit records from standard input and displays them on standard output in human-readable form. Usually, the input is either piped from auditreduce or a single audit file. Input can also be produced with cat to concatenate several files or tail for a current audit file.

praudit can generate three output formats: default, short (-s option), and raw (-r option). By default, output is produced with one token per line. The -l option requests a whole record on each line. The -d option changes the delimiter used between token fields, and between tokens, if -l is also specified.

In -s format, the type is the audit event table name for the event (such as AUE_IOCTL), and in -r format, it is the event number (in this case, 158). That is the only distinction between -s and default format. In -r format, all numeric values (user IDs, group IDs, and so forth) are displayed numerically (in decimal, except for Internet addresses, which are in hex, and for modes, which are in octal). Here is the output from praudit for a header token:

header,240,1,ioctl(2),es,Tue Sept  1 16:11:44 1992, + 270000 msec

And here is the output from praudit -r for the same header token:

20,240,1,158,0003,699754304, + 270000 msec

It is sometimes useful to manipulate the output as lines of text; for example to perform selections that cannot be done with auditreduce. A simple shell script can process the output of praudit. The following example is called praudit_grep:

praudit | sed -e '1,2d' -e '$s/^file.*$//' -e 's/^header/^aheader/' \\
| tr '\\012\\001' '\\002\\012' \\
| grep "$1" \\
| tr '\\002' '\\012'

The example script marks the header tokens by prefixing them with Control-A. (Note that the ^a is Control-a, not the two characters ^ and a. Prefixing is necessary to distinguish them from the string header that might appear as text.) The script then combines all the tokens for a record onto one line while preserving the line breaks as Control-a, runs grep, and restores the original new lines.

In the default output format of praudit, each record can always be identified unambiguously as a sequence of tokens (each on a separate line) beginning with a header token. Each record, therefore, is easily identified and processed with awk, for example.