test

Trusted Solaris Audit Administration

Chapter 3 Audit Trail Management and Analysis

The tools described in this chapter manage the audit files generated on a workstation or on a distributed system. Managing the audit trail involves file tasks and interpretive tasks. File tasks handle disk space issues, such as combining multiple audit files into one and renaming files. Interpretive tasks cover audit analysis, such as selecting audit records based on audit event, user, host machine, and time of day. Sophisticated postprocessing using shell scripts can create auditing reports.

The chapter includes procedures in the following areas:

The Audit Trail

The collection of all audit files in a distributed system is called the audit trail. The audit trail may consist of audit files in several audit directories, or an audit directory may contain several audit trails. Most often the audit directories will be separate audit file system partitions. Even though they can be included in other file systems, this is not recommended.

Audit files by default are stored in the audit root directory, defined as /etc/security/audit/*/files. Once each workstation has created an audit root directory, and the directories have been mounted (with mount points that follow the naming convention) on the audit administration server, the management tools, auditreduce and praudit, can examine the entire audit trail. See "Basic Audit Setup Procedures" for how to set up an audit trail.

Even though it is possible to locate audit directories within other file systems that are not dedicated to auditing, this is not recommended. If other factors dictate placing audit files on a partition not dedicated to auditing, only do so for directories of last resort. Directories of last resort would be directories where audit files would be written only when there is no other suitable directory available. One other scenario where locating audit directories outside of dedicated audit file systems could be acceptable would be in an environment where auditing is optional, and where it is more important to make full use of disk space than to keep an audit trail. Putting audit directories within other file systems is unworkable in a security-conscious production environment.

How the Audit Trail Is Created

The audit trail is created by the audit daemon, auditd(1M). The audit daemon starts on each workstation when the workstation is booted. After auditd starts, it is responsible for collecting the audit trail data and writing the audit records into audit files, which are also called audit log files. See the audit.log(4) man page for a description of the file format.

Figure 3-1 How Auditing Works

Graphic

The audit daemon runs as root. All files it creates are owned by root. Even when auditd has no classes to audit, auditd continuously operates, looking for a place to put audit records. The auditd operations continue even if the rest of the workstation's activities are suspended because the kernel's audit buffers are full. The audit operations can continue because auditd is not audited.

Audit Record Format

Audit files consist of self-contained audit records of user-level and kernel-level events that have been preselected for auditing by the security administrator. An audit record consists of a sequence of audit tokens, each of which describes an attribute of the event being audited. Each auditable event in the system generates a particular type of audit record. The audit record for each event has certain tokens within the record that describe the event. An audit record does not describe the audit event class to which the event belongs; that mapping is determined by an external table, the /etc/security/audit_event file.

Each audit token starts with a one-byte token type, followed by one or more data elements in an order determined by the type. The different audit records are distinguished by event type and different sets of tokens within the record. Some tokens, such as the text token, contain only a single data element, while others, such as the process token, contain several (including the audit user ID, real user ID, and effective user ID).

Audit records are stored and manipulated in binary form; however, the byte order and size of data is predetermined to simplify compatibility between different workstations.

"Audit Token Structure", gives a detailed description of each data element in each token and shows sample output. "Audit Records" lists all the audit records generated by Trusted Solaris 7 auditing. The records are listed alphabetically by kernel event and by user event. A cross-reference table to the audit records is found in Appendix A, Event-to-Class Mappings.

Order of Audit Tokens

Each audit record begins with a header token and ends (optionally) with a trailer token. One or more tokens between the header and trailer describe the event. For user-level and kernel events, the tokens describe the process that performed the event, the objects on which it was performed, and the objects' attributes, such as the owner or mode.

For example, the AUE_LSTAT kernel event, whose audit record is described in Table B-70, has the following tokens:

If the trail policy has been turned on using the auditconfig command, the trailer token appears in the audit record after the return token.

Human-Readable Audit Record Format

This section provides examples of audit records in text format. Audit records are stored in binary format. Running the binary records through the praudit command produces text output, which can be sent to standard output, a printer, or a scripting program to produce reports. For a complete description of praudit, see the praudit(1M) man page. For an example of a scripting program, see "To Perform Selections Using a praudit Script".

Reading an Audit Token

The following examples of a header token show the form that praudit produces by default. Examples are also provided of raw (-r) and short (-s) options.

Every audit record begins with a header token. The header token gives information common to all audit records. When displayed by praudit in default format, a header token looks like the following example from ioctl(): 

header,240,1,ioctl(2),,Tue Sept  7 16:11:44 1999, + 270000 msec

The fields are:

Using praudit -s, the event description (ioctl(2) in the default praudit example above) is replaced with the event name (AUE_IOCTL), like this:

header,240,1,AUE_IOCTL,,Tue Sept 7 16:11:44 1999, + 270000 msec

Using praudit -r, all fields are displayed as numbers (that may be decimal, octal, or hex), where 20 is the header token ID and 158 is the event number for this event. 

20,240,1,158,,699754304, + 270000 msec

Note that praudit displays the time to millisecond resolution.

Reading an Audit Record

Every audit record contains at least the header token and one other token. For example, the audit record for the audit event AUE_login contains five tokens. See Table B-247 for a full description of its audit record format.

When displayed by praudit in default format, the audit record for AUE_login looks like this, one token per line: 

header,90,3,login - local,,Tue Jul 8 15:12:01 1997, +520002000 msec,
text,emily
text,successful login
subject,emily,emily,staff,emily,staff,14094,14094,0 0 willet,
return,success,0
sequence,17
trailer,90

The tokens are:

When this audit file collected records, the audit policy tokens sequence and trailer were turned on, so all audit records including this one contain the following tokens:

Note the following features in the audit record:

Because each audit record contains an audit ID that identifies the user who generated the event, and because audit records are self-contained, you can look at individual audit records and get meaningful information without looking back through the audit trail.

The Trusted Solaris 7 audit records contain all the relevant information about an event and do not require you to refer to other audit records to interpret what occurred. For example, an audit record describing a file event contains the file's full path name starting at the root directory and a time and date stamp of the file's opening or closing.

Note -

You should archive system administration files with audit file archives. Information that is referred to in the audit trail but changes as site personnel and equipment change, such as users and their UIDs, affects your ability to interpret records.

Using praudit -l, the audit record displays on one line, like this: 

header,90,3,login - local,,Tue Jul 8 15:12:01 1997, +520002000 msec,text,emily,text,successful
login,subject,emily,emily, staff,emily,staff,14094,14094,0 0 willet,return,success,0,
sequence,17,trailer,90

Using praudit -r the audit record displays like this: 

20,90,3,6152,0x0000,872028721,520002000
40,emily
40,successful login
36,6001,6001,10,6001,10,14094,14094,0 0 129.150.110.2
39,0,0
47,17
19,90

Audit Files

Each audit file is a self-contained collection of records; the file's name identifies the time span during which the records were generated and the workstation that generated them. The contents of the audit files are binary, protected at the sensitivity label admin_high, and accessible in a profile shell only by an administrative role with the Audit Review profile.

Audit File Naming

Audit files that are complete have names of the following form: 

start-time.finish-time.workstation

where start-time is the time of the first audit record in the audit file, finish-time is the time of the last record, and workstation is the name of the workstation that generated the file. Some examples of these names can be found in "Example of a Closed Audit File Name ".

If the audit log file is still active, it has a name of the following form:

start-time.not_terminated.workstation

How Audit File Names Are Used

The file name time stamps are used by the auditreduce command to locate files containing records for the specific time range that has been requested. This is important because there may be a month's supply or more of audit files online, and searching them all for records generated in the last 24 hours would be expensive.

Time-Stamp Format and Interpretation

The start-time and finish-time are time stamps with one-second resolution; they are specified in Greenwich mean time. The format is four digits for the year, followed by two for each month, day, hour, minute, and second, as shown below. 

YYYYMMDDHHMMSS

The time stamps are in GMT to ensure that they will sort in proper order even across a daylight saving time boundary. Because they are in GMT, the date and hour must be translated to the current time zone to be meaningful; beware of this whenever manipulating these files with standard file commands rather than with auditreduce.

Example of a File Name for a Still-Active File

The format of a file name of a still-active file is shown below: 

YYYYMMDDHHMMSS.not_terminated.hostname

Here is an example: 

19900327225243.not_terminated.patchwork

The audit log files are named by the beginning date, so the example above was started in 1997, on March 27, at 10:52:43 PM, GMT. The not_terminated in the file name means either that the file is still active or that auditd was unexpectedly interrupted. The name patchwork at the end is the host name whose audit data is being collected.

Example of a Closed Audit File Name

The format of the name of a closed audit log file is shown below: 

YYYYMMDDHHMMSS.YYYYMMDDHHMMSS.hostname

Here is an example: 

19970320005243.19970327225351.patchwork

The example above was started in 1997, on March 20, at 12:52:43 AM, GMT. The file was closed March 27, at 10:53:51 PM, GMT. The name patchwork at the end is the host name of the workstation whose audit data is being collected.

Audit Files Management

Two commands, praudit(1M) and auditreduce(1M), enable the audit reviewer to process the Trusted Solaris audit records. The praudit command makes the records readable, and the auditreduce command enables selecting particular audit records and merging the records into one audit trail.

Note -

The auditreduce command can only find records that have been preselected by the security administrator. Events that are not recorded in the audit trail are unavailable to postselection tools.

Merging the Audit Trail

The auditreduce command merges audit records from one or more input audit files to create a single, chronologically ordered output file. On a distributed system, the input audit files originate from different workstations. Therefore, when issued from the audit administration server, the auditreduce command treats the distributed system as if it were one workstation. This treatment simplifies audit administration. Coupled with backup audit partitions, the distributed system is robust in the face of workstation failures.

The auditreduce command also includes options for selecting sets of records to examine. For instance, records from the past 24 hours can be selected to generate a daily report; all records generated by a specific user can be selected to examine that user's activities; or all records caused by a specific event type can be selected to see how often that type occurs.

Selecting Records from the Audit Trail

Options to the auditreduce(1M) command allow you to select audit records based on file characteristics and record characteristics, as shown in the following table.

Table 3-1 Some Options to the auditreduce Command

Characteristic 

Option(s) 

Time, date (start, finish) 

-d, -a, -f 

Host (workstation) ID 

-M, -h, -S 

Audit class 

-c 

Audit event 

-m 

Audit session ID 

 

Audit User ID - AUID 

-u 

Effective and Real User ID - EUID, RUID 

-e, -r 

Effective and Real Group ID - EGID, RGID 

-f, -g 

Process ID - PID 

-j 

Sensitivity label 

-s 

Information label 

-i 

Filename 

filename

Uppercase options select operations or parameters for files, and lowercase options select parameters for records. When piped through praudit, audit files processed by the auditreduce command are readable. Otherwise, they remain in binary format.

The merging and selecting functions of auditreduce are logically independent. The auditreduce command selects messages from the input files as the records are read, before the files are merged and written to disk.

Using the auditreduce and praudit Commands

This section describes a few common uses of auditreduce and praudit to select and manage data. See the auditreduce(1M) man page for more examples.

Prerequisites for running the auditreduce and praudit commands:

To access the audit trail for a distributed system:

To Read a Closed Audit File

The praudit command enables you to display audit records interactively and create very basic reports. For multiple files, the input is piped from auditreduce.

    Specify the audit file as the file argument to the praudit command.


    $ praudit 19970401000000.19970601000000.grebe

    This displays audit token per line to standard output.

    Specify the audit file as the file argument to the praudit -l command.


    $ praudit -l 19970401000000.19970601000000.grebe

    This displays one audit record per line to standard output.

To Read a Current Audit File

    Use the tail(1) command to see what is currently being written to an active audit file. 


    $ praudit | tail -40 19970401000000.not_terminated.grebe

    This displays the latest 40 tokens that were recorded to standard output.

To Display Several Audit Files as One Audit File

    To display several audit files in chronological order in the terminal window, pipe the output of auditreduce into praudit. 


    $ auditreduce 19970413000000.19970413235959.willet \
19970413000000.19970413235959.grebe | praudit

    To display the whole audit trail in the terminal window, pipe the output of auditreduce into praudit. 


    $ auditreduce | praudit
    Note -

    The auditreduce command without options does not disturb open audit files.

To Print an Audit Log

    Use praudit with a pipe to lp, to send the output of one file to the printer. 


    $ praudit 19970413000000.19970413235959.audubon | lp

    Use auditreduce piped through praudit with a pipe to lp, to send the output of all closed audit files to the printer. 


    $ auditreduce | praudit | lp
    Note -

    In the Trusted Solaris environment, the printer must be able to accept admin_high print jobs.

To Display User Activity on a Selected Date

    Use the -d option to the auditreduce command to see audit information collected during a specified 24-hour period.

    In the following example, the security administrator checks to see when a user named doris logged in and logged out on April 13, 1997, by requesting the lo message class. The short-form date is in the form yymmdd. (The long form is described in the auditreduce(1M) man page.) 


    $ auditreduce -d 970413 --u doris -c lo | praudit

To Print User Activity on a Selected Date

    Use the auditreduce command with a pipe through praudit to lp, to send selected output to a printer.

    Note -

    In the Trusted Solaris environment, the printer must be able to accept admin_high print jobs.


    $ auditreduce -d 970413 -u doris -c lo | praudit | lp

To Copy Login/Logout Messages to a Single File

In this example, login/logout messages for a particular day are summarized in a file. The target file is written in a directory other than the normal audit root. 


$ auditreduce -c lo -d 970413 \
-O /usr/audit_summary/logins

The -O option creates an audit file in the /usr/audit_summary directory. The file name has 14-character timestamps for both start-time and end-time, and the suffix logins: 

/usr/audit_summary/19970413000000.19970413235959.logins

To Display Audit Records Created Before or After a Designated Date

The date-time options -b and -a allow specifying records before or after a particular day and time. A day begins at yyyymmdd00:00:00 and ends at yyyymmdd23:59:59. The six parameters of a day are: year, month, day, hour, minute, and second. The digits (19) of the year are assumed and need not be specified.

The auditreduce -a command with the date shown in the following screen example sends all audit records created after midnight on July 15, 1997 through praudit to standard output. 


$ auditreduce -a 97071500:00:00 | praudit

If -a is not specified, auditreduce defaults to 00:00:00, January 1, 1970.

The auditreduce -b command with the same date shown above sends all audit records created before midnight on July 15, 1997 through praudit to standard output. 


$ auditreduce -b 97071500:00:00 | praudit

If -b is not specified, auditreduce defaults to the current time of day (GMT). The -d option selects a particular 24-hour period, as shown in "To Copy Login/Logout Messages to a Single File ".

To Find an Audit Event

    Use the message type selection for auditreduce (-m option) to find a particular audit event.

    The -m option accepts either numeric message identifiers or AUE_xxxxx event names. The screen example below finds all kernel-level login events in the audit trail and displays them to standard output.


    $ auditreduce -m AUE_LOGIN | praudit

    The auditreduce command rejects an incorrect format, but does not describe the correct format.

To Combine Selected Audit Files

Although auditreduce can do this type of combination and deletion automatically (see the -C and -D options in the auditreduce(1M) man page), it is often easier to select the files manually (perhaps with find) and use the auditreduce command to combine just the named set of files.

  1. List the audit files as arguments to the auditreduce command.

    In the following example, a recurring job that starts a bit before midnight merges the audit files from two days before. The final time on the file is the time the job ended, here just before midnight, Greenwich Mean Time (GMT).


    $ auditreduce 19970413000000.19970413235959.grebe \
19970413000000.19970413235959.willet \
19970413000000.19970413235959.sora
$ ls *audubon 19970413000000.19970414235959.audubon

  2. Delete the input files and move the output file to the audit root directory on the administration server.

    In this example, the auditreduce(1M) command was run on the audit administration server, audubon, and then placed in its audit root directory so that future calls to auditreduce locate the file. 


    $ rm /etc/security/tsol/grebe/files/19970413000000.19970413235959.grebe
$ rm /etc/security/tsol/willet/files/19970413000000.19970413235959.willet
$ rm /etc/security/tsol/sora/files/19970413000000.19970413235959.sora
$ mv 19970413000000.19970414235959.audubon /etc/security/audit/audubon/files/

To Reduce Audit Files

The auditreduce program can also reduce the number of records in its output file by eliminating the less interesting ones as it combines the input files.

You might use auditreduce to eliminate all except the login/logout events in audit files over a month old, assuming that if you needed to retrieve the complete audit trail you could recover it from backup tapes. The following example selects just the audit records from April 1997. 


$ auditreduce -m AUE_LOGIN -a 19970401000000 \
-b 19970501000000 \
-O /usr/audit_summary/logins_april97

The output is a smaller file containing just the April 1997 login/logout records. Note that the end-time stamp is the date (in GMT) that the command was run (June 1, 1997), not the last date of the merged records. You specified the file suffix, logins_april97, on the command line with the directory name. 

/usr/audit_summary/19970401000000.19970601000000.logins_april97

To Change the praudit Field Separator to a Tab

When the praudit command displays an audit token, it separates the data fields with commas. However, if a field (such as a time stamp) contains a comma, this cannot be distinguished from a field-separating comma.

    Press the Tab key as the value of the -d option to praudit(1M).


    $ praudit -d"<press Tab key>" 19970413120429.19970413180433.grebe

    There is no space between the -d option and the delimiter. Surround the delimiter with double quotes. The delimiter can be up to four characters long.

To Change the praudit Token Separator to a Tab

Audit tokens are separated by newlines by default. When audit records are printed one per line using the -l option, the audit token separator is the same as the audit field separator. In the following screen example, the audit tokens are separated by tabs, as are the audit fields. 


$ praudit -l -d"<press Tab key>" 19970413120429.19970413180433.grebe

To Perform Selections Using a praudit Script

To accomplish more sophisticated display and reports, process the output from praudit with sed or awk, or write programs to interpret and process the binary audit records.

It is sometimes useful to manipulate praudit output as lines of text; for example to perform selections that cannot be done with auditreduce. A simple shell script can process the output of praudit. The following example is called praudit_grep: 

#!/bin/sh
praudit | sed -e '1,2d' -e '$s/^file.*$//' -e 's/^header/^aheader/' \\
| tr '\\012\\001' '\\002\\012' \\
| grep "$1" \\
| tr '\\002' '\\012'

The example script marks the header tokens by prefixing them with Control-A. (Note that the ^a is Control-A, not the two characters ^ and a. Prefixing is necessary to distinguish them from the string header that might appear as text.) The script then combines all the tokens for a record onto one line while preserving the line breaks as Control-A, runs the grep command, and restores the original newlines.

To run the script in the Trusted Solaris environment, the following conditions must be met:

Audit Files Backup and Recovery

Audit files occupy disk space. The disk space needs to be freed up in order to make space for subsequent audit files. By default, the role oper handles audit file backup via the profile Media Backup and the role admin handles audit file restore via the profile Media Restore.

To Back Up Audit Files

  1. As the role oper in an admin_high workspace, go to the workstation's audit files directory.


    $ cd /etc/security/audit/workstation_name[.n]/files

  2. Allocate, at the label admin_high, the tape drive that you are going to use for backup.

    If you are unfamiliar with device allocation, see "To Allocate and Deallocate Devices".

  3. Use the tar(1) command to copy the completed audit files and their Trusted Solaris security attributes, such as the label, to the tape.

    For example,


    $ tar cvT \
/etc/security/audit/workstation_name/files/19980413120429.19980413180433.grebe \
/etc/security/audit/workstation_name/files/19980502120429.19980502180433.grebe \
/etc/security/audit/workstation_name/files/19980513120429.19980513180433.grebe

  4. Deallocate the tape drive when finished, remove the tape, and label it admin_high.

  5. At the same time, in an admin_low workspace, back up system files that capture information about the users, labels, roles, and execution profiles on the workstation.

    Store the audit tapes with the current system information tape(s).

  6. As root, at label admin_high, remove the audit files that have been backed up.

    For example,


    $ rm \
/etc/security/audit/workstation_name/files/19980413120429.19980413180433.grebe \
/etc/security/audit/workstation_name/files/19980502120429.19980502180433.grebe \
/etc/security/audit/workstation_name/files/19980513120429.19980513180433.grebe

To Restore Audit Files

  1. As role admin, in an admin_high workspace, go to the directory where the audit files are to be placed.


    $ cd /etc/security/audit/workstation_name[.n]/reports

  2. Allocate, at the label admin_high, the tape drive that you are going to use to restore the files.

    If you are unfamiliar with device allocation, see "To Allocate and Deallocate Devices".

  3. Use the tar(1) command to copy the audit files and their Trusted Solaris security attributes, such as the label, from the tape.

    For example,


    $ tar xvT \
/etc/security/audit/workstation_name/files/19980513120429.19980513180433.grebe

  4. Deallocate the tape drive when finished and follow the Device Manager's instructions.

  5. Use the restored audit files.

    You may need to restore or refer to other system information from the audit backup's associated system backup.

  6. As role admin, at label admin_high, remove the audit files when you are done.


    $ rm \
/etc/security/audit/workstation_name/reports/19980513120429.19980513180433.grebe