Troubleshooting the Audit Service (Tasks)

This section covers various auditing error messages, preferences, and the auditing that is provided by other tools. These procedures can help you record required audit events and debug audit problems.

Troubleshooting the Audit Service (Task Map)

The following task map points to procedures for troubleshooting auditing.

Problem	Solution	For Instructions
Why are audit records not being logged when I have configured auditing?	Troubleshoot the audit service.	How to Determine That Auditing Is Running
How can I reduce the amount of audit information that is being collected?	Audit just the events that you want to audit.	How to Lessen the Volume of Audit Records That Are Produced
How can I audit everything that a user does on the system?	Audit one or more users for every command.	How to Audit All Commands by Users
How can I change the audit events that are being recorded and have the change affect existing sessions?	Update a user's preselection mask.	How to Update the Preselection Mask of Logged In Users
How can I locate modifications to particular files?	Audit file modifications, then use the `auditreduce` command to find particular files.	How to Find Audit Records of Changes to Specific Files
How can I reduce the size of my audit files?	Limit the size of the binary audit file.	How to Limit the Size of Binary Audit Files
How can I use less file system space for audit files?	Use ZFS quotas and compression.	How to Compress Audit Files on a Dedicated File System
How can I remove audit events from the `audit_event` file?	Correctly update the `audit_event` file.	How to Prevent the Auditing of Specific Events
How can I audit all logins to an Oracle Solaris system?	Audit logins from any system.	How to Audit Logins From Other Operating Systems
Why are auditing records not being kept for my FTP transfers?	Use the appropriate auditing tool for utilities that generate their own logs.	How to Audit FTP and SFTP File Transfers

How to Determine That Auditing Is Running

Auditing is enabled by default. If you believe that auditing has not been disabled, but no audit records are being sent to the active plugin, use the following procedure to isolate the issue.

Before You Begin

To modify a system file, you must be in the root role. To configure auditing, you must be assigned the Audit Configuration rights profile.

Determine that auditing is running.

Use any of the following methods:

Verify the current audit condition.
The following listing indicates that auditing is not running:
```
# auditconfig -getcond
audit condition = noaudit
```
The following listing indicates that auditing is running:
```
# auditconfig -getcond
audit condition = auditing
```

Verify that the audit service is running.

The following listing indicates that auditing is not running:

# svcs -x auditd
svc:/system/auditd:default (Solaris audit daemon)
 State: disabled since Sun Oct 10 10:10:10 2010
Reason: Disabled by an administrator.
   See: http://sun.com/msg/SMF-8000-05
   See: auditd(1M)
   See: audit(1M)
   See: auditconfig(1M)
   See: audit_flags(5)
   See: audit_binfile(5)
   See: audit_syslog(5)
   See: audit_remote(5)
   See: /var/svc/log/system-auditd:default.log
Impact: This service is not running.

The following listing indicates that the audit service is running:

# svcs auditd
STATE          STIME    FMRI
online         10:10:10 svc:/system/auditd:default

If the audit service is not running, enable it. For the procedure, see How to Enable the Audit Service.

Verify that at least one plugin is active.
```
# audit -v
```
If no plugin is active, make one active.
```
# auditconfig -setplugin audit_binfile active
```
If you created a customized audit class, verify that you assigned events to the class.
For example, the following list of flags contains the pf class, which Oracle Solaris software did not deliver:
```
# auditconfig -getflags
active user default audit flags = pf,lo(0x0100000000000000,00x0100000000001000)
configured user default audit flags = pf,lo(0x0100000000000000,00x0100000000001000)
```
For a description of creating the pf class, see How to Add an Audit Class.
1. Verify that the class is defined in the audit_class file.
  The audit class must be defined, and its mask must be unique.
```
# grep pf /etc/security/audit_classVerify class exists
0x0100000000000000:pf:profile
# grep 0x08000000 /etc/security/audit_classEnsure mask is unique
0x0100000000000000:pf:profile
```
  Replace a mask that is not unique. If the class is not defined, define it. Otherwise, run the auditconfig -setflags command with valid values to reset the current flags.
2. Verify that events have been assigned to the class.
  Use one of the following methods:
```
# auditconfig -lsevent | egrep " pf|,pf|pf,"
AUE_PFEXEC      116 pf execve(2) with pfexec enabled
```
```
# auditrecord -c pf
List of audit events assigned to pf class
```
  If events are not assigned to the class, assign the appropriate events to this class.
If the previous steps did not indicate a problem, review your email and the log files.
1. Read the email sent to the audit_warn alias.
  The audit_warn script sends alert messages to the audit_warn email alias. In the absence of a correctly configured alias, the messages are sent to the root account.
2. Review the log files for the audit service.
  The output from the svcs -s auditd command lists the full path to the audit logs that the audit service produces. For an example, see the listing in Step 1.
3. Review the system log files.
  The audit_warn script writes daemon.alert messages to the /var/log/syslog file.
  The /var/adm/messages file might contain information.
After you locate and fix the problems, enable or restart the audit service.
```
# audit -s
```

How to Lessen the Volume of Audit Records That Are Produced

After you have determined which events must be audited at your site, use the following suggestions to create manageable audit files.

Before You Begin

To preselect audit classes and set audit policy, you must be assigned the Audit Configuration rights profile. To modify system files and to assign audit flags to users, roles, and rights profiles, you must be in the root role.

Use the default audit policy.
Specifically, avoid adding events and audit tokens to the audit trail. The following policies grow the size of the audit trail.
- arge policy – Adds environment variables to execv audit events.
- argv policy – Adds command parameters to execv audit events.
- public policy – If file events are being audited, adds an event to the audit trail every time an auditable event happens to a public object. File classes include fa, fc, fd, fm, fr, fw, and cl. For the definition of a public file, see Audit Terminology and Concepts.
- path policy – Adds a path token to audit events that include an optional path token.
- group policy – Adds a group token to audit events that include an optional newgroups token.
- seq policy – Adds a sequence token to every audit event.
- trail policy – Adds a trailer token to every audit event.
- windata_down policy – On a system that is configured with Trusted Extensions, adds events when information in a labeled window is downgraded.
- windata_up policy – On a system that is configured with Trusted Extensions, adds events when information in a labeled window is upgraded.
- zonename policy – Adds the zone name to every audit event. If the global zone is the only configured zone, adds the string zone, global to every audit event.
The following audit record shows the use of the ls command. The ex class is being audited and the default policy is in use:
```
header,129,2,AUE_EXECVE,,mach1,2010-10-14 11:39:22.480 -07:00
path,/usr/bin/ls
attribute,100555,root,bin,21,320271,18446744073709551615
subject,jdoe,root,root,root,root,2404,50036632,82 0 mach1
return,success,0
```
The following is the same record when all policies are turned on:
```
header,1578,2,AUE_EXECVE,,mach1,2010-10-14 11:45:46.658 -07:00
path,/usr/bin/ls
attribute,100555,root,bin,21,320271,18446744073709551615
exec_args,2,ls,/etc/security
exec_env,49,MANPATH=/usr/share/man,USER=jdoe,GDM_KEYBOARD_LAYOUT=us,EDITOR=gedit,
  LANG=en_US.UTF-8,GDM_LANG=en_US.UTF-8,PS1=#,GDMSESSION=gnome,SESSIONTYPE=1,SHLVL=2,
  HOME=/home/jdoe,LOGNAME=jdoe,G_FILENAME_ENCODING=@locale,UTF-8, PRINTER=example-dbl,
...
path,/lib/ld.so.1
attribute,100755,root,bin,21,393073,18446744073709551615
subject,jdoe,root,root,root,root,2424,50036632,82 0 mach1
group,root,other,bin,sys,adm,uucp,mail,tty,lp,nuucp,daemon
return,success,0
zone,global
sequence,197
trailer,1578
```
Use the audit_syslog plugin to send some audit events to syslog.
And do not send those audit events to the audit_binfile or audit_remote plugin. This strategy works only if you are not required to keep binary records of the audit events that you send to the syslog logs.
Set fewer system-wide audit flags and audit individual users.
Reduce the amount of auditing for all users by reducing the number of audit classes that are audited system-wide.
Use the audit_flags keyword to the roleadd, rolemod, useradd, and usermod commands to audit events for specific users and roles. For examples, see Example 28-18 and the usermod(1M) man page.
Use the always_audit and never_audit properties of the profiles command to audit events for specific rights profiles. For information, see the profiles(1) man page.

Note - Like other security attributes, audit flags are affected by search order. For more information, see Order of Search for Assigned Security Attributes.

Create your own customized audit class.

You can create audit classes at your site. Into these classes, put only those audit events that you need to monitor. For the procedure, see How to Add an Audit Class.

Caution - If you modify existing audit class assignments, your modifications might be kept when you upgrade to a newer version of the Oracle Solaris OS. However, the newer version of the file from Oracle Solaris might include changes that you must manually incorporate into the installation. Carefully review the installation logs. For more information, see the description of preserve=renamenew in the pkg(5) man page.

How to Audit All Commands by Users

As part of site security policy, some sites require audit records of all commands that are run by the root account and administrative roles. Some sites can require audit records of all commands by all users. Additionally, sites can require that the command arguments and environment be recorded.

Before You Begin

To preselect audit classes and set audit policy, you must be assigned the Audit Configuration rights profile. To assign audit flags to users, roles, and rights profiles, you must be in the root role.

Become an administrator with the required security attributes.
For more information, see How to Obtain Administrative Rights.

Audit the lo and ex classes.

The ex class audits all calls to the exec() and execve() functions.

The lo class audits logins, logouts, and screen locks. The following output lists all the events in the ex and lo classes.

% auditconfig -lsevent | grep " lo "
AUE_login                       6152 lo login - local
AUE_logout                      6153 lo logout
AUE_telnet                      6154 lo login - telnet
AUE_rlogin                      6155 lo login - rlogin
AUE_rshd                        6158 lo rsh access
AUE_su                          6159 lo su
AUE_rexecd                      6162 lo rexecd
AUE_passwd                      6163 lo passwd
AUE_rexd                        6164 lo rexd
AUE_ftpd                        6165 lo ftp access
AUE_ftpd_logout                 6171 lo ftp logout
AUE_ssh                         6172 lo login - ssh
AUE_role_login                  6173 lo role login
AUE_newgrp_login                6212 lo newgrp login
AUE_admin_authenticate          6213 lo admin login
AUE_screenlock                  6221 lo screenlock - lock
AUE_screenunlock                6222 lo screenlock - unlock
AUE_zlogin                      6227 lo login - zlogin
AUE_su_logout                   6228 lo su logout
AUE_role_logout                 6229 lo role logout
AUE_smbd_session                6244 lo smbd(1m) session setup
AUE_smbd_logoff                 6245 lo smbd(1m) session logoff
AUE_ClientConnect               9101 lo client connection to x server
AUE_ClientDisconnect            9102 lo client disconn. from x server
% auditconfig -lsevent | egrep " ex |,ex |ex,"
AUE_EXECVE                        23 ex,ps execve(2)

To audit these classes for administrative roles, modify the roles' security attributes.
In the following example, root is a role. The site has created three roles, sysadm, auditadm, and netadm. All roles are audited for the success and failure of events in the ex and lo classes.
```
# rolemod -K audit_flags=lo,ex:no root
# rolemod -K audit_flags=lo,ex:no sysadm
# rolemod -K audit_flags=lo,ex:no auditadm
# rolemod -K audit_flags=lo,ex:no netadm
```

To audit these classes for all users, set the system-wide flags.

# auditconfig -setflags lo,ex

The output appears similar to the following:

header,129,2,AUE_EXECVE,,mach1,2010-10-14 12:17:12.616 -07:00
path,/usr/bin/ls
attribute,100555,root,bin,21,320271,18446744073709551615
subject,jdoe,root,root,root,root,2486,50036632,82 0 mach1
return,success,0

To record the arguments to commands, add the argv policy.

# auditconfig -setpolicy +argv

The exec_args token records the command arguments:

header,151,2,AUE_EXECVE,,mach1,2010-10-14 12:26:17.373 -07:00
path,/usr/bin/ls
attribute,100555,root,bin,21,320271,18446744073709551615
exec_args,2,ls,/etc/security
subject,jdoe,root,root,root,root,2494,50036632,82 0 mach1
return,success,0

To record the environment in which the command is run, add the arge policy.

# auditconfig -setpolicy +arge

The exec_env token records the command environment:

header,1460,2,AUE_EXECVE,,mach1,2010-10-14 12:29:39.679 -07:00
path,/usr/bin/ls
attribute,100555,root,bin,21,320271,18446744073709551615
exec_args,2,ls,/etc/security
exec_env,49,MANPATH=/usr/share/man,USER=jdoe,GDM_KEYBOARD_LAYOUT=us,EDITOR=gedit,
LANG=en_US.UTF-8,GDM_LANG=en_US.UTF-8,PS1=#,GDMSESSION=gnome,SESSIONTYPE=1,SHLVL=2,
HOME=/home/jdoe,LOGNAME=jdoe,G_FILENAME_ENCODING=@locale,UTF-8,
PRINTER=example-dbl,...,_=/usr/bin/ls
subject,jdoe,root,root,root,root,2502,50036632,82 0 mach1
return,success,0

How to Find Audit Records of Changes to Specific Files

If your goal is to log file writes against a limited number of files, such as /etc/passwd and the files in the /etc/default directory, you use the auditreduce command to locate the files.

Before You Begin

You must be assigned the Audit Configuration rights profile to use the auditconfig command. You must be assigned the Audit Review rights profile to use the auditreduce command. To assign audit flags to users and roles, you must be in the root role.

Become an administrator with the required security attributes.
For more information, see How to Obtain Administrative Rights.

Audit the fw class.

Adding the class to the audit flags of a user or role generates fewer records than adding the class to the system-wide audit preselection mask. Perform one of the following steps:

Add the fw class to specific roles.

# rolemod -K audit_flags=fw:no root
# rolemod -K audit_flags=fw:no sysadm
# rolemod -K audit_flags=fw:no auditadm
# rolemod -K audit_flags=fw:no netadm

Add the fw class to the system-wide flags.

# auditconfig -getflags
active user default audit flags = lo(0x1000,0x1000)
configured user default audit flags = lo(0x1000,0x1000)
# auditconfig -setflags lo,fw
user default audit flags = lo,fw(0x1002,0x1002)

Or, audit successful file-writes.

Auditing successes generates fewer records than auditing failures and successes. Perform one of the following steps:

Add the +fw flag to specific roles.

# rolemod -K audit_flags=+fw:no root
# rolemod -K audit_flags=+fw:no sysadm
# rolemod -K audit_flags=+fw:no auditadm
# rolemod -K audit_flags=+fw:no netadm

Add the +fw flag to the system-wide flags.

# auditconfig -getflags
active user default audit flags = lo(0x1000,0x1000)
configured user default audit flags = lo(0x1000,0x1000)
# auditconfig -setflags lo,+fw
user default audit flags = lo,+fw(0x1002,0x1000)

If the system-wide flags are auditing for success and for failure, set exceptions for specific users and roles.

# auditconfig -getflags
active user default audit flags = lo,fw(0x1002,0x1002)
configured user default audit flags = lo,fw(0x1002,0x1002)
# rolemod -K audit_flags=^-fw:no root
# rolemod -K audit_flags=^-fw:no sysadm
# rolemod -K audit_flags=^-fw:no auditadm
# rolemod -K audit_flags=^-fw:no netadm

The system-wide flags are still unchanged, but the preselection mask for these four roles is changed.

# auditconfig -getflags
active user default audit flags = lo,fw(0x1002,0x1000)
configured user default audit flags = lo,fw(0x1002,0x1000)

To find the audit records for specific files, use the auditreduce command.
```
# auditreduce -o file=/etc/passwd,/etc/default -O filechg
```
The auditreduce command searches the audit trail for all instances of the file argument. The command creates a binary file with the suffix filechg which contains all records that include the pathnames of the files of interest. See the auditreduce(1M) man page for the syntax of the -o file=pathname option.
To read the filechg file, use the praudit command.
```
# praudit *filechg
```

How to Update the Preselection Mask of Logged In Users

You want the users who are already logged in to be audited for changes to the system-wide audit preselection mask.

Before You Begin

You must be assigned the Audit Configuration rights profile. To terminate user sessions, you must be assigned the Process Management rights profile.

Become an administrator with the required security attributes.
For more information, see How to Obtain Administrative Rights.
Update the preselection mask of users who are already logged in.
You have two options. You can terminate the existing sessions or use the auditconfig command to update the preselection masks.
- Terminate the users' existing sessions.
  Users can log out and log back in. Or, you in a role that is assigned the Process Management rights profile can manually terminate (kill) active sessions. The new sessions will inherit the new preselection mask. However, terminating users could be impractical.
- Dynamically change each logged-in user's preselection mask.
  In a role that includes the Audit Configuration rights profile, assume that you changed the system-wide audit preselection mask from lo to lo,ex.
```
# auditconfig -setflags lo,ex
```
  1. List the regular users who are logged in and their process IDs.
```
# who -a
jdoe  - vt/2         Jan 25 07:56  4:10   1597   (:0)
jdoe  + pts/1        Jan 25 10:10   .     1706   (:0.0)
...
jdoe  + pts/2        Jan 25 11:36  3:41   1706   (:0.0)
```
  2. For later comparison, display each user's preselection mask.
```
# auditconfig -getpinfo 1706
audit id = jdoe(1234)
process preselection mask = lo(0x1000,0x1000)
terminal id (maj,min,host) = 9426,65559,mach1(192.168.123.234)
audit session id = 103203403
```
  3. Modify the user's preselection mask.
```
# auditconfig -setumask jdoe lo,ex /* for this user */
```
```
# auditconfig -setsmask 103203403 lo,ex /* for this session */
```
```
# auditconfig -setpmask 1706 lo,ex /* for this process */
```
  4. Verify that the preselection mask for the user has changed.
    For example, check a process that existed before you changed the mask.
```
# auditconfig -getpinfo 1706
audit id = jdoe(1234)
process preselection mask = ex,lo(0x40001000,0x40001000) 
terminal id (maj,min,host) = 9426,65559,mach1(192.168.123.234)
audit session id = 103203403
```

How to Prevent the Auditing of Specific Events

For maintenance purposes, sometimes a site wants to prevent events from being audited.

Before You Begin

You must be in the root role.

Change the class of the event to the no class.
For example, events 26 and 27 belong to the pm class.
```
## audit_event file
...
25:AUE_VFORK:vfork(2):ps
26:AUE_SETGROUPS:setgroups(2):pm
27:AUE_SETPGRP:setpgrp(2):pm
28:AUE_SWAPON:swapon(2):no
...
```
Change these events to the no class.
```
## audit_event file
...
25:AUE_VFORK:vfork(2):ps
26:AUE_SETGROUPS:setgroups(2):no
27:AUE_SETPGRP:setpgrp(2):no
28:AUE_SWAPON:swapon(2):no
...
```
If the pm class is currently being audited, existing sessions will still audit events 26 and 27. To stop these events from being audited, you must update the users' preselection masks by following the instructions in How to Update the Preselection Mask of Logged In Users.

Caution - Never comment out events in the audit_event file. This file is used by the praudit command to read binary audit files. Archived audit files might contain events that are listed in the file.

Refresh the kernel events.

# auditconfig -conf
Configured 283 kernel events.

How to Limit the Size of Binary Audit Files

Binary audit files grow without limit. For ease of archiving and searching, you might want to limit the size. You can also create smaller binary files from the original file.

Before You Begin

You must be assigned the Audit Configuration rights profile to set the p_fsize attribute. You must be assigned the Audit Review rights profile to use the auditreduce command.

Become an administrator with the required security attributes.
For more information, see How to Obtain Administrative Rights.
Use the p_fsize attribute to limit the size of individual binary audit files.
For a description of the p_fsize attribute, see the OBJECT ATTRIBUTES section of the audit_binfile(5) man page.
For an example, see Example 28-14.
Use the auditreduce command to select records and write those records to a smaller file for further analysis.
The auditreduce -lowercase options find specific records.
The auditreduce -Uppercase options write your selections to a file. For more information, see the auditreduce(1M) man page.

How to Compress Audit Files on a Dedicated File System

Audit files can grow large. You can set an upper limit to the size of a file, as shown in Example 28-14. In this procedure, you use compression to reduce the size.

Before You Begin

You must be assigned the ZFS File System Management and ZFS Storage Management rights profiles. The latter profile enables you to create storage pools.

Become an administrator with the required security attributes.
For more information, see How to Obtain Administrative Rights.
Dedicate a ZFS file system for audit files.
For the procedure, see How to Create ZFS File Systems for Audit Files.
Compress the ZFS storage pool by using one of the following options.
With both options, the audit file system is compressed. After the audit service is refreshed, the compression ratio is displayed.
To set compression, use the zfs set compression=on dataset command. In the following examples, the ZFS pool auditp/auditf is the dataset.
- Use the default compression algorithm.
```
# zfs set compression=on auditp/auditf
# audit -s
# zfs get compressratio auditp/auditf
NAME           PROPERTY       VALUE  SOURCE
auditp/auditf  compressratio  4.54x  -
```
- Use a higher compression algorithm.
```
# zfs set compression=gzip-9 auditp/auditf
# zfs get compression auditp/auditf
NAME           PROPERTY     VALUE     SOURCE
auditp/auditf  compression  gzip-9    local
# audit -s
# zfs get compressratio auditp/auditf
NAME           PROPERTY       VALUE  SOURCE
auditp/auditf  compressratio  16.89x  -
```
  The gzip-9 compression algorithm results in files that occupy one-third less space than the default compression algorithm, lzjb. For more information, see Chapter 6, Managing Oracle Solaris ZFS File Systems, in Oracle Solaris Administration: ZFS File Systems.

How to Audit Logins From Other Operating Systems

The Oracle Solaris OS can audit all logins, independent of source.

Before You Begin

You must be assigned the Audit Configuration rights profile.

Become an administrator with the required security attributes.
For more information, see How to Obtain Administrative Rights.

Audit the lo class for attributable events and non-attributable events.

This class audits logins, logouts, and screen locks. These classes are audited by default.

# auditconfig -getflags
active user default audit flags = lo(0x1000,0x1000)
configured user default audit flags = lo(0x1000,0x1000)
# auditconfig -getnaflags
active non-attributable audit flags = lo(0x1000,0x1000)
configured non-attributable audit flags = lo(0x1000,0x1000)

If the values have been changed, add the lo flag.

# auditconfig -getflags
active user default audit flags = as,st(0x20800,0x20800)
configured user default audit flags = as,st(0x20800,0x20800)
# auditconfig -setflags lo,as,st
user default audit flags = as,lo,st(0x21800,0x21800)
# auditconfig -getnaflags
active non-attributable audit flags = na(0x400,0x400)
configured non-attributable audit flags = na(0x400,0x400)
# auditconfig -setnaflags lo,na
non-attributable audit flags = lo,na(0x1400,0x1400)

Note - To audit ssh logins, your system must be running the ssh daemon from Oracle Solaris. This daemon is modified for the audit service on an Oracle Solaris system. For more information, see Secure Shell and the OpenSSH Project.

How to Audit FTP and SFTP File Transfers

The FTP service creates logs of its file transfers. The SFTP service, which runs under the ssh protocol, can be audited by preselecting the ft audit class. Logins to both services can be audited.

Before You Begin

You must be assigned the Audit Configuration rights profile.

Become an administrator with the required security attributes.
For more information, see How to Obtain Administrative Rights.
To log commands and file transfers of the FTP service, see the proftpd(8) man page.
For the available logging options, read the “Logging Capabilities” section. In particular, the log commands and log transfers options might provide useful logs.

To log sftp access and file transfers, audit the ft class.

The ft class includes the following SFTP transactions:

% auditrecord -c ft
file transfer: chmod ...
file transfer: chown ...
file transfer: download ...
file transfer: mkdir ...
file transfer: upload ...
file transfer: remove ...
file transfer: rename ...
file transfer: rmdir ...
file transfer: session start ...
file transfer: session end ...
file transfer: symlink ...
file transfer: utimes

To record access to the FTP server, audit the lo class.

As the following output indicates, logging in to and out of the ftpd daemon generates audit records.

% auditrecord -c lo | more
...
in.ftpd
  program     /usr/sbin/in.ftpd    See ftp access
  event ID    6165                 AUE_ftpd
  class       lo                   (0x0000000000001000)
      subject
      [text]                       error message
      return

in.ftpd
  program     /usr/sbin/in.ftpd    See ftp logout
  event ID    6171                 AUE_ftpd_logout
  class       lo                   (0x0000000000001000)
      subject
      return
...

Skip Navigation Links
Exit Print View
	Oracle Solaris Administration: Security Services Oracle Solaris 11 Information Library