Troubleshooting Solaris Auditing (Task Map)

The following task map points to procedures for troubleshooting Solaris auditing.

How to Determine That Solaris Auditing Is Running

If you believe that auditing has been activated, but no audit records are in your primary audit directory, try the following.

1. Determine that auditing is running.

   • Verify that the c2audit kernel module is loaded.

     # modinfo | grep c2audit

     No listing indicates that auditing is not running. The following listing indicates that auditing is running:

     40 132ce90 14230 186 1 c2audit (C2 system call)

   • Verify that the audit daemon is running.

     Verify the status of the auditd service. The following listing indicates that auditing is not running:

     # svcs -x auditd svc:/system/auditd:default (Solaris audit daemon) State: disabled since Fri Aug 14 19:02:35 2009 Reason: Disabled by an administrator. See: http://sun.com/msg/SMF-8000-05 See: auditd(1M) See: audit(1M) Impact: This service is not running.

     The following listing indicates that the audit service is running:

     # svcs auditd STATE STIME FMRI online 10:10:10 svc:/system/auditd:default

   • Verify the current audit condition.

     The following listing indicates that auditing is not running:

     # auditconfig -getcond auditconfig: auditon(2) failed. auditconfig: error = Operation not supported(48)

     The following listing indicates that auditing is running:

     # auditconfig -getcond audit condition = auditing

   If the audit service is not running, enable it. For the procedure, see How to Enable the Audit Service.

2. Verify the syntax of the audit_control file.

   # audit -v /etc/security/audit_control audit: audit_control must have either a valid "dir:" entry or a valid "plugin:" entry with "p_dir:" specified.

   Correct the errors. The message syntax ok indicates that the file is syntactically correct.

3. Verify that the audit_control file has valid values for the flags and naflags keywords.

   # grep flags /etc/security/audit_control flags:lo naflags:na,lp

   Supply valid values if the audit_control file has invalid values. In the preceding example, lp is an invalid class.

4. Verify that the audit_user file has valid values for every user.

   # tail audit_user ... # User Level Audit User File # # File Format # # username:always:never # root:lo:no admin:lp:no

   Supply valid values if the audit_user file contains invalid values. In the preceding example, lp is an invalid class.

5. If you created a customized audit class, verify that you assigned events to the class.

   For example, the following audit_control file contains a class that Sun did not deliver:

   # grep flags /etc/security/audit_control flags:lo,pf naflags:na,lo

   For a description of creating the pf class, see How to Add an Audit Class.

   1. Verify that the class is defined in the audit_class file.

      The audit class mask must be unique.

      # grep pf /etc/security/audit_class 0x10000000:pf:profile command

      If the class is not defined, define it. Otherwise, remove the class from the audit_control and audit_user files.

   2. Verify that events have been assigned to the class.

      # grep pf /etc/security/audit_event 6180:AUE_prof_cmd:profile command:ua,as,pf

      If events are not assigned to the class, assign the appropriate events to this class.

6. If the previous steps did not indicate a problem, review the system log files, /var/adm/messages and /var/log/syslog.

   1. Locate and fix the problems.

   2. Then, if the audit service is running, restart it.

      # audit -s

   3. If the audit service is not running, enable it.

      For the procedure, see How to Enable the Audit Service.

How to Lessen the Volume of Audit Records That Are Produced

After you have determined which events must be audited at your site, use the following suggestions to create manageable audit files.

1. Use the default audit policy.

   Specifically, avoid adding events and audit tokens to the audit trail. The following policies affect the size of the audit trail.

   • arge policy – Adds environment variables to exec audit events.

   • argv policy – Adds command parameters to exec audit events.

   • public policy – If file events are being audited, adds an event to the audit trail every time an auditable event happens to a public file. File classes include fa, fc, fd, fm, fr, fw, and cl. For the definition of a public file, see Audit Terminology and Concepts.

   • path policy – Adds a path token to audit events that include an optional path token.

   • group policy – Adds a group token to audit events that include an optional newgroups token.

   • seq policy – Adds a sequence token to every audit event.

   • trail policy – Adds a trailer token to every audit event.

   • windata_down policy – On a system that is configured with Trusted Extensions, adds events when information in a labeled window is downgraded.

   • windata_up policy – On a system that is configured with Trusted Extensions, adds events when information in a labeled window is upgraded.

   • zonename policy – Adds the zone name to every audit event. If the global zone is the only configured zone, adds zone, global to every audit event.

   The following audit record shows the use of the ls command. The ex class is being audited and the default policy is in use:

   header,375,2,execve(2),,mach1,2009-08-06 11:19:57.388 -07:00 path,/usr/bin/ls subject,jdoe,root,root,root,root,1401,737,0 0 mach1 return,success,0

   The following is the same record when all policies are turned on:

   header,375,2,execve(2),,mach1,2009-08-06 11:19:57.388 -07:00 path,/usr/bin/ls attribute,100555,root,bin,136,432,0 exec_args,1,ls exec_env,9,HOME=/,HZ=,LANG=C,LOGNAME=root,MAIL=/var/mail/root,PATH=/u sr/sbin:/usr/bin,SHELL=/sbin/sh,TERM=xterm,TZ=US/Pacific path,/lib/ld.so.1 attribute,100755,root,bin,136,4289,0 subject,jdoe,root,root,root,root,1401,737,0 0 mach1 group,root,other,bin,sys,adm,uucp,mail,tty,lp,nuucp,daemon return,success,0 zone,global sequence,313540 trailer,375

2. Use the audit_syslog.so plugin to send some audit events to syslog.

   This strategy works only if you are not required to keep binary records of the audit events that you send to the syslog logs. By using the auditreduce command, you can then strip the binary files of these records, thus reducing the size of the binary files.

3. Use the audit_user file to audit events for specific users and roles.

   Reduce the amount of auditing for all users by reducing the number of audit classes in the audit_control file. In the audit_user file, add audit classes for specific users and roles.

4. Create your own customized audit class.

   You can create audit classes at your site. Into these classes, put all the audit events that you need to monitor. For the procedure, see How to Add an Audit Class.

   ---

   Note –

   If you modify existing audit class assignments, your modifications might be lost when you upgrade to a newer version of the Solaris OS. Carefully review the install logs.

   ---

How to Audit All Commands by Users

As part of site security policy, some sites require audit records of all commands that are run by the root user or by administrative roles. Some sites also require audit records of all commands that are run by users.

1. Audit the lo and ex classes.

   The ex class audits all calls to the exec() and execve() functions. The lo class audits logins, logouts, and screen locks. The following ouput lists all the events in the ex and lo classes.

   7:AUE_EXEC:exec(2):ps,ex 23:AUE_EXECVE:execve(2):ps,ex ... 6152:AUE_login:login - local:lo 6153:AUE_logout:logout:lo 6154:AUE_telnet:login - telnet:lo 6155:AUE_rlogin:login - rlogin:lo 6158:AUE_rshd:rsh access:lo 6159:AUE_su:su:lo 6162:AUE_rexecd:rexecd:lo 6163:AUE_passwd:passwd:lo 6164:AUE_rexd:rexd:lo 6165:AUE_ftpd:ftp access:lo 6171:AUE_ftpd_logout:ftp logout:lo 6172:AUE_ssh:login - ssh:lo 6173:AUE_role_login:role login:lo 6212:AUE_newgrp_login:newgrp login:lo 6213:AUE_admin_authenticate:admin login:lo 6221:AUE_screenlock:screenlock - lock:lo 6222:AUE_screenunlock:screenlock - unlock:lo 6227:AUE_zlogin:login - zlogin:lo

   • To audit these classes for administrators, modify the audit_user file.

     In the following example, the site has created three roles, sysadm, auditadm, and netadm. These roles and the root account are audited for the exec and lo classes:

     ## audit_user file root:lo,ex:no sysadm:lo,ex:no auditadm:lo,ex:no netadm:lo,ex:no

   • To audit the lo class for non-attributable events, modify the audit_control file.

     ## audit_control file ... naflags:lo ...

   • To audit these classes for all users, modify the audit_control file.

     ## audit_control file flags:lo,ex naflags:lo ...

     The output appears similar to the following:

     header,375,2,execve(2),,mach1,2009-08-06 11:19:57.388 -07:00 path,/usr/bin/ls subject,jdoe,root,root,root,root,1401,737,0 0 mach1 return,success,0

2. To record the arguments to commands, set the argv policy.

   # auditconfig -setpolicy +argv

   The exec_args token records the command arguments:

   header,375,2,execve(2),,mach1,2009-08-06 11:19:57.388 -07:00 path,/usr/bin/ls exec_args,1,ls subject,jdoe,root,root,root,root,1401,737,0 0 mach1 return,success,0

3. To record the environment in which the command is run, set the arge policy.

   # auditconfig -setpolicy +arge

   The exec_env token records the command environment:

   header,375,2,execve(2),,mach1,2009-08-06 11:19:57.388 -07:00 path,/usr/bin/ls exec_env,9,HOME=/,HZ=,LANG=C,LOGNAME=root,MAIL=/var/mail/root, PATH=/usr/sbin:/usr/bin,SHELL=/sbin/sh,TERM=xterm,TZ=US/Pacific subject,jdoe,root,root,root,root,1401,737,0 0 mach1 return,success,0

4. To record the arguments and the command environment, set both policies.

   # auditconfig -setpolicy +argv

   The output appears similar to the following:

   header,375,2,execve(2),,mach1,2009-08-06 11:19:57.388 -07:00 path,/usr/bin/ls exec_args,1,ls exec_env,9,HOME=/,HZ=,LANG=C,LOGNAME=root,MAIL=/var/mail/root, PATH=/usr/sbin:/usr/bin,SHELL=/sbin/sh,TERM=xterm,TZ=US/Pacific subject,jdoe,root,root,root,root,1401,737,0 0 mach1 return,success,0

How to Find Audit Records of Changes to Specific Files

If your goal is to log file writes against a limited number of files, such as /etc/passwd and the files in the /etc/default directory, you use the auditreduce command to locate the files.

1. Audit the fw class.

   Adding the class to the audit_user file generates fewer records than adding the class to the audit_control file.

   • Add the fw class to the audit_user file.

     ## audit_user file root:fw:no sysadm:fw:no auditadm:fw:no netadm:fw:no

   • Add the fw class to the audit_control file.

     ## audit_control file flags:lo,fw ...

2. To find the audit records for specific files, use the auditreduce command.

   # /usr/sbin/auditreduce -o file=/etc/passwd,/etc/default -O filechg

   The auditreduce command searches the audit trail for all instances of the file argument. The command creates a binary file with the suffix filechg which contains all records that include the pathnames of the files of interest. See the auditreduce(1M) man page for the syntax of the -o file=pathname option.

3. To read the filechg file, use the praudit command.

   # /usr/sbin/praudit *filechg

How to Modify a User's Preselection Mask

If you modify the audit_control or audit_user file, the preselection mask of users who are already logged in does not change. You must force the preselection mask to change.

Before You Begin

You enabled auditing, users logged in, and then you changed the value of flags or naflags in the audit_control file. You want the users who are already logged in to be audited for these newly selected audit classes.

1. Update the preselection mask of users who are already logged in.

   You have two options. You can terminate the existing sessions or use the auditconfig command to update the users' preselection masks.

   • Terminate the users' existing sessions.

     Users can log out and log back in, or the administrator can manually terminate (kill) active sessions. The new sessions will inherit the new preselection mask. However, terminating users could be impractical.

   • Dynamically change each user's preselection mask.

     Assume that the flags attribute in the audit_control file was changed from lo to lo,ex.

     1. Determine the user's audit ID and audit session ID.

        First, find all regular users. In the following example, the administrator finds all processes that are not owned by root, daemon, or lp:

        # /usr/bin/pgrep -v -u root,daemon,lp | more .. 3941 3948 3949 10640 ...

        Then, use one of the user's processes to find the user's audit ID:

        # auditconfig -getpinfo 3941 audit id = jdoe(1002) process preselection mask = lo(0x1000,0x1000) terminal id (maj,min,host) = 9426,65559,mach1(192.168.123.234) audit session id = 713

        Note that the user's preselection mask includes the lo class and does not include the newly added ex class.

     The user's audit ID is 1002. The user's audit session ID is 713.

2. Change the user's preselection mask

   Use one of the following two methods:

   • Use the user's audit session ID to change the user's preselection mask.

     # /usr/sbin/auditconfig -setsmask lo,ex 713

   • Use the user's audit ID to change the user's preselection mask.

     # /usr/sbin/auditconfig -setumask lo,ex 1002

3. Verify that the preselection mask has changed.

   # auditconfig -getpinfo 3941 audit id = jdoe(1002) process preselection mask = ex,lo(0x40001000,0x40001000) terminal id (maj,min,host) = 9426,65559,mach1(192.168.123.234) audit session id = 713

How to Prevent the Auditing of Certain Events

For maintenance purposes, sometimes a site wants to prevent audit events from being audited.

1. Change the class of the event to the no class.

   For example, events 26 and 27 belong to the pm class.

   ## audit_event file ... 25:AUE_VFORK:vfork(2):ps 26:AUE_SETGROUPS:setgroups(2):pm 27:AUE_SETPGRP:setpgrp(2):pm 28:AUE_SWAPON:swapon(2):no ...

   Change these events to the no class.

   ## audit_event file ... 25:AUE_VFORK:vfork(2):ps 26:AUE_SETGROUPS:setgroups(2):no 27:AUE_SETPGRP:setpgrp(2):no 28:AUE_SWAPON:swapon(2):no ...

   If the pm class is currently being audited, existing sessions will still audit events 26 and 27. To stop these events from being audited, you must update the users' preselection masks.

   ---

   Caution –

   Never comment out events in the audit_event file. This file is used by the praudit command to read binary audit files. Archived audit files might contain events that are listed in the file.

   ---

2. To update the preselection masks of users, follow the instructions in How to Modify a User's Preselection Mask.

How to Limit the Size of Binary Audit Files

Binary audit files grow without limit. For ease of archiving and searching, you might want to limit the size. You can also create smaller binary files from the original file.

1. Use the p_fsize attribute to limit the size of individual binary audit files.

   The p_fsize attribute to the audit_binfile.so plugin enables you to limit the size of an audit file. The default value is zero (0), which allows the file to grow without limit. The value is specified in bytes, from 512,000 to 2,147,483,647. When the specified size is reached, the current audit file is closed and a new file is opened.

   In the following example, you limit the size of and audit file to 1Mbyte:

   plugin:name=audit_binfile.so; p_dir:/var/audit; p_fsize=1024000

2. Use the auditreduce command to select records and write those records to a file for further analysis.

   The auditreduce -lowercase options find specific records.

   The auditreduce -Uppercase options write your selections to a file. For more information, see the auditreduce(1M) man page.

How to Audit Logins From Other OSes

The Solaris OS can audit all logins, independent of source.

1. Audit the lo class for attributable and for non-attributable events.

   This class audits logins, logouts, and screen locks.

   ## audit_control file flags:lo naflags:lo ...

   ---

   Note –

   To audit ssh logins, your Solaris system must be running the Solaris ssh daemon. This daemon is modified for Solaris auditing. For more information, see Solaris Secure Shell and the OpenSSH Project.

   ---

How to Audit FTP and SFTP File Transfers

The FTP service creates logs of its file transfers. The SFTP service, which runs under the SSH protocol, can be audited by Solaris auditing. Logins to both services can be audited by Solaris auditing.

1. To log commands and file transfers of the FTP service, see the ftpaccess(4) man page.

   For the available logging options, read the “Logging Capabilities” section. In particular, the log commands and log transfers options might provide useful logs.

2. To log sftp file transfers, perform one or both of the following:

   • Audit file-reads.

     File transfers over an SSH connection use the sftp command. These transfers can be recorded by using the +fr audit flag. To audit failed sftp file transfers, audit the -fr audit flag.

     The following output is from a successful sftp session:

     header,138,2,open(2) - read,,ma2,2009-08-25 14:48:58.770 -07:00 path,/home/jdoe/vpn_connect attribute,100644,jdoe,staff,391,437,0 subject,jdoe,jdoe,staff,jdoe,staff,4444,120289379,8457 65558 ma1 return,success,6

   • Use the verbose option to the sftp command.

     The -v option can be repeated up to three times.

     # sftp -vvv [ other options ] hostname

3. To record access to the FTP and SFTP services, audit the lo class.

   As the following output indicates, logging in to and out of the ftpd daemon generates audit records.

   % bsmrecord -c lo | more ... in.ftpd program /usr/sbin/in.ftpd See ftp access event ID 6165 AUE_ftpd class lo (0x00001000) header subject [text] error message return in.ftpd program /usr/sbin/in.ftpd See ftp logout event ID 6171 AUE_ftpd_logout class lo (0x00001000) header subject return ...

   The SSH login records all accesses to the sftp command.

   ... /usr/lib/ssh/sshd program /usr/lib/ssh/sshd See login - ssh event ID 6172 AUE_ssh class lo (0x00001000) header subject [text] error message return