This chapter provides reference information about system accounting.
This is a list of the reference information in this chapter.
For more information about system accounting tasks, see Chapter 9, Managing System Accounting (Tasks).
The main daily accounting script, runacct, is normally invoked by the cron command outside of normal business hours. The runacct script processes connect, fee, disk, and process accounting files. This script also prepares daily and cumulative summary files for use by the prdaily and monacct scripts for billing purposes.
The runacct script takes care not to damage files if errors occur.
A series of protection mechanisms that are used to perform the following tasks:
Recognize an error
Provide intelligent diagnostics
Complete processing in such a way that the runacct script can be restarted with minimal intervention
This script records its progress by writing descriptive messages to the active file. Files used by the runacct script are assumed to be in the /var/adm/acct/nite directory, unless otherwise noted. All diagnostic output during the execution of the runacct script is written to the fd2log file.
When the runacct script is invoked, it creates the lock and lock1 files. These files are used to prevent simultaneous execution of the runacct script. The runacct program prints an error message if these files exist when it is invoked. The lastdate file contains the month and day the runacct script was last invoked, and is used to prevent more than one execution per day.
If the runacct script detects an error, the following occurs:
A message is written to the console
Email is sent to root and adm
Locks might be removed
Diagnostics are saved
Execution is ended
For instructions on how to restart the runacct script, see How to Restart the runacct Script.
To allow the runacct script to be restarted, processing is broken down into separate re-entrant states. The statefile file is used to track the last state completed. When each state is completed, the statefile file is updated to reflect the next state. After processing for the state is complete, the statefile file is read and the next state is processed. When the runacct script reaches the CLEANUP state, it removes the locks and ends. States are executed as shown in the following table.
Table 10–1 States of the runacct Script
State |
Description |
---|---|
SETUP |
The turnacct switch command is executed to create a new pacct file. The /var/adm/pacctn process accounting files (except for the pacct file) are moved to the /var/adm/Spacctn.MMDD files. The /var/adm/wtmpx file is moved to the /var/adm/acct/nite/wtmp.MMDD file (with the current time record added on the end) and a new /var/adm/wtmp file is created. The closewtmp and utmp2wtmp programs add records to the wtmp.MMDD file and the new wtmpx file to account for users who are currently logged in. |
WTMPFIX |
The wtmpfix program checks the wtmp.MMDD file in the nite directory for accuracy. Because some date changes cause the acctcon program to fail, the wtmpfix program attempts to adjust the time stamps in the wtmpx file if a record of a date change appears. This program also deletes any corrupted entries from the wtmpx file. The fixed version of the wtmp.MMDD file is written to the tmpwtmp file. |
CONNECT |
The acctcon program is used to record connect accounting records in the file ctacct.MMDD. These records are in tacct.h format. In addition, the acctcon program creates the lineuse and reboots files. The reboots file records all the boot records found in the wtmpx file. |
PROCESS |
The acctprc program is used to convert the /var/adm/Spacctn.MMDD process accounting files into complete accounting records in the ptacctn.MMDD files. The Spacct and ptacct files are correlated by number so that if the runacct script fails, the Spacct files are not processed. |
MERGE |
The acctmerg program merges the process accounting records with the connect accounting records to form the daytacct file. |
FEES |
The acctmerg program merges ASCII tacct records from the fee file into the daytacct file. |
DISK |
The dodisk script produces the disktacct file. If the dodisk script has been run, which produces the disktacct file, the DISK program merges the file into the daytacct file and moves the disktacct file to the /tmp/disktacct.MMDD file. |
MERGETACCT |
The acctmerg program merges the daytacct file with the sum/tacct file, the cumulative total accounting file. Each day, the daytacct file is saved in the sum/tacct.MMDD file so that the sum/tacct file can be re-created if it is corrupted or lost. |
CMS |
The acctcms program is run several times. This program is first run to generate the command summary by using the Spacctn files and write the data to the sum/daycms file. The acctcms program is then run to merge the sum/daycms file with the sum/cms cumulative command summary file. Finally, the acctcms program is run to produce nite/daycms and nite/cms, the ASCII command summary files from the sum/daycms and sum/cms files, respectively. The lastlogin program is used to create the /var/adm/acct/sum/loginlog log file. This file reports when each user last logged in. If the runacct script is run after midnight, the dates showing the time last logged in by some users will be incorrect by one day. |
USEREXIT |
Any installation-dependent (local) accounting program can be run at this point. The runacct script expects this program to be called the /usr/lib/acct/runacct.local program. |
CLEANUP |
This state cleans up temporary files, runs the prdaily script and saves its output in the sum/rpt.MMDD file, removes the locks, and then exits. |
When restarting the runacct script in the CLEANUP state, remove the last ptacct file because this file will not be complete.
The runacct shell script generates five basic reports upon each invocation. The following table describes these reports.
Table 10–2 Daily Accounting Reports
Report Type |
Description |
---|---|
Shows terminal line utilization by tty number. |
|
Indicates usage of system resources by users (listed in order of user ID). |
|
Indicates usage of system resources by commands, listed in descending order of memory use. In other words, the command that used the most memory is listed first. This same information is reported for the month in the monthly command summary. |
|
A cumulative summary that reflects the data accumulated since the last invocation of the monacct program. |
|
Shows the last time each user logged in (listed in chronological order). |
This report gives information about each terminal line used. The following is a sample Daily Report.
Jan 16 02:30 2004 DAILY REPORT FOR venus Page 1 from Mon Jan 15 02:30:02 2004 to Tue Oan 16 02:30:01 2004 1 runacct 1 acctcon TOTAL DURATION IS 1440 MINUTES LINE MINUTES PERCENT # SESS # ON # OFF console 868 60 1 1 2 TOTALS 868 -- 1 1 2 |
The from and to lines specify the time period reflected in the report. This time period covers the time the last Daily Report was generated to the time the current Daily Report was generated. Then, the report presents a log of system reboots, shutdowns, power failure recoveries, and any other record written to the /var/adm/wtmpx file by the acctwtmp program. For more information, see the acct(1M) man page.
The second part of the report is a breakdown of terminal line utilization. The TOTAL DURATION tells how long the system was in multiuser mode (accessible through the terminal lines). The following table describes the data provided by the Daily Report.
Table 10–3 Daily Report Data
During real time, you should monitor the /var/adm/wtmpx file because it is the file from which the connect accounting is derived. If the wtmpx file grows rapidly, execute the following command to see which tty line is the noisiest.
# /usr/lib/acct/acctcon -l file < /var/adm/wtmpx |
If interruption is occurring frequently, general system performance will be affected. Additionally, the wtmp file might become corrupted. To correct this problem, see How to Fix a Corrupted wtmpx File.
The Daily Usage Report breaks down system resource utilization by user. A sample of this report follows.
Jan 16 02:30 2004 DAILY USAGE REPORT FOR skisun Page 1 LOGIN CPU (MINS) KCORE- MINS CONNECT (MINS) DISK # OF # OF # DISK FEE UID NAME PRIME NPRIME PRIME NPRIME PRIME NPRIME BLOCKS PROCS SESS SAMPLES 0 TOTAL 72 148 11006173 51168 26230634 57792 539 330 0 2150 1 0 root 32 76 11006164 33664 26230616 22784 0 0 0 127 0 4 adm 0 0 22 51 0 0 0 420 0 0 0 101 rimmer 39 72 894385 1766020 539 330 0 1603 1 0 0 |
The following table describes the data provided by the Daily Usage Report.
Table 10–4 Daily Usage Report Data
The Daily Command Summary report shows the system resource utilization by command. With this report, you can identify the most heavily used commands. Based on how those commands use system resources, you can then gain insight on how best to tune the system.
These reports are sorted by TOTAL KCOREMIN, which is an arbitrary gauge but often useful for calculating drain on a system.
A sample Daily Command Summary follows.
TOTAL COMMAND SUMMARY COMMAND NUMBER TOTAL TOTAL TOTAL MEAN MEAN HOG CHARS BLOCKS NAME CMDS KCOREMIN CPU-MIN REAL-MIN SIZE-K CPU-MIN FACTOR TRNSFD READ TOTALS 2150 1334999.75 219.59 724258.50 6079.48 0.10 0.00 397338982 419448 netscape 43 2456898.50 92.03 54503.12 26695.51 2.14 0.00 947774912 225568 adeptedi 7 88328.22 4.03 404.12 21914.95 0.58 0.01 93155160 8774 dtmail 1 54919.17 5.33 17716.57 10308.94 5.33 0.00 213843968 40192 acroread 8 31218.02 2.67 17744.57 11682.66 0.33 0.00 331454464 11260 dtwm 1 16252.93 2.53 17716.57 6416.05 2.53 0.00 158662656 12848 dtterm 5 4762.71 1.30 76300.29 3658.93 0.26 0.00 33828352 11604 dtaction 23 1389.72 0.33 0.60 4196.43 0.01 0.55 18653184 539 dtsessio 1 1174.87 0.24 17716.57 4932.97 0.24 0.00 23535616 5421 dtcm 1 866.30 0.18 17716.57 4826.21 0.18 0.00 3012096 6490 |
The following table describes the data provided by the Daily Command Summary.
Table 10–5 Daily Command Summary Data
Column |
Description |
---|---|
COMMAND NAME |
Name of the command. All shell procedures are lumped together under the name sh because only object modules are reported by the process accounting system. You should monitor the frequency of programs called a.out or core, or any other unexpected name. You can use the acctcom program to determine who executed an oddly named command and if superuser privileges were used. |
NUMBER CMDS |
Total number of times this command was run. |
TOTAL KCOREMIN |
Total cumulative measurement of the Kbyte segments of memory used by a process per minute of run time. |
TOTAL CPU-MIN |
Total processing time this program accumulated. |
TOTAL REAL-MIN |
Total real-time (wall-clock) minutes this program accumulated. |
MEAN SIZE-K |
Mean (average) of the TOTAL KCOREMIN over the number of invocations reflected by the NUMBER CMDS. |
MEAN CPU-MIN |
Mean (average) derived from the NUMBER CMDS and the TOTAL CPU-MIN. |
HOG FACTOR |
Total CPU time divided by elapsed time. Shows the ratio of system availability to system utilization, providing a relative measure of total available CPU time consumed by the process during its execution. |
CHARS TRNSFD |
Total number of characters transferred by the read and write system calls. Might be negative due to overflow. |
BLOCKS READ |
Total number of the physical block reads and writes that a process performed. |
The format of the Daily Command Summary and the Monthly Command Summary reports are virtually the same. However, the daily summary reports only on the current accounting period while the monthly summary reports on the start of the fiscal period to the current date. In other words, the monthly report is a cumulative summary that reflects the data accumulated since the last invocation of the monacct program.
A sample Monthly Command Summary follows.
Jan 16 02:30 2004 MONTHLY TOTAL COMMAND SUMMARY Page 1 TOTAL COMMAND SUMMARY COMMAND NUMBER TOTAL TOTAL TOTAL MEAN MEAN HOG CHARS BLOCKS NAME CMDS KCOREMIN CPU-MIN REAL-MIN SIZE-K CPU-MIN FACTOR TRNSFD READ TOTALS 42718 4398793.50 361.92 956039.00 12154.09 0.01 0.00 16100942848 825171 netscape 789 3110437.25 121.03 79101.12 25699.58 0.15 0.00 3930527232 302486 adeptedi 84 1214419.00 50.20 4174.65 24193.62 0.60 0.01 890216640 107237 acroread 145 165297.78 7.01 18180.74 23566.84 0.05 0.00 1900504064 26053 dtmail 2 64208.90 6.35 20557.14 10112.43 3.17 0.00 250445824 43280 dtaction 800 47602.28 11.26 15.37 4226.93 0.01 0.73 640057536 8095 soffice. 13 35506.79 0.97 9.23 36510.84 0.07 0.11 134754320 5712 dtwm 2 20350.98 3.17 20557.14 6419.87 1.59 0.00 190636032 14049 |
For a description of the data provided by the Monthly Command Summary, see Daily Command Summary.
This report gives the date when a particular login was last used. You can use this information to find unused logins and login directories that can be archived and deleted. A Last Login Report follows.
Jan 16 02:30 2004 LAST LOGIN Page 1 01-06-12 kryten 01-09-08 protoA 01-10-14 ripley 01-07-14 lister 01-09-08 protoB 01-10-15 scutter1 01-08-16 pmorph 01-10-12 rimmer 01-10-16 scutter2 |
At any time, you can examine the contents of the /var/adm/pacctn files, or any file with records in the acct.h format, by using the acctcom program. If you do not specify any files and do not provide any standard input when you run this command, the acctcom command reads the pacct file. Each record read by the acctcom command represents information about a terminated process. Active processes can be examined by running the ps command.
The default output of the acctcom command provides the following information:
# acctcom COMMAND START END REAL CPU MEAN NAME USER TTYNAME TIME TIME (SECS) (SECS) SIZE(K) #accton root ? 02:30:01 02:30:01 0.03 0.01 304.00 turnacct adm ? 02:30:01 02:30:01 0.42 0.01 320.00 mv adm ? 02:30:01 02:30:01 0.07 0.01 504.00 utmp_upd adm ? 02:30:01 02:30:01 0.03 0.01 712.00 utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 824.00 utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 912.00 utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 920.00 utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 1136.00 utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 576.00 closewtm adm ? 02:30:01 02:30:01 0.10 0.01 664.00 |
Field |
Explanation |
---|---|
COMMAND NAME |
Command name (pound (#) sign if the command was executed with superuser privileges) |
USER |
User name |
TTYNAME |
tty name (listed as ? if unknown) |
START TIME |
Command execution starting time |
END TIME |
Command execution ending time |
REAL (SECS) |
Real time (in seconds) |
CPU (SECS) |
CPU time (in seconds) |
MEAN SIZE (K) |
Mean size (in Kbytes) |
You can obtain the following information by using acctcom command options.
State of the fork/exec flag (1 for fork without exec)
System exit status
Hog factor
Total kcore minutes
CPU factor
Characters transferred
Blocks read
The following table describes the acctcom command options.
Table 10–6 Options for the acctcom Command
Option |
Description |
---|---|
-a |
Shows average statistics about the processes selected. The statistics are printed after the output is recorded. |
-b |
Reads the files backward, showing latest commands first. This option has no effect if reading standard input. |
-f |
Prints the fork/exec flag and system exit status columns. The output is an octal number. |
-h |
Instead of mean memory size, shows the hog factor, which is the fraction of total available CPU time consumed by the process during its execution. Hog factor = total-CPU-time/elapsed-time. |
-i |
Prints columns that contains the I/O counts in the output. |
-k |
Shows total kcore minutes instead of memory size. |
-m |
Shows mean core size. This size is the default. |
-q |
Prints average statistics, not output records. |
-r |
Shows CPU factor: user-time/(system-time + user-time). |
-t |
Shows separate system and user CPU times. |
-v |
Excludes column headings from the output. |
-C sec |
Shows only processes with total CPU time (system plus user) that exceeds sec seconds. |
-e time |
Shows processes existing at or before time, given in the format hr[:min[:sec]] |
-E time |
Shows processes starting at or before time, given in the format hr[:min[:sec]]. Using the same time for both -S and -E, shows processes that existed at the time. |
-g group |
Shows only processes that belong to group. |
-H factor |
Shows only processes that exceed factor, where factor is the “hog factor” (see the -h option). |
-I chars |
Shows only processes that transferred more characters than the cutoff number specified by chars. |
-l line |
Show only processes that belong to the terminal /dev/line. |
-n pattern |
Shows only commands that match pattern (a regular expression except that “+” means one or more occurrences). |
-o ofile |
Instead of printing the records, copies them in acct.h format to ofile. |
-O sec |
Shows only processes with CPU system time that exceeds sec seconds. |
-s time |
Show processes existing at or after time, given in the format hr[:min[:sec]]. |
-S time |
Show processes starting at or after time, given in the format hr[:min[:sec]]. |
-u user |
Shows only processes that belong to user. |
The /var/adm directory contains the active data collection files. The following table describes the accounting files in this directory.
Table 10–7 Files in the /var/adm Directory
File |
Description |
---|---|
dtmp |
Output from the acctdusg program |
fee |
Output from the chargefee program, which are the ASCII tacct records |
pacct |
Active process accounting file |
pacctn |
Process accounting files that are switched by running the turnacct script |
Spacctn.MMDD |
Process accounting files for MMDD during execution of the runacct script |
The /var/adm/acct directory contains the nite, sum, and fiscal directories. These directories contain the actual data collection files. For example, the nite directory contains files that are reused daily by the runacct script. A brief summary of the files in the /var/adm/acct/nite directory follows.
Table 10–8 Files in the /var/adm/acct/nite Directory
File |
Description |
---|---|
active |
Used by the runacct script to record progress and print warning and error messages |
active.MMDD |
Same as the active file after the runacct script detects an error |
cms |
ASCII total command summary used by the prdaily script |
ctacct.MMDD |
Connect accounting records in tacct.h format |
ctmp |
Output of acctcon1 program, which consists of connect session records in ctmp.h format (acctcon1 and acctcon2 are provided for compatibility purposes) |
daycms |
ASCII daily command summary used by the prdaily script |
Total accounting records for one day in tacct.h format |
|
disktacct |
Disk accounting records in tacct.h format, created by the dodisk script |
fd2log |
Diagnostic output during execution of the runacct script |
lastdate |
Last day the runacct script executed (in date +%m%d format) |
lineuse |
tty line usage report used by the prdaily script |
lock |
Used to control serial use of the runacct script |
log |
Diagnostic output from the acctcon program |
log.MMDD |
Same as the log file after the runacct script detects an error |
owtmpx |
Previous day's wtmpx file |
reboots |
Beginning and ending dates from the wtmpx file, and a listing of reboots |
statefile |
Used to record current state during execution of the runacct script |
tmpwtmp |
wtmpx file corrected by the wtmpfix program |
wtmperror |
Contains wtmpfix error messages |
wtmperror MMDD |
Same as the wtmperror file after the runacct script detects an error |
The runacct script's copy of the wtmpx file |
The sum directory contains the cumulative summary files updated by the runacct script and used by the monacct script. The following table summarizes the files in the /var/adm/acct/sum directory.
Table 10–9 Files in the /var/adm/acct/sum Directory
File |
Description |
---|---|
cms |
Total command summary file for current fiscal period in binary format |
cmsprev |
Command summary file without latest update |
daycms |
Command summary file for the day's usage in internal summary format |
loginlog |
Record of last date each user logged in; created by the lastlogin script and used in the prdaily script |
rprt.MMDD |
Saved output of prdaily script |
tacct |
Cumulative total accounting file for current fiscal period |
tacctprev |
Same as the tacct file without latest update |
tacct.MMDD |
Total accounting file for MMDD |
The fiscal directory contains periodic summary files that are created by the monacct script. The following table summarizes the files in the /var/adm/acct/fiscal directory.
Table 10–10 Files in the /var/adm/acct/fiscal Directory
File |
Description |
---|---|
cmsn |
Total command summary file for fiscal period n in internal summary format |
fiscrptn |
Report similar to rprtn for fiscal period n |
tacctn |
Total accounting file for fiscal period n |
The following table summarizes the most useful files produced by the runacct script. These files are found in the /var/adm/acct directory.
Table 10–11 Files Created by the runacct Script
File |
Description |
---|---|
The total accounting file for the day in tacct.h format. |
|
The runacct script calls the acctcon program to gather data on terminal line usage from the /var/adm/acct/nite/tmpwtmp file and writes the data to the /var/adm/acct/nite/lineuse file. The prdaily script uses this data to report line usage. This report is especially useful for detecting bad lines. If the ratio between the number of logouts to logins is greater than three to one, the line is very likely failing. |
|
sum/cms |
This file is the accumulation of each day's command summaries. The accumulation restarts when the monacct script is executed. The ASCII version is the nite/cms file. |
sum/daycms |
The runacct script calls the acctcms program to process the commands used during the day to create the Daily Command Summary report and stores the data in the /var/adm/acct/sum/daycms file. The ASCII version is the /var/adm/acct/nite/daycms file. |
sum/loginlog |
The runacct script calls the lastlogin script to update the last date logged in for the logins in the /var/adm/acct/sum/loginlog file. The lastlogin command also removes from this file any logins that are no longer valid. |
sum/rprt.MMDD |
Each execution of the runacct script saves a copy of the daily report that was printed by the prdaily script. |
sum/tacct |
Contains the accumulation of each day's nite/daytacct data and is used for billing purposes. The monacct script restarts accumulating this data each month or fiscal period. |