This section describes the tasks for monitoring and displaying system performance information.
In addition to traditional command line tools such as iostat and mpstat, Oracle Solaris includes a browser-based performance visualization tool. The Oracle Solaris System Web Interface enables you to visualize both historical and current performance data and relationships among data from different sources and system events. For more information, see Using Oracle Solaris 11.4 StatsStore and System Web Interface.
You can use the vmstat command to report virtual memory statistics and information about system events such as CPU load, paging, number of context switches, device interrupts, and system calls. The vmstat command can also display statistics on swapping, cache flushing, and interrupts.
Reports the number of kernel threads in the following states:
The number of kernel threads in the dispatch queue.
The number of blocked kernel threads that are waiting for resources.
The number of swapped-out LWPs that are waiting for processing resources to finish.
Reports on usage of real memory and virtual memory.
Available swap space.
Size of the free list.
Reports on page faults and paging activity, in units per second.
Pages reclaimed.
Minor faults and major faults.
Kilobytes paged in.
Kilobytes paged out.
Kilobytes freed.
Anticipated memory that is needed by recently swapped-in processes.
Pages scanned by the page daemon not currently in use. If sr does not equal zero, the page daemon has been running.
Reports the number of disk operations per second, showing data on up to four disks.
Reports the trap/interrupt rates per second.
Interrupts per second.
System calls per second.
CPU context switch rate.
Reports on the use of CPU time.
User time.
System time.
Idle time.
$ vmstat 5 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr dd f0 s1 -- in sy cs us sy id 0 0 0 863160 365680 0 3 1 0 0 0 0 0 0 0 0 406 378 209 1 0 99 0 0 0 765640 208568 0 36 0 0 0 0 0 0 0 0 0 479 4445 1378 3 3 94 0 0 0 765640 208568 0 0 0 0 0 0 0 0 0 0 0 423 214 235 0 0 100 0 0 0 765712 208640 0 0 0 0 0 0 0 3 0 0 0 412 158 181 0 0 100 0 0 0 765832 208760 0 0 0 0 0 0 0 0 0 0 0 402 157 179 0 0 100 0 0 0 765832 208760 0 0 0 0 0 0 0 0 0 0 0 403 153 182 0 0 100 0 0 0 765832 208760 0 0 0 0 0 0 0 0 0 0 0 402 168 177 0 0 100 0 0 0 765832 208760 0 0 0 0 0 0 0 0 0 0 0 402 153 178 0 0 100 0 0 0 765832 208760 0 18 0 0 0 0 0 0 0 0 0 407 165 186 0 0 100
For more information, see the vmstat(8) man page.
You can use the vmstat -s command to show how many system events have taken place since the last time the system was booted.
$ vmstat -s 0 swap ins 0 swap outs 0 pages swapped in 0 pages swapped out 522586 total address trans. faults taken 17006 page ins 25 page outs 23361 pages paged in 28 pages paged out 45594 total reclaims 45592 reclaims from free list 0 micro (hat) faults 522586 minor (as) faults 16189 major faults 98241 copy-on-write faults 137280 zero fill page faults 45052 pages examined by the clock daemon 0 revolutions of the clock hand 26 pages freed by the clock daemon 2857 forks 78 vforks 1647 execs 34673885 cpu context switches 65943468 device interrupts 711250 traps 63957605 system calls 3523925 total name lookups (cache hits 99%) 92590 user cpu 65952 system cpu 16085832 idle cpu 7450 wait cpu
You can use the vmstat -S command to show swapping statistics.
$ vmstat -S kthr memory page disk faults cpu r b w swap free si so pi po fr de sr dd f0 s1 -- in sy cs us sy id 0 0 0 862608 364792 0 0 1 0 0 0 0 0 0 0 0 406 394 213 1 0 99
The swapping statistics fields are described in the following list. For a description of the other fields, see Displaying Virtual Memory Statistics.
Average number of LWPs that are swapped per second
Number of whole processes that are swapped out
You can use the vmstat -i command to show the number of interrupts per device.
$ vmstat -i interrupt total rate -------------------------------- clock 52163269 100 esp0 2600077 4 zsc0 25341 0 zsc1 48917 0 cgsixc0 459 0 lec0 400882 0 fdc0 14 0 bppc0 0 0 audiocs0 0 0 -------------------------------- Total 55238959 105
You can use the iostat command to report statistics about disk input and output, and to produce measures of throughput, utilization, queue lengths, transaction rates, and service time. For a detailed description of this command, see the iostat(8) man page.
You can display disk utilization information by using the iostat command with a time interval in seconds.
$ iostat 5 tty fd0 sd3 nfs1 nfs31 cpu tin tout kps tps serv kps tps serv kps tps serv kps tps serv us sy wt id 0 1 0 0 410 3 0 29 0 0 9 3 0 47 4 2 0 94
The first line of output shows the statistics since the last time the system was booted. Each subsequent line shows the interval statistics. The default is to show statistics for the terminal (tty), disks (fd and sd), and CPU (cpu).
The following example shows disk statistics that were gathered every five seconds.
$ iostat 5 tty sd0 sd6 nfs1 nfs49 cpu tin tout kps tps serv kps tps serv kps tps serv kps tps serv us sy wt id 0 0 1 0 49 0 0 0 0 0 0 0 0 15 0 0 0 100 0 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 44 6 132 0 0 0 0 0 0 0 0 0 0 0 1 99 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 3 1 23 0 0 0 0 0 0 0 0 0 0 0 1 99 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100
The following list describes the fields in the output of the iostat n command:
Number of characters in the terminal input queue.
Number of characters in the terminal output queue.
Blocks per second.
Transactions per second.
Average service time, in milliseconds.
In user mode.
In system mode.
Waiting for I/O.
Idle.
You can use the iostat -xt command to display extended disk statistics.
$ iostat -xt extended device statistics tty device r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b tin tout sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 1 449 sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 sd2 0.3 0.6 84.9 205.9 0.0 0.0 0.0 6.6 0 0 sd3 2.2 7.3 54.4 187.2 0.0 0.1 0.0 5.7 0 2 ... nfs3102 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 nfs3104 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0 0 nfs3106 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.6 0 0 nfs3107 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 nfs3108 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 nfs3109 0.0 0.0 0.0 0.0 0.0 0.0 0.0 117.1 0 0
The iostat -xt command displays a line of output for each disk. The output fields are as follows:
Reads per second
Writes per second
Kbytes read per second
Kbytes written per second
Average number of transactions that are waiting for service (queue length)
Average number of transactions that are actively being serviced
Average service time, in milliseconds
Percentage of time that the queue is not empty
Percentage of time that the disk is busy
You can use the df command to show the amount of free disk space on each mounted disk. The usable disk space that is reported by df reflects only 90 percent of full capacity because the reporting statistics allows for 10 percent above the total available space. This head room normally stays empty for better performance.
The percentage of disk space actually reported by the df command is used space divided by usable space.
If the file system exceeds 90 percent capacity, you can transfer files to a disk that is not as full by using the cp command. Alternately, you can transfer files to a tape by using the tar or cpio commands or you can remove the files.
You can use the df -k command to display disk space information in kilobytes.
$ df -k Filesystem kbytes used avail capacity Mounted on /dev/dsk/c0t3d0s0 192807 40231 133296 24% /Example 14 Displaying File System Information
The following example shows output from the df -k command on a SPARC based system.
$ df -k Filesystem 1024-blocks Used Available Capacity Mounted on rpool/ROOT/solaris-161 191987712 6004395 140577816 5% / /devices 0 0 0 0% /devices /dev 0 0 0 0% /dev ctfs 0 0 0 0% /system/contract proc 0 0 0 0% /proc mnttab 0 0 0 0% /etc/mnttab swap 4184236 496 4183740 1% /system/volatile objfs 0 0 0 0% /system/object sharefs 0 0 0 0% /etc/dfs/sharetab /usr/lib/libc/libc_hwcap1.so.1 146582211 6004395 140577816 5% /lib/libc.so.1 fd 0 0 0 0% /dev/fd swap 4183784 60 4183724 1% /tmp rpool/export 191987712 35 140577816 1% /export rpool/export/home 191987712 32 140577816 1% /export/home rpool/export/home/123 191987712 13108813 140577816 9% /export/home/123 rpool/export/repo 191987712 11187204 140577816 8% /export/repo rpool/export/repo2010_11 191987712 31 140577816 1% /export/repo2010_11 rpool 191987712 5238974 140577816 4% /rpool /export/home/123 153686630 13108813 140577816 9% /home/123
The output fields of the df -k command are as follows:
Total size of usable space in the file system
Amount of space used
Amount of space available for use
Amount of space used, as a percentage of the total capacity
Mount point
The following example shows a list of all the mounted file systems, when the df command is used without any options or operands.
$ df / (rpool/ROOT/solaris):100715496 blocks 100715496 files /devices (/devices ): 0 blocks 0 files /dev (/dev ): 0 blocks 0 files /system/contract (ctfs ): 0 blocks 2147483601 files /proc (proc ): 0 blocks 29946 files /etc/mnttab (mnttab ): 0 blocks 0 files /system/volatile (swap ):42257568 blocks 2276112 files /system/object (objfs ): 0 blocks 2147483441 files /etc/dfs/sharetab (sharefs ): 0 blocks 2147483646 files /dev/fd (fd ): 0 blocks 0 files /tmp (swap ):42257568 blocks 2276112 files /export (rpool/export ):100715496 blocks 100715496 files /export/home (rpool/export/home ):100715496 blocks 100715496 files /export/home/admin (rpool/export/home/admin):100715496 blocks 100715496 files /rpool (rpool ):100715496 blocks 100715496 files /export/repo2010_11(rpool/export/repo2010_11):281155639 blocks 281155639 files /rpool (rpool ):281155639 blocks 281155639 files
For more information, see the df(8) man page.
The daxstat command reports the utilization and performance statistics for Data Analytics Accelerator (DAX) on systems that have the SPARC M7, SPARC M8, SPARC T7, or SPARC T8 chip. You must be in the root role to use the daxstat command. This command reports DAX statistics per-DAX, per-CPU, and per-queue in a tabular form. The first table summarizes all activity since boot. Each subsequent table summarizes the activity for the preceding interval. The values are reported as rates (events per second), unless mentioned otherwise.
The daxstat command has the following syntax:
/usr/bin/daxstat [[-T u | d] [-c processor_id] | [-d dax_id [-q queue_id]]] \ | [-[x]d dax_id]] [interval [count]]
/usr/bin/daxstat -a [[-T u | d] [-c] | [-d [-q ]]] [-[x]d]] [interval [count]]
The following attributes are supported by the daxstat command.
The output is aggregated into a single value for all the CPUs, queues, and DAX units. Do not enter any processor_id, dax_id, or queue_id with this option.
Displays CPU statistics for specified CPUs.
Displays DAX statistics for specified DAX units.
Displays queue statistics for specified queues in specified DAX units.
Displays per-dax extended statistics for specified DAX units. This option is only valid when used with the –d option.
Prints a time stamp before each report, in either standard date format, d, or the internal representation of time, u. For more information, see the time(2) and date(1) man pages.
Reports once each interval second.
Prints only count reports.
The following example shows DAX statistics over a three-second interval in two reports.
$ daxstat -ad 3 2 DAX commands fallbacks input output %busy ALL 201757 194975 74.0M 0.0M 0 ALL 53388 52066 9.0G 31.0M 0
The output fields of the daxstat command for statistics per-DAX are as follows:
DAX ID
Number of commands completed by DAX
Number of commands completed by the software but not completed by DAX
Total input processed by DAX in megabytes per second (M) or gigabytes per second (G)
Total output produced by DAX in megabytes per second (M) or gigabytes per second (G)
Percentage of DAX cycles spent processing a command
The following example shows DAX statistics for CPUs 0 and 1 in one report.
$ daxstat -c 0-1 CPU calls time success fallbacks 0 6486811 9276154005 6486811 0 1 6553856 9272570130 6553856 0
The output fields of the daxstat command for DAX statistics per-CPU are as follows:
CPU ID
Number of dax_submit fast trap calls
Total kernel and hypervisor time in seconds spent submitting DAX commands
Number of ccb_submit hyper-calls that returned success
Number of ccb_submit hyper-calls that returned an error
The following example shows DAX statistics for queues 0-3 in DAX unit 4.
$ daxstat -d 4 -q 0-3 DAX QUEUE commands 4 0 105 1 196 2 496 3 196
where QUEUE is the queue ID.
For more information, see the daxstat(8) man page.
For more information about DAX, see the Breaking New Ground with Software in Silicon” blog and Accelerate Analytics with Oracle Database In-Memory and Software in Silicon at the Oracle Developer Community site.
You can use the daxinfo command to display the static configuration of DAX hardware available on systems that have the SPARC M7, SPARC M8, SPARC T7, or SPARC T8 chip. Information available includes DAX version, DAX operation codes, and the number of DAX instances on the system.
Example 19 Displaying DAX InformationThis example shows how to display information related to DAX by using the daxinfo command.
$ daxinfo Version: DAX1 Opcodes: Extract Scan Select Translate Enabled: 6 Disabled:0
where Version is the DAX version, Opcodes is the list of DAX operation codes that are supported, Enabled is the number of available DAX instances, and Disabled is the number of DAX instances with hardware error.
To display some of the selected fields, use –o and –p options with the daxinfo command as shown in the following example:
$ daxinfo -p -o version,enabled DAX2:8
where –p displays the output in machine-readable format.
For more information, see the daxinfo (8) man page.