iostat - report I/O statistics
/usr/bin/iostat [-cCdDeEiILmMnpPrstwxXYuz] [-l n] [-T u | d] [disk]... [interval [count]]
The iostat utility iteratively reports terminal, disk, and tape I/O activity, as well as CPU utilization. The first line of output is for all time since boot; each subsequent line is for the prior interval only.
To compute this information, the kernel maintains a number of counters. For each disk, the kernel counts reads, writes, bytes read, and bytes written. The kernel also takes hi-res time stamps at queue entry and exit points, which allows it to keep track of the residence time and cumulative residence-length product for each queue. Using these values, iostat produces highly accurate measures of throughput, utilization, queue lengths, transaction rates and service time. For terminals collectively, the kernel simply counts the number of input and output characters.
During execution of the kernel status command, the state of the system can change. If relevant, a state change message is included in the iostat output, in one of the following forms:
<<device added: sd0>> <<device removed: sd0>> <<partition added: sd0,a>> <<partition removed: sd0,a>> <<NFS mounted: nfs1>> <<NFS unmounted: nfs1>> <<multi-path added: ssd4>> <<multi-path removed: ssd4>> <<controller added: c1>> <<controller removed: c1>> <<processors added: 1, 3>> <<processors removed: 1, 3>>
Note that the names printed in these state change messages are affected by the –n and –m options as appropriate.
For more general system statistics, use sar(1), sar(8), or vmstat(8).
The output of the iostat utility includes the following information.
name of the disk
reads per second
writes per second
kilobytes read per second
The average I/O size during the interval can be computed from kr/s divided by r/s.
kilobytes written per second
The average I/O size during the interval can be computed from kw/s divided by w/s.
average number of transactions waiting for service (queue length)
This is the number of I/O operations held in the device driver queue waiting for acceptance by the device.
average number of transactions actively being serviced (removed from the queue but not yet completed)
This is the number of I/O operations accepted, but not yet serviced, by the device.
average response time of transactions, in milliseconds
The svc_t output reports the overall response time, rather than the service time, of a device. The overall time includes the time that transactions are in queue and the time that transactions are being serviced. The time spent in queue is shown with the –x option in the wsvc_t output column. The time spent servicing transactions is the true service time. Service time is also shown with the –x option and appears in the asvc_t output column of the same report.
percent of time there are transactions waiting for service (queue non-empty)
For NFS, this is the percent of time that there is at least one asynchronous request waiting to be started
percent of time the disk is busy (transactions in progress)
For NFS, this is the percent of time that there is at least one pending NFS RPC, that is, the system is waiting for a reply from the server
average service time in wait queue, in milliseconds
average service time of active transactions, in milliseconds
percent of CPU time which was taken off by the hypervisor. Always zero when running on physical hardware.
The following options are supported:
Report the percentages of time the system has spent in user mode, in system mode, time taken off by the hypervisor (see the definition of st below), and idling. For more information, see the NOTES section.
When the –x option is also selected, report extended disk statistics aggregated by controller id.
For each disk, report the number of kilobytes transferred per second, the number of transfers per second, and the average service time in milliseconds.
For each disk, report the reads per second, writes per second, and percentage disk utilization.
Display device error summary statistics. The total errors, hard errors, soft errors, and transport errors are displayed.
Display all device error statistics.
In –E output, display the Device ID instead of the Serial No. The Device Id is a unique identifier registered by a driver through ddi_devid_register(9F).
Report the counts in each interval, rather than rates (where applicable).
Limit the number of disks included in the report to n; the disk limit defaults to 4 for –d and –D, and unlimited for –x. Note that disks explicitly requested (see disk below) are not subject to this disk limit.
Report IO latency distribution statistics. This option can be used to observe disk IO latency statistics with –x option. For MPxIO enabled system(s), this option can also be used to observe latency statistics for each path with the –Y option. Note that the label '1us' means '1024ns' and '1ms' means '1024*1024ns'. This option cannot be used in conjunction with –t or –c options and will be ignored when used with the –E option. Numbers reported in the distribution may not add up to the exact same value as the total as snopshots may be taken over very slightly distjoint time periods.
Report file system mount points. This option is most useful if the –P or –p option is also specified or used in conjunction with –Xn or –en. The –m option is useful only if the mount point is actually listed in the output. This option can only be used in conjunction with the –n option.
Display data throughput in MB/sec instead of KB/sec.
Display names in descriptive format. For example, cXtYdZ, rmt/N, server:/export/path.
By default, disks are identified by instance names such as ssd23 or md301. Combining the –n option with the –x option causes disk names to display in the cXtYdZsN format which is more easily associated with physical hardware characteristics. The cXtYdZsN format is particularly useful in Fibre Channel (FC) environments where the FC World Wide Name appears in the t field.
For each disk, report per-partition statistics in addition to per-device statistics.
For each disk, report per-partition statistics only, no per-device statistics.
Display data in a comma-separated format.
Suppress messages related to state changes.
Report the number of characters read and written to terminals per second.
Display a time stamp.
Specify u for a printed representation of the internal representation of time. See time(2). Specify d for standard date format. See date(1).
Wide display format.
Using this option along with either of the options –x, –X, –Y or their combinations will display the output in wider format. The headers will be more descriptive and two space gap is provided between each column in this display.
Refer the Example section for sample outputs.
For disks under scsi_vhci(4D) control, in addition to disk lun statistics, also report statistics for lun.controller.
Report extended disk statistics. By default, disks are identified by instance names such as ssd23 or md301. Combining the x option with the –n option causes disk names to display in the cXtYdZsN format, more easily associated with physical hardware characteristics. Using the cXtYdZsN format is particularly helpful in the FibreChannel environments where the FC World Wide Name appears in the t field.
If no output display is requested (no –x, –e, –E), –x is implied.
For disks under scsi_vhci(4D) control, in addition to disk lun statistics, also report statistics for lun.targetport and lun.targetport.controller.
In –n (descriptive) mode the targetport is shown in using the target-port property of the path. Without –n the targetport is shown using the shorter port-id. All target ports with the same target-port property value share the same port-id. The target-port-to-port-id association does not persist across reboot.
If no output display is requested (no –x, –e, –E), –x is implied.
Print micro-second resolution for wsvc_t and asvc_t fields. Must be used with -x.
Do not print lines whose underlying data values are all zeros.
The option set –xcnCXTdz interval is particularly useful for determining whether disk I/O problems exist and for identifying problems.
The following operands are supported:
Display only count reports.
Explicitly specify the disks to be reported; in addition to any explicit disks, any active disks up to the disk limit (see –l above) will also be reported.
Report once each interval seconds.
The following command displays two reports of extended device statistics, aggregated by controller id, for user (us) and system (sy) operations. Because the –n option is used with the –x option, devices are identified by controller names.
example% iostat –xcnCXTdz 5
Thursday, May 23, 2013 09:17:03 AM PDT
cpu
us sy st id
14 31 0 20
extended device statistics
r/s w/s kr/s kw wait actv wsvc_t asvc_t %w %b device
3.8 29.9 145.8 44.0 0.0 0.2 0.1 6.4 0 5 c0
666.3 814.8 12577.6 17591.1 91.3 82.3 61.6 55.6 0 2 c12
180.0 234.6 4401.1 5712.6 0.0 147.7 0.0 356.3 0 98 d10
Thursday, May 23, 2013 09:17:03 AM PDT
cpu
us sy st id
11 31 0 22
extended device statistics
r/s w/s kr/s kw wait actv wsvc_t asvc_t %w %b device
0.8 41.0 5.2 20.5 0.0 0.2 0.2 4.4 0 6 c0
565.3 581.7 8573.2 10458.9 0.0 26.6 0.0 23.2 0 3 c12
106.5 81.3 3393.2 1948.6 0.0 5.7 0.0 30.1 0 99 d10
Example 2 Using iostat to Generate TTY Statistics
The following command displays two reports on the activity of five disks in different modes of operation. Because the –x option is used, disks are identified by instance names.
example% iostat -xt 5 2
extended device statistics tty
device r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b tin tout
blkdev0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0 1
sd0 0.1 19.3 1.4 92.4 0.0 0.0 0.2 1.6 0 1
sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
nfs9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0 0
nfs10 0.0 0.0 0.0 0.0 0.0 0.0 0.0 7.6 0 0
nfs11 0.0 0.0 0.0 0.0 0.0 0.0 0.0 15.6 0 0
nfs12 0.3 0.0 1.9 0.0 0.0 0.0 0.0 30.5 0 1
extended device statistics tty
device r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b tin tout
blkdev0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0 132
sd0 0.0 30.8 0.0 345.6 0.0 0.1 0.0 2.1 0 1
sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
nfs9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
nfs10 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
nfs11 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
nfs12 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
Example 3 Using iostat to Generate Partition and Device Statistics
The following command generates partition and device statistics for each disk. Because the –n option is used with the –x option, disks are identified by controller names.
example% iostat -xnp
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.4 0.3 10.4 7.9 0.0 0.0 0.0 36.9 0 1 c0t0d0
0.3 0.3 9.0 7.3 0.0 0.0 0.0 37.2 0 1 c0t0d0s0
0.0 0.0 0.1 0.5 0.0 0.0 0.0 34.0 0 0 c0t0d0s1
0.0 0.0 0.0 0.1 0.0 0.0 0.6 35.0 0 0 fuji:/export/home/user3
Example 4 Show Translation from Instance Name to Descriptive Name
The following example illustrates the use of iostat to translate a specific instance name to a descriptive name.
example% iostat -xn sd1
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c8t1d0
Example 5 Show Target Port and Controller Activity for a Specific Disk
In the following example, there are four controllers, all connected to the same target port.
# iostat -Y ssd6
extended device statistics
device r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b
ssd6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.7 0 0
ssd6.t2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
ssd6.t2.fp4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
ssd6.t4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
ssd6.t4.fp1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
Example 6 Using iostat to Generate disk IO Latency Distribution
The following command generates path latency distribution statistics for the specified disk.
example% iostat -Lx sd1
extended device statistics
device r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b
sd1 14.4 16.5 426.7 758.2 0.0 0.4 0.1 13.6 0 11
latency range count density distribution
<32us 0 0.00% 0.00%
32-64us 696 0.87% 0.87%
64-128us 1622 2.03% 2.90%
128-256us 5609 7.01% 9.90%
256-512us 2655 3.32% 13.22%
512-1024us 4745 5.93% 19.15%
1-2ms 4463 5.58% 24.73%
2-4ms 7159 8.94% 33.67%
4-8ms 16899 21.11% 54.78%
8-16ms 14824 18.52% 73.30%
16-32ms 12630 15.78% 89.08%
32-64ms 6491 8.11% 97.19%
64-128ms 1908 2.38% 99.58%
128-256ms 301 0.38% 99.95%
256-512ms 38 0.05% 100.00%
>1024ms 0 0.00% 100.00%
total 80040
Example 7 iostat wide display sample-1
example% iostat -xw sd1
extended device statistics
Device Reads/s Writes/s Read KB/s Written KB/s WaitQ SvcQ AvSvcTimeInWaitQ AvSvcTimeActTrans %Wait %Busy
sd1 0.7 33.0 19.6 239.6 0.0 0.0 0.0 0.2 0 0
Example 8 iostat wide display sample-2
example% iostat -Yw sd1
extended device statistics
Device Reads/s Writes/s Read KB/s Written KB/s WaitQ SvcQ AvSvcTimeInWaitQ AvSvcTimeActTrans %Wait %Busy
sd1 0.7 33.0 19.6 239.8 0.0 0.0 0.0 0.2 0 0
sd1.t9 0.7 33.0 40.4 263.2 0.0 0.0 0.0 0.2 0 0
sd1.t9.lmrc1 0.7 33.0 40.4 263.2 0.0 0.0 0.0 0.2 0 0
Example 9 iostat wide display sample-3
example% iostat -IxnwCX sd1
extended device statistics
Reads/i Writes/i Read KB/i Written KB/i WaitQ SvcQ AvSvcTimeInWaitQ AvSvcTimeActTrans %Wait %Busy Device
37124.0 1911845.0 1116521.1 13937283.0 0.0 0.0 0.0 0.2 0 0 c0
37124.0 1911845.0 1116521.1 13937283.0 0.0 0.0 0.0 0.2 0 0 c0t1d0
38413.0 1911778.0 2302469.1 15273619.0 0.0 0.0 0.0 0.2 0 0 c0t1d0.c1
See attributes(7) for descriptions of the following attributes:
|
Invocation is evolving. Human readable output is unstable.
date(1), sar(1), time(2), scsi_vhci(4D), attributes(7), mpstat(8), sar(8), vmstat(8)
The sum of CPU utilization might vary slightly from 100 because of rounding errors in the production of a percentage figure.
The svc_t response time is not particularly significant when the I/0 (r/s+w/s) rates are under 0.5 per second. Harmless spikes are fairly normal in such cases.
The mpstat utility reports the same st, usr, and sys statistics. For more information, see the mpstat(8) man page.
When executed in a zone and if the pools facility is active, iostat will only provide information for those processors in the processor set of the pool to which the zone is bound.
For NFS, %w indicates asynchronous activity such as read-ahead or write-behind. These requests are typically processed in parallel with application I/Os. A large value does not necessarily indicate a performance problem.