Disk: I/O Operations

This statistic shows the back-end I/O operations per second to the disks (disk IOPS) after Oracle ZFS Storage Appliance has processed logical I/O into physical I/O based on share settings and software RAID settings. To configure the RAID settings, see Configuring Storage in Oracle ZFS Storage Appliance Administration Guide, Release OS8.8.x.

For example, 16 sequential 8 Kbyte NFSv3 writes might become a single 128 Kbyte write sometime later after the data has been buffered in the ARC DRAM cache, which might then become multiple disk writes due to RAID, such as two writes to each half of a mirror. It can help to monitor I/O at all layers at the same time to examine this behavior; for example, by viewing:

• Protocol: NFSv[2-4] Operations - NFS writes (logical)

• Disk: ZFS Logical I/O Operations - Share I/O (logical)

• Disk: I/O Operations - I/O to the disks (physical)

This statistic includes a breakdown of disk I/O latency, which is a direct measure of performance for synchronous I/O, and also useful as a measure of the magnitude of back-end disk load. It is difficult to identify issues from disk IOPS alone without considering latency: A single disk might be performing well at 400 IOPS (sequential and small I/O hitting mostly from the disk's on-board DRAM cache), yet poorly at 110 IOPS (random I/O causing head seek and waiting on disk rotation).

The latency breakdown is presented as a heat map, showing the pattern of I/O latency, along with outliers. Mouse over the outliers icon to see the current value, and click on the icon to toggle between different percentages of outlier elimination. Disk I/O latency is often related to the performance of the delivered logical I/O, such as with synchronous reads (non-prefetch) and synchronous writes. Sometimes the latency is not directly related to logical I/O performance, such as for asynchronous writes that are later flushed to disk, and for prefetch reads.

Because it is difficult to determine a per-disk IOPS limit, also examine the disk IOPS by the offset, which helps identify the IOPS type as either random or sequential, and the I/O size. Use the following breakdowns to observe these attributes:

• Disk: I/O Operations - Broken down by offset

• Disk: I/O Operations - Broken down by size

When viewing a breakdown, select an individual result in the pane to highlight it and display it separately, by color, in the graph. Select an already highlighted result to not display it separately in the graph.

When viewing the disk IOPS by the disk breakdown, mousing over a disk in the disk breakdown pane displays a box with the following information:

• Disk name - Controller or disk shelf name/label: I/O operations per second

• Disk Type: - Typically HDD or SSD

• Type: - Typically System, Data, Cache, or Log

• Size

• RPM - Not displayed for SSDs

To display the hierarchy view for all disks, click View Hierarchy, below the disk breakdown pane. The I/O operations per second are shown for the controller and each disk shelf. Click Refresh hierarchy to refresh the hierarchical breakdown visible in the graph. To close this view, click the close icon .

When to Check I/O Operations

Use this statistic to understand the nature of back-end disk I/O, based on disk I/O operations per second (IOPS), after an issue based on disk utilization or latency has been observed.

Use the disk breakdown pane and the hierarchy view to determine if the disk shelves are balanced in regard to disk IOPS. When examining disk IOPS, it is common that cache and log devices have higher throughput than other storage pool disks.

I/O Operations Breakdowns

Table 5-32 Breakdowns of I/O Operations

Breakdown	Description
type of operation	Read or write.
disk	Storage pool disk or system disk. This breakdown can identify system disk I/O versus pool disk I/O, and I/O to cache and log devices.
size	Heat map showing the distribution of I/O sizes.
latency	Heat map showing the latency of disk I/O, as measured from when the I/O was requested to the disk to when the disk returned the completion.
offset	Heat map showing the disk location offset of disk I/O. This can be used to identify random or sequential disk IOPS. To better see details, use the zoom-in icon .

Further Analysis

For the best measure of disk utilization, see Disk: Disks. To examine bytes/second instead of operations/second, see Disk: I/O Bytes.