JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle Solaris 11.1 Tunable Parameters Reference Manual     Oracle Solaris 11.1 Information Library
search filter icon
search icon

Document Information


1.  Overview of Oracle Solaris System Tuning

2.  Oracle Solaris Kernel Tunable Parameters

3.  Oracle Solaris ZFS Tunable Parameters

Where to Find Tunable Parameter Information

Tuning ZFS Considerations

ZFS ARC Parameters



ZFS File-Level Prefetch


ZFS Device I/O Queue Depth


Tuning ZFS When Using Flash Storage

Adding Flash Devices as ZFS Log or Cache Devices

Ensuring Proper Cache Flush Behavior for Flash and NVRAM Storage Devices

SCSI Unmap Considerations for Flash Devices

Tuning ZFS for Database Products

Tuning ZFS for an Oracle Database

Using ZFS with MySQL Considerations

4.  NFS Tunable Parameters

5.  Internet Protocol Suite Tunable Parameters

6.  System Facility Parameters

A.  Tunable Parameters Change History

B.  Revision History for This Manual


Tuning ZFS When Using Flash Storage

The following information applies to Flash SSDs, F20 PCIe Accelerator Card, F40 PCIe Accelerator Card, and F5100 Flash Storage Array.

Review the following general comments when using ZFS with Flash storage:

Adding Flash Devices as ZFS Log or Cache Devices

Review the following recommendations when adding flash devices as ZFS log or cache devices.

A flash device, c4t1d0, can be added as a ZFS log device:

# zpool add pool log c4t1d0

If 2 flash devices are available, you can add mirrored log devices:

# zpool add pool log mirror c4t1d0 c4t2d0

Available flash devices can be added as a cache device for reads.

# zpool add pool cache c4t3d0

You can't mirror cache devices, they will be striped together.

# zpool add pool cache c4t3d0 c4t4d0

Ensuring Proper Cache Flush Behavior for Flash and NVRAM Storage Devices

ZFS is designed to work with storage devices that manage a disk-level cache. ZFS commonly asks the storage device to ensure that data is safely placed on stable storage by requesting a cache flush. For JBOD storage, this works as designed and without problems. For many NVRAM-based storage arrays, a performance problem might occur if the array takes the cache flush request and actually does something with it, rather than ignoring it. Some storage arrays flush their large caches despite the fact that the NVRAM protection makes those caches as good as stable storage.

ZFS issues infrequent flushes (every 5 second or so) after the uberblock updates. The flushing infrequency is fairly inconsequential so no tuning is warranted here. ZFS also issues a flush every time an application requests a synchronous write (O_DSYNC, fsync, NFS commit, and so on). The completion of this type of flush is waited upon by the application and impacts performance. Greatly so, in fact. From a performance standpoint, this neutralizes the benefits of having an NVRAM-based storage.

Cache flush tuning was recently shown to help flash device performance when used as log devices. When all LUNs exposed to ZFS come from NVRAM-protected storage array and procedures ensure that no unprotected LUNs will be added in the future, ZFS can be tuned to not issue the flush requests by setting zfs_nocacheflush. If some LUNs exposed to ZFS are not protected by NVRAM, then this tuning can lead to data loss, application level corruption, or even pool corruption. In some NVRAM-protected storage arrays, the cache flush command is a no-op, so tuning in this situation makes no performance difference.

A recent OS change is that the flush request semantic has been qualified to instruct storage devices to ignore the requests if they have the proper protection. This change requires a fix to our disk drivers and for the NVRAM device to support the updated semantics. If the NVRAM device does not recognize this improvement, use these instructions to tell the Solaris OS not to send any synchronize cache commands to the array. If you use these instructions, make sure all targeted LUNS are indeed protected by NVRAM.

Occasionally, flash and NVRAM devices do not properly advertise to the OS that they are non- devices, and that caches do not need to be flushed. Cache flushing is an expensive operation. Unnecessary flushes can drastically impede performance in some cases.

Review the following zfs_nocacheflush syntax restrictions before applying the tuning entries below:


Caution - All cache sync commands are ignored by the device. Use at your own risk.

  1. Use the format utility to run the inquiry subcommand on a LUN from the storage array. For example:

    # format
    Specify disk (enter its number): x
    format> inquiry
    Vendor:   ATA     
    Product:  Marvell      
    Revision: XXXX
  2. Select one of the following based on your architecture:

    • For F40 flash devices, add the following entry to /kernel/drv/sd.conf. In the entry below, ensure that "ATA " is padded to 8 characters, and "3E128-TS2-550B01" contains 16 characters. Total string length is 24.

      sd-config-list="ATA     3E128-TS2-550B01","disksort:false, cache-non:true";
    • For F20 and F5100 flash devices, choose one of the following based on your architecture. In the entries below, "ATA " is padded to 8 characters, and "MARVELL SD88SA02" contains 16 characters. The total string length is 24.

    • Many SPARC architectures – Add the following entry to /kernel/drv/ssd.conf:

      ssd-config-list = "ATA     MARVELL SD88SA02","throttle-max:32, disksort:false, cache-non:true";
    • x64 and a few SPARC drivers – Add the following entry to /kernel/drv/sd.conf

      ssd-config-list="ATA     MARVELL SD88SA02","throttle-max:32, disksort:false, cache-non:true";
  3. Carefully add whitespace to make the vendor ID (VID) 8 characters long (here ATA) and Product ID (PID) 16 characters long (here MARVELL) in the sd-config-list entry as illustrated.

  4. Reboot the system.

    You can tune zfs_nocacheflush back to it's default value (0) with no adverse effect on performance.

SCSI Unmap Considerations for Flash Devices

The Solaris 11.1 OS has a problem that resulted in excessive calls to SCSI unmap routines. This particular problem is detrimental to flash performance. The workaround is to disable the unmap feature as follows: