ZFS uses a pool storage model where storage devices are aggregated into a storage pool. File systems within the storage pool use all of the storage in the pool.
Specific root pool device and boot disk requirements
See these references:
General root pool creation best practices
You must create the root pool as a mirrored configuration or as a single-disk configuration. Neither a RAID-Z nor a striped configuration is supported. You cannot add additional disks to create multiple mirrored top-level virtual devices by using the zpool add command. To expand a mirrored virtual device, use the zpool attach command.
The root pool cannot have a separate log device.
You can set pool properties during an installation with AI by using the pool_options keyword syntax, but the gzip compression algorithm is not supported on root pools.
Do not rename the root pool after it is created by an initial installation. Renaming the root pool might result in an unbootable system.
Do not create a root pool on a USB stick for a production system because root pool disks are critical for continuous operation, particularly in an enterprise environment. Consider using a system's internal disks for the root pool, or at minimum, use the same quality disks that you would use for your non-root data. In addition, a USB stick might not be large enough to support a dump volume size that is equivalent to at least 1/2 the size of physical memory.
Consider keeping root pool components separate from non-root pool data.
Non-root pool creation best practices
Create non-root pools with whole disks by using the d* identifier. Do not use the p* identifier.
ZFS works best without any additional volume management software.
For better performance, use individual disks or at least logical unit numbers (LUNs) that are made up of just a few disks. By providing ZFS with more visibility into the LUN setup, ZFS is able to make better I/O scheduling decisions.
Mirrored storage pools – Consume more disk space but generally perform better with small random reads, as shown in the following example:
# zpool create tank mirror c1d0 c2d0 mirror c3d0 c4d0
Mirrored storage pools are also more flexible, in that you can detach, attach, and replace existing devices in the pool.
RAID-Z storage pools
You can create RAID-Z storage pools with 3 parity strategies, where parity equals 1 (raidz), 2 (raidz2), or 3 (raidz3).
A RAID-Z configuration maximizes disk space and generally performs well when data is written and read in large chunks (128K or more). Create a single-parity RAIDZ (raidz) configuration at 3 disks (2+1).
A RAIDZ-2 configuration offers better data availability, and performs similarly to RAID-Z. RAIDZ-2 has significantly better mean time to data loss (MTTDL) than either RAID-Z or 2-way mirrors. Create a double-parity RAID-Z (raidz2) configuration at 6 disks (4+2).
A RAIDZ-3 configuration maximizes disk space and offers excellent availability because it can withstand 3 disk failures. Create a triple-parity RAID-Z (raidz3) configuration at 8 disks (5+3).
Non-redundant pools
If you create a non-redundant pool, you will see a message similar to the following:
# zpool create pond c8t2d0 c8t3d0 'pond' successfully created, but with no redundancy; failure of one device will cause loss of the pool
Creating a pool with no redundancy is not recommended because a device failure could mean that the data is unrecoverable. Consider creating a ZFS storage pool with redundancy as follows:
# zpool create pond mirror c8t2d0 c8t3d0
Refer to the following best practices for monitoring ZFS storage pools:
Make sure that pool usage is below 90% of pool capacity for best performance.
Note that the zpool list command does not count RAID-Z parity as used space, nor does it subtract it from the pool capacity. RAID-Z pool capacity might be below 90%, while being nearly full. Use the zfs list pool command to check. See Displaying ZFS File System Information.
Use the zpool scrub command on a regular basis to identify data integrity problems:
If you have consumer quality drives, consider a weekly scrubbing schedule.
If you have datacenter quality drives, consider a monthly scrubbing schedule.
You should also run a scrub prior to replacing devices to ensure that all devices are currently operational.
Use the zpool status command on a weekly basis to monitor pool and pool device status. Also use the fmdump or fmdump –ev command to determine if any device faults or errors have occurred.
Review the following new diagnostic descriptions and features:
Failed devices – Review the zpool status –l output to identify the physical location of the failed device and replace it. For information about replacing a failed disk, see Replacing or Repairing a Damaged Device in Managing ZFS File Systems in Oracle Solaris 11.3.
Failed device notification – You can configure the smtp-notify service to send email notifications in response to various fault management events, such as when a hardware component has been diagnosed as faulty. See the notification parameters section of smf(5).
By default, some notifications are set up automatically to be sent to the root user. If you add an alias for your user account as root in the /etc/aliases file, you will receive email notifications.
Moving devices – Devices that are part of a ZFS storage pool contain a device ID if the device driver creates or fabricates device IDs. Like all file systems, ZFS has a very close relationship with its underlying devices. If you attempt to upgrade a system's firmware, move a pool device to a different controller, or change a device's cabling, you might consider exporting the pool first. If the device ID does not follow the device change, which can happen with non-Oracle hardware, then the pool and pool data might become unavailable. In general, Oracle's Sun hardware can recover if a device is changed under a live pool because these drivers fully support device IDs. However, you might consider exporting the pool before making any hardware changes.