You should configure your storage pools using ZFS redundancy. Without redundancy, the risk of losing data is great. Moreover, without ZFS redundancy, the pool can only report data inconsistencies, but cannot repair those inconsistencies. ZFS provides data redundancy and self-healing properties in mirrored and RAID-Z configurations.
A mirrored storage pool configuration requires at least two disks, preferably on separate controllers. A mirrored configuration can be simple or complex, where more than one mirror exists in each pool.
For information about creating simple or complex mirrored storage pools, see Creating a Mirrored Storage Pool.
ZFS supports a RAID-Z configuration with the following fault tolerance levels:
Single-parity (raidz or raidz1) – Similar to RAID-5.
Double-parity (raidz2) – Similar to RAID-6.
Triple-parity (raidz3) – For more information, see Triple-Parity RAID and Beyond (https://queue.acm.org/detail.cfm?id=1670144).
In RAID-Z, ZFS uses variable-width RAID stripes so that all writes are full-stripe writes. ZFS integrates file system and device management in such a way that the file system's metadata has enough information about the underlying data redundancy model to handle variable-width RAID stripes. Thus, RAID-Z avoids issues encountered in traditional RAID algorithms such as RAID-5's write hole problem.
ZFS provides self-healing data in a mirrored or RAID-Z configuration. When a bad data block is detected, ZFS fetches the correct data from another redundant copy and repairs the bad data by replacing it with the good copy.
A RAID-Z configuration with n disks of size x with p parity disks can hold approximately (n-p)*x bytes and can withstand p devices failing before data integrity is compromised. You need at least two disks for a single-parity RAID-Z configuration and at least three disks for a double-parity RAID-Z configuration, and so on. For example, if you have three disks in a single-parity RAID-Z configuration, parity data occupies disk space equal to one of the three disks. Otherwise, no special hardware is required to create a RAID-Z configuration.
Just like mirrored configurations, RAID-Z configurations can either be simple or complex.
If you are creating a RAID-Z configuration with many disks, consider splitting the disks into multiple groupings. For example, a RAID-Z configuration with 14 disks is better split into two 7-disk groupings. RAID-Z configurations with single-digit groupings of disks commonly perform better.
See the following sources for more information:
Creating a RAID-Z Storage Pool – provides information about creating a RAID-Z storage pool.
https://blogs.oracle.com/roch/when-to-and-not-to-use-raid-z.: – presents guidelines on choosing between a mirrored configuration or a RAID-Z configuration based on performance and disk space considerations.
Recommended Oracle Solaris ZFS Practices – covers additional RAID-Z storage pool recommendations.
The ZFS hybrid storage pool, available in Oracle's Sun Storage 7000 product series, combines DRAM, SSDs, and HDDs to improve performance and increase capacity, while reducing power consumption. With this product's management interface, you can select the ZFS redundancy configuration of the storage pool and easily manage other configuration options.
For more information, see the documentation in https://www.oracle.com/technetwork/documentation/old-unified-ss-1882427.html.
ZFS dynamically stripes data across all top-level virtual devices. The decision about where to place data is done at write time, so no fixed-width stripes are created at allocation time.
When new virtual devices are added to a pool, ZFS gradually allocates data to the new device in order to maintain performance and disk space allocation policies. Each virtual device can also be a mirror or a RAID-Z device that contains other disk devices or files. This configuration gives you flexibility in controlling the fault characteristics of your pool. For example, you could create the following configurations out of four disks:
Four disks using dynamic striping
One four-way RAID-Z configuration
Two two-way mirrors using dynamic striping
To ensure efficient use of ZFS, use top-level virtual devices of the same type with the same redundancy level in each device. Do not combine different types of virtual devices within the same pool, such as using a two-way mirror and a three-way RAID-Z configuration.