Skip Navigation Links | |
Exit Print View | |
Oracle Solaris ZFS Administration Guide Oracle Solaris 10 8/11 Information Library |
1. Oracle Solaris ZFS File System (Introduction)
New Oracle Solaris ZFS Installation Features
ZFS Snapshot Differences (zfs diff)
ZFS Storage Pool Recovery and Performance Enhancements
Tuning ZFS Synchronous Behavior
ZFS ACL Interoperability Enhancements
Splitting a Mirrored ZFS Storage Pool (zpool split)
Enhancements to the zpool list Command
ZFS Device Replacement Enhancements
ZFS and Flash Installation Support
ZFS ACL Pass Through Inheritance for Execute Permission
Using Cache Devices in Your ZFS Storage Pool
Zone Migration in a ZFS Environment
ZFS Installation and Boot Support
Rolling Back a Dataset Without Unmounting
Enhancements to the zfs send Command
ZFS Quotas and Reservations for File System Data Only
ZFS Command History Enhancements (zpool history)
Upgrading ZFS File Systems (zfs upgrade)
Setting Up Separate ZFS Log Devices
Creating Intermediate ZFS Datasets
Recursively Renaming ZFS Snapshots (zfs rename -r)
gzip Compression Is Available for ZFS
Storing Multiple Copies of ZFS User Data
ZFS and Solaris iSCSI Improvements
ZFS Command History (zpool history)
Setting Properties When Creating ZFS File Systems
Displaying All ZFS File System Information
Hot Spares for ZFS Storage Pool Devices
Replacing a ZFS File System With a ZFS Clone (zfs promote)
Upgrading ZFS Storage Pools (zpool upgrade)
ZFS Backup and Restore Commands Are Renamed
Recovering Destroyed Storage Pools
ZFS Is Integrated With Fault Manager
Checksums and Self-Healing Data
ZFS Component Naming Requirements
2. Getting Started With Oracle Solaris ZFS
3. Oracle Solaris ZFS and Traditional File System Differences
4. Managing Oracle Solaris ZFS Storage Pools
5. Installing and Booting an Oracle Solaris ZFS Root File System
6. Managing Oracle Solaris ZFS File Systems
7. Working With Oracle Solaris ZFS Snapshots and Clones
8. Using ACLs and Attributes to Protect Oracle Solaris ZFS Files
9. Oracle Solaris ZFS Delegated Administration
10. Oracle Solaris ZFS Advanced Topics
11. Oracle Solaris ZFS Troubleshooting and Pool Recovery
12. Recommended Oracle Solaris ZFS Practices
This section summarizes new features in the ZFS file system.
Oracle Solaris 10 8/11 Release: In this release, the following new installation features are available:
You can use the text mode installation method to install a system with a ZFS flash archive. For more information, see Example 5-3.
You can use the Oracle Solaris Live Upgrade luupgrade command to install a ZFS root flash archive. For more information, see Example 5-8.
You can use the Oracle Solaris Live Upgrade lucreate command to specify a separate /var file system. For more information, see Example 5-5.
Oracle Solaris 10 8/11 Release: In this release, you can set file system properties that are sent and received in a snapshot stream. These enhancements provide flexibility in applying file system properties in a send stream to the receiving file system or in determining whether the local file system properties, such as the mountpoint property value, should be ignored when received.
For more information, see Applying Different Property Values to a ZFS Snapshot Stream.
Oracle Solaris 10 8/11 Release: In this release, you can determine ZFS snapshot differences by using the zfs diff command.
For example, assume that the following two snapshots are created:
$ ls /tank/cindy fileA $ zfs snapshot tank/cindy@0913 $ ls /tank/cindy fileA fileB $ zfs snapshot tank/cindy@0914
For example, to identify the differences between two snapshots, use syntax similar to the following:
$ zfs diff tank/cindy@0913 tank/cindy@0914 M /tank/cindy/ + /tank/cindy/fileB
In the output, the M indicates that the directory has been modified. The + indicates that fileB exists in the later snapshot.
For more information, see Identifying ZFS Snapshot Differences (zfs diff).
Oracle Solaris 10 8/11 Release: In this release, the following new ZFS storage pool features are provided:
You can import a pool with a missing log by using the zpool import -m command. For more information, see Importing a Pool With a Missing Log Device.
You can import a pool in read-only mode. This feature is primarily for pool recovery. If a damaged pool cannot be accessed because the underlying devices are damaged, you can import the pool read-only to recover the data. For more information, see Importing a Pool in Read-Only Mode.
A RAID-Z (raidz1, raidz2, or raidz3) storage pool that is created in this release and upgraded to at least pool version 29 will have some latency-sensitive metadata automatically mirrored to improve read I/O throughput performance. For existing RAID-Z pools that are upgraded to at least pool version 29, some metadata will be mirrored for all newly written data.
Mirrored metadata in a RAID-Z pool does not provide additional protection against hardware failures, similar to what a mirrored storage pool provides. Additional space is consumed by mirrored metadata, but the RAID-Z protection remains the same as in previous releases. This enhancement is for performance purposes only.
Solaris 10 8/11 Release: In this release, you can determine a ZFS file system's synchronous behavior by using the sync property.
The default synchronous behavior is to write all synchronous file system transactions to the intent log and to flush all devices to ensure that the data is stable. Disabling the default synchronous behavior is not recommended. Applications that depend on synchronous support might be affected, and data loss could occur.
The sync property can be set before or after the file system is created. In either case, the property value takes effect immediately. For example:
# zfs set sync=always tank/neil
The zil_disable parameter is no longer available in Oracle Solaris releases that include the sync property.
For more information, see Table 6-1.
Oracle Solaris 10 8/11 Release: In this release, you can use the -T option to provide an interval and count value for the zpool list and zpool status commands to display additional information.
In addition, more pool scrub and resilver information is provided by the zpool status command as follows:
Resilver in-progress report. For example:
scan: resilver in progress since Thu May 26 11:26:32 2011 1.26G scanned out of 2.40G at 6.15M/s, 0h3m to go 1.26G resilvered, 56.3% done
Scrub in-progress report. For example:
scan: scrub in progress since Fri May 27 08:24:17 2011 18.0M scanned out of 2.35G at 8.99M/s, 0h4m to go 0 repaired, 0.75% done
Resilver completion message. For example:
scan: resilvered 2.34G in 1h2m with 0 errors on Thu May 26 11:56:40 2011
Scrub completion message. For example:
scan: scrub repaired 512B in 1h2m with 0 errors on Fri May 27 08:54:50 2011
Ongoing scrub cancellation message. For example:
scan: scrub canceled on Wed Fri Jun 10 09:06:24 2011
Scrub and resilver completion messages persist across system reboots
The following syntax uses the interval and count option to display ongoing pool resilvering information. You can use the -T d value to display the information in standard date format or -T u to display the information in an internal format.
# zpool status -T d tank 3 2 Wed Jun 22 14:35:40 GMT 2011 pool: tank state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Wed Jun 22 14:33:29 2011 3.42G scanned out of 7.75G at 28.2M/s, 0h2m to go 3.39G resilvered, 44.13% done config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c2t3d0 ONLINE 0 0 0 c2t4d0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c2t7d0 ONLINE 0 0 0 c2t8d0 ONLINE 0 0 0 (resilvering) errors: No known data errors
Oracle Solaris 10 8/11 Release: In this release, the following ACL enhancements are provided:
Trivial ACLs do not require deny ACEs except for unusual permissions. For example, a mode of 0644, 0755, or 0664 does not require deny ACEs, but a mode, such as 0705, 0060, and so on, does require deny ACEs.
The old behavior includes deny ACEs in a trivial ACL like 644. For example:
# ls -v file.1 -rw-r--r-- 1 root root 206663 Jun 14 11:52 file.1 0:owner@:execute:deny 1:owner@:read_data/write_data/append_data/write_xattr/write_attributes /write_acl/write_owner:allow 2:group@:write_data/append_data/execute:deny 3:group@:read_data:allow 4:everyone@:write_data/append_data/write_xattr/execute/write_attributes /write_acl/write_owner:deny 5:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow
The new behavior for a trivial ACL like 644 does not include the deny ACEs. For example:
# ls -v file.1 -rw-r--r-- 1 root root 206663 Jun 22 14:30 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow
ACLs are no longer split into multiple ACEs during inheritance to try to preserve the original unmodified permission. Instead, the permissions are modified as necessary to enforce the file creation mode.
The aclinherit property behavior includes a reduction in permissions when the property is set to restricted, which means that ACLs are no longer split into multiple ACEs during inheritance.
An existing ACL is discarded during chmod(2) operations by default. This change means that the ZFS aclmode property is no longer available.
A new permission mode calculation rule specifies that if an ACL has a user ACE that is also the file owner, then those permissions are included in the permission mode computation. The same rule applies if a group ACE is the group owner of the file.
For more information, see Chapter 8, Using ACLs and Attributes to Protect Oracle Solaris ZFS Files.
Oracle Solaris 10 9/10 Release: In this release, you can use the zpool split command to split a mirrored storage pool, which detaches a disk or disks in the original mirrored pool to create another identical pool.
For more information, see Creating a New Pool By Splitting a Mirrored ZFS Storage Pool.
Oracle Solaris 10 9/10 Release: In this release, each ZFS storage pool has an associated process, zpool-poolname. The threads in this process are the pool's I/O processing threads to handle I/O tasks, such as compression and checksum validation, that are associated with the pool. The purpose of this process is to provide visibility into each storage pool's CPU utilization.
Information about these running processes can be reviewed by using the ps and prstat commands. These processes are only available in the global zone. For more information, see SDC(7).
Oracle Solaris 10 9/10 Release: In this release, the zpool list output has changed to provide better space allocation information. For example:
# zpool list tank NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 136G 55.2G 80.8G 40% ONLINE -
The previous USED and AVAIL fields have been replaced with ALLOC and FREE.
The ALLOC field identifies the amount of physical space allocated to all datasets and internal metadata. The FREE field identifies the amount of unallocated space in the pool.
For more information, see Displaying Information About ZFS Storage Pools.
Oracle Solaris 10 9/10 Release: A storage pool can become damaged if underlying devices become unavailable, if a power failure occurs, or if more than the supported number of devices fail in a redundant ZFS configuration. This release provides new command features for recovering your damaged storage pool. However, using this recovery feature means that the last few transactions that occurred prior to the pool outage might be lost.
Both the zpool clear and zpool import commands support the -F option to possibly recover a damaged pool. In addition, running the zpool status, zpool clear, or zpool import command automatically reports a damaged pool, and these commands describe how to recover the pool.
For more information, see Repairing ZFS Storage Pool-Wide Damage.
Oracle Solaris 10 9/10 Release: The following log device enhancements are available:
The logbias property – You can use this property to instruct ZFS about how to handle synchronous requests for a specific dataset. If logbias is set to latency, ZFS uses the pool's separate log devices, if any, to handle the requests at low latency. If logbias is set to throughput, ZFS does not use the pool's separate log devices. Instead, ZFS optimizes synchronous operations for global pool throughput and efficient use of resources. The default value is latency. For most configurations, the default value is recommended. Using the logbias=throughput value might improve performance for writing database files.
Log device removal – You can now remove a log device from a ZFS storage pool by using the zpool remove command. You can remove a single log device by specifying the device name. You can remove a mirrored log device by specifying the top-level mirror for the log. When you remove a separate log device from the system, ZIL transaction records are written to the main pool.
Redundant top-level virtual devices are now identified with a numeric identifier. For example, in a mirrored storage pool of two disks, the top-level virtual device is mirror-0. This enhancement means that a mirrored log device can be removed by specifying its numeric identifier.
For more information, see Example 4-3.
Oracle Solaris 10 9/10 Release: In this release, a redundant RAID-Z configuration can now have either single-parity, double-parity, or triple-parity protection, which means that one, two, or three device failures can be sustained respectively, without any data loss. You can specify the raidz3 keyword for a triple-parity RAID-Z configuration. For more information, see Creating a RAID-Z Storage Pool.
Oracle Solaris 10 9/10 Release: If you implement different automatic snapshot policies such that older snapshots are being inadvertently destroyed by zfs receive because they no longer exist on the sending side, you might consider using the snapshots hold feature in this release.
Holding a snapshot prevents it from being destroyed. In addition, this feature allows a snapshot with clones to be deleted, pending the removal of the last clone, by using the zfs destroy -d command.
You can hold a snapshot or set of snapshots. For example, the following syntax puts a hold tag, keep, on tank/home/cindy/snap@1:
# zfs hold keep tank/home/cindy@snap1
For more information, see Holding ZFS Snapshots.
Oracle Solaris 10 9/10 Release: In this release, a system event or sysevent is provided when the disks in a pool are replaced with larger disks. ZFS has been enhanced to recognize these events and adjusts the pool based on the new size of the disk, depending on the setting of the autoexpand property. You can use the autoexpand pool property to enable or disable automatic pool expansion when a larger disk replaces a smaller disk.
These enhancements enable you to increase the pool size without having to export and import pool or reboot the system.
For example, automatic LUN expansion is enabled on the tank pool.
# zpool set autoexpand=on tank
Or, you can create the pool with the autoexpand property enabled.
# zpool create -o autoexpand=on tank c1t13d0
The autoexpand property is disabled by default so you can decide whether you want the pool size expanded when a larger disk replaces a smaller disk.
The pool size can also be expanded by using the zpool online -e command. For example:
# zpool online -e tank c1t6d0
Or, you can reset the autoexpand property after a larger disk is attached or made available by using the zpool replace command. For example, the following pool is created with one 8-GB disk (c0t0d0). The 8-GB disk is replaced with a 16-GB disk (c1t13d0), but the pool size is not expanded until the autoexpand property is enabled.
# zpool create pool c0t0d0 # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 8.44G 76.5K 8.44G 0% ONLINE - # zpool replace pool c0t0d0 c1t13d0 # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 8.44G 91.5K 8.44G 0% ONLINE - # zpool set autoexpand=on pool # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 16.8G 91.5K 16.8G 0% ONLINE -
Another way to expand the disk without enabling the autoexpand property, is to use the zpool online -e command even though the device is already online. For example:
# zpool create tank c0t0d0 # zpool list tank NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 8.44G 76.5K 8.44G 0% ONLINE - # zpool replace tank c0t0d0 c1t13d0 # zpool list tank NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 8.44G 91.5K 8.44G 0% ONLINE - # zpool online -e tank c1t13d0 # zpool list tank NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 16.8G 90K 16.8G 0% ONLINE -
Additional device replacement enhancements in this release include the following:
In previous releases, ZFS was unable to replace an existing disk with another disk or attach a disk if the replacement disk was a slightly different size. In this release, you can replace an existing disk with another disk or attach a new disk that is almost the same size provided that the pool is not already full.
In this release, you do not need to reboot the system or export and import a pool to expand the pool size. As described previously, you can enable the autoexpand property or use the zpool online -e command to expand the pool size.
For more information about replacing devices, see Replacing Devices in a Storage Pool.
Solaris 10 10/09 Release: In this release, you can set up a JumpStart profile to identify a flash archive of a ZFS root pool. For more information, see Installing a ZFS Root File System (Oracle Solaris Flash Archive Installation).
Solaris 10 10/09 Release: In previous releases, you could apply quotas and reservations to ZFS file systems to manage and reserve disk space.
In this release, you can set a quota on the amount of disk space consumed by files that are owned by a particular user or group. You might consider setting user and group quotas in an environment with a large number of users or groups.
You can set a user quota by using the zfs userquota property. To set a group quota, use the zfs groupquota property. For example:
# zfs set userquota@user1=5G tank/data # zfs set groupquota@staff=10G tank/staff/admins
You can display a user's or a group's current quota value as follows:
# zfs get userquota@user1 tank/data NAME PROPERTY VALUE SOURCE tank/data userquota@user1 5G local # zfs get groupquota@staff tank/staff/admins NAME PROPERTY VALUE SOURCE tank/staff/admins groupquota@staff 10G local
You can display general quota information as follows:
# zfs userspace tank/data TYPE NAME USED QUOTA POSIX User root 3K none POSIX User user1 0 5G
# zfs groupspace tank/staff/admins TYPE NAME USED QUOTA POSIX Group root 3K none POSIX Group staff 0 10G
You can display an individual user's disk space usage by viewing the userused@user property value. You can display a group's disk space usage by viewing the groupused@group property value. For example:
# zfs get userused@user1 tank/staff NAME PROPERTY VALUE SOURCE tank/staff userused@user1 213M local # zfs get groupused@staff tank/staff NAME PROPERTY VALUE SOURCE tank/staff groupused@staff 213M local
For more information about setting user quotas, see Setting ZFS Quotas and Reservations.
Solaris 10 10/09 Release: In previous releases, you could apply ACL inheritance so that all files were created with 0664 or 0666 permissions. In this release, if you want to optionally include the execute bit from the file creation mode into the inherited ACL, you can set the aclinherit mode to pass the execute permission to the inherited ACL.
If aclinherit=passthrough-x is enabled on a ZFS dataset, you can include the execute permission for an output file that is generated from cc or gcc compiler tools. If the inherited ACL does not include the execute permission, then the executable output from the compiler won't be executable until you use the chmod command to change the file's permissions.
For more information, see Example 8-12.
Solaris 10 10/09 and Oracle Solaris 10 9/10 Releases: The following ZFS file system enhancements are included in these releases.
ZFS Snapshot Stream Property Enhancements – You can set a received property that is different from its local property setting. For example, you might receive a stream with the compression property disabled, but you want compression enabled in the receiving file system. This means that the received stream has a received compression value of off and a local compression value of on. Since the local value overrides the received value, you don't have to worry about the setting on the sending side replacing the received side value. The zfs get command shows the effective value of the compression property under the VALUE column.
New ZFS command options and properties to support send and local property values are as follows:
Use the zfs inherit -S to revert a local property value to the received value, if any. If a property does not have a received value, the behavior of the zfs inherit -S command is the same as the zfs inherit command without the -S option. If the property does have a received value, the zfs inherit command masks the received value with the inherited value until issuing a zfs inherit -S command reverts it to the received value.
You can use the zfs get -o to include the new non-default RECEIVED column. Or, use the zfs get -o all command to include all columns, including RECEIVED.
You can use the zfs send -p option to include properties in the send stream without the -R option.
In addition, you can use the zfs send -e option to use the last element of the sent snapshot name to determine the new snapshot name. The following example sends the poola/bee/cee@1 snapshot to the poold/eee file system and only uses the last element (cee@1) of the snapshot name to create the received file system and snapshot.
# zfs list -rt all poola NAME USED AVAIL REFER MOUNTPOINT poola 134K 134G 23K /poola poola/bee 44K 134G 23K /poola/bee poola/bee/cee 21K 134G 21K /poola/bee/cee poola/bee/cee@1 0 - 21K - # zfs send -R poola/bee/cee@1 | zfs receive -e poold/eee # zfs list -rt all poold NAME USED AVAIL REFER MOUNTPOINT poold 134K 134G 23K /poold poold/eee 44K 134G 23K /poold/eee poold/eee/cee 21K 134G 21K /poold/eee/cee poold/eee/cee@1 0 - 21K -
Setting ZFS file system properties at pool creation time – You can set ZFS file system properties when a storage pool is created. In the following example, compression is enabled on the ZFS file system that is created when the pool is created:
# zpool create -O compression=on pool mirror c0t1d0 c0t2d0
Setting cache properties on a ZFS file system – Two new ZFS file system properties enable you to control what is cached in the primary cache (ARC) and the secondary cache (L2ARC). The cache properties are set as follows:
primarycache – Controls what is cached in the ARC.
secondarycache – Controls what is cached in the L2ARC.
Possible values for both properties – all, none, and metadata. If set to all, both user data and metadata are cached. If set to none, neither user data nor metadata is cached. If set to metadata, only metadata is cached. The default is all.
You can set these properties on an existing file system or when a file system is created. For example:
# zfs set primarycache=metadata tank/datab # zfs create -o primarycache=metadata tank/newdatab
When these properties are set on existing file systems, only new I/O is cache based on the values of these properties.
Some database environments might benefit from not caching user data. You must determine if setting cache properties is appropriate for your environment.
Viewing disk space accounting properties – New read-only file system properties help you identify disk space usage for clones, file systems, and volumes, and snapshots. The properties are as follows:
usedbychildren – Identifies the amount of disk space that is used by children of this dataset, which would be freed if all the dataset's children were destroyed. The property abbreviation is usedchild.
usedbydataset – Identifies the amount of disk space that is used by this dataset itself, which would be freed if the dataset was destroyed, after first destroying any snapshots and removing any refreservation. The property abbreviation is usedds.
usedbyrefreservation – Identifies the amount of disk space that is used by a refreservation set on this dataset, which would be freed if the refreservation was removed. The property abbreviation is usedrefreserv.
usedbysnapshots – Identifies the amount of disk space that is consumed by snapshots of this dataset, which would be freed if all of this dataset's snapshots were destroyed. Note that this is not the sum of the snapshots' used properties, because disk space can be shared by multiple snapshots. The property abbreviation is usedsnap.
These new properties break down the value of the used property into the various elements that consume disk space. In particular, the value of the used property breaks down as follows:
used property = usedbychildren + usedbydataset + usedbyrefreservation + usedbysnapshots
You can view these properties by using the zfs list -o space command. For example:
$ zfs list -o space NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD rpool 25.4G 7.79G 0 64K 0 7.79G rpool/ROOT 25.4G 6.29G 0 18K 0 6.29G rpool/ROOT/snv_98 25.4G 6.29G 0 6.29G 0 0 rpool/dump 25.4G 1.00G 0 1.00G 0 0 rpool/export 25.4G 38K 0 20K 0 18K rpool/export/home 25.4G 18K 0 18K 0 0 rpool/swap 25.8G 512M 0 111M 401M 0
The preceding command is equivalent to the zfs list -o name,avail,used,usedsnap,usedds,usedrefreserv,usedchild -t filesystem,volume command.
Listing snapshots – The listsnapshots pool property controls whether snapshot information is displayed by the zfs list command. The default value is on, which means snapshot information is displayed by default.
If your system has many ZFS snapshots and you wish to disable the display of snapshot information in the zfs list command, disable the listsnapshots property as follows:
# zpool get listsnapshots pool NAME PROPERTY VALUE SOURCE pool listsnapshots on default # zpool set listsnaps=off pool
If you disable the listsnapshots property, you can use the zfs list -t snapshots command to list snapshot information. For example:
# zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT pool/home@today 16K - 22K - pool/home/user1@today 0 - 18K - pool/home/user2@today 0 - 18K - pool/home/user3@today 0 - 18K -
Solaris 10 10/09 Release: In this release, ZFS identifies intent log failures in the zpool status command output. Fault Management Architecture (FMA) reports these errors as well. Both ZFS and FMA describe how to recover from an intent log failure.
For example, if the system shuts down abruptly before synchronous write operations are committed to a pool with a separate log device, you see messages similar to the following:
# zpool status -x pool: pool state: FAULTED status: One or more of the intent logs could not be read. Waiting for adminstrator intervention to fix the faulted pool. action: Either restore the affected device(s) and run 'zpool online', or ignore the intent log records by running 'zpool clear'. scrub: none requested config: NAME STATE READ WRITE CKSUM pool FAULTED 0 0 0 bad intent log mirror ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 c0t4d0 ONLINE 0 0 0 logs FAULTED 0 0 0 bad intent log c0t5d0 UNAVAIL 0 0 0 cannot open
You can resolve the log device failure in the following ways:
Replace or recover the log device. In this example, the log device is c0t5d0.
Bring the log device back online.
# zpool online pool c0t5d0
Reset the failed log device error condition.
# zpool clear pool
To recover from this error without replacing the failed log device, you can clear the error with the zpool clear command. In this scenario, the pool will operate in a degraded mode and the log records will be written to the main pool until the separate log device is replaced.
Consider using mirrored log devices to avoid the log device failure scenario.
Solaris 10 10/09 Release: In this release, when you create a pool, you can specify cache devices, which are used to cache storage pool data.
Cache devices provide an additional layer of caching between main memory and disk. Using cache devices provides the greatest performance improvement for random-read workloads of mostly static content.
One or more cache devices can be specified when the pool is created. For example:
# zpool create pool mirror c0t2d0 c0t4d0 cache c0t0d0 # zpool status pool pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 mirror ONLINE 0 0 0 c0t2d0 ONLINE 0 0 0 c0t4d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 errors: No known data errors
After cache devices are added, they gradually fill with content from main memory. Depending on the size of your cache device, it could take over an hour for the device to fill. Capacity and reads can be monitored by using the zpool iostat command as follows:
# zpool iostat -v pool 5
Cache devices can be added or removed from a pool after the pool is created.
For more information, see Creating a ZFS Storage Pool With Cache Devices and Example 4-4.
Solaris 10 5/09 Release: This release extends support for migrating zones in a ZFS environment with Oracle Solaris Live Upgrade. For more information, see Using Oracle Solaris Live Upgrade to Migrate or Upgrade a System With Zones (at Least Solaris 10 5/09).
For a list of known issues with this release, see the Solaris 10 5/09 release notes.
Solaris 10 10/08 Release: This release enables you to install and boot a ZFS root file system. You can use the initial installation option or the JumpStart feature to install a ZFS root file system. Or, you can use Oracle Solaris Live Upgrade to migrate a UFS root file system to a ZFS root file system. ZFS support for swap and dump devices is also provided. For more information, see Chapter 5, Installing and Booting an Oracle Solaris ZFS Root File System.
For a list of known issues with this release, go to the following site:
http://hub.opensolaris.org/bin/view/Community+Group+zfs/boot
Also, see the Solaris 10 10/08 release notes.
Solaris 10 10/08 Release: This release enables you to roll back a dataset without unmounting it first. Thus, the zfs rollback -f option is no longer needed to force an unmount operation. The -f option is no longer supported and is ignored, if specified.
Solaris 10 10/08 Release: This release includes the following enhancements to the zfs send command. Using this command, you can now perform the following tasks:
Send all incremental streams from one snapshot to a cumulative snapshot. For example:
# zfs list NAME USED AVAIL REFER MOUNTPOINT pool 428K 16.5G 20K /pool pool/fs 71K 16.5G 21K /pool/fs pool/fs@snapA 16K - 18.5K - pool/fs@snapB 17K - 20K - pool/fs@snapC 17K - 20.5K - pool/fs@snapD 0 - 21K - # zfs send -I pool/fs@snapA pool/fs@snapD > /snaps/fs@combo
This syntax sends all incremental snapshots between fs@snapA to fs@snapD to fs@combo.
Send an incremental stream from the original snapshot to create a clone. T he original snapshot must already exist on the receiving side to accept the incremental stream. For example:
# zfs send -I pool/fs@snap1 pool/clone@snapA > /snaps/fsclonesnap-I . . # zfs receive -F pool/clone < /snaps/fsclonesnap-I
Send a replication stream of all descendent file systems, up to the named snapshots. When received, all properties, snapshots, descendent file systems, and clones are preserved. For example:
# zfs send -R pool/fs@snap > snaps/fs-R
For an extended example, see Example 7-1.
Send an incremental replication stream. For example:
# zfs send -R -[iI] @snapA pool/fs@snapD
For an extended example, see Example 7-1.
For more information, see Sending and Receiving Complex ZFS Snapshot Streams.
Solaris 10 10/08 Release : In addition to the existing ZFS quota and reservation features, this release includes dataset quotas and reservations that do not include descendents, such as snapshots and clones, in the disk space accounting.
The refquota property enforces a hard limit on the amount of disk space that a dataset can consume. This hard limit does not include disk space used by descendents, such as snapshots and clones.
The refreservation property sets the minimum amount of disk space that is guaranteed for a dataset, not including its descendents.
For example, you can set a 10-GB refquota limit for studentA that sets a 10-GB hard limit of referenced disk space. For additional flexibility, you can set a 20-GB quota that enables you to manage studentA's snapshots.
# zfs set refquota=10g tank/studentA # zfs set quota=20g tank/studentA
For more information, see Setting ZFS Quotas and Reservations.
Solaris 10 10/08 Release: ZFS storage pool properties were introduced in an earlier release. This release provides two properties, cachefile and failmode.
The following describes the new storage pool properties in this release:
The cachefile property – This property controls where pool configuration information is cached. All pools in the cache are automatically imported when the system boots. However, installation and clustering environments might require this information to be cached in a different location so that pools are not automatically imported.
You can set this property to cache pool configuration in a different location that can be imported later by using the zpool import -c command. For most ZFS configurations, this property would not be used.
The cachefile property is not persistent and is not stored on disk. This property replaces the temporary property that was used to indicate that pool information should not be cached in previous Solaris releases.
The failmode property – This property determines the behavior of a catastrophic pool failure due to a loss of device connectivity or the failure of all devices in the pool. The failmode property can be set to these values: wait, continue, or panic. The default value is wait, which means you must reconnect the device or replace a failed device, and then clear the error with the zpool clear command.
The failmode property is set like other settable ZFS properties, which can be set either before or after the pool is created. For example:
# zpool set failmode=continue tank # zpool get failmode tank NAME PROPERTY VALUE SOURCE tank failmode continue local
# zpool create -o failmode=continue users mirror c0t1d0 c1t1d0
For a description of pool properties, see Table 4-1.
Solaris 10 10/08 Release: The zpool history command has been enhanced to provide the following new features:
ZFS file system event information is now displayed.
The -l option can be used to display a long format that includes the user name, the host name, and the zone in which the operation was performed.
The -i option can be used to display internal event information for diagnostic purposes.
For more information about using the zpool history command, see Resolving Problems With ZFS.
Solaris 10 10/08 Release: The zfs upgrade command is included in this release to provide future ZFS file system enhancements to existing file systems. ZFS storage pools have a similar upgrade feature to provide pool enhancements to existing storage pools.
For example:
# zfs upgrade This system is currently running ZFS filesystem version 3. All filesystems are formatted with the current version.
Note - File systems that are upgraded and any streams created from those upgraded file systems by the zfs send command are not accessible on systems that are running older software releases.
Solaris 10 10/08 Release: In this release, you can grant fine-grained permissions to allow nonprivileged users to perform ZFS administration tasks.
You can use the zfs allow and zfs unallow commands to delegate and remove permissions.
You can modify delegated administration with the pool's delegation property. For example:
# zpool get delegation users NAME PROPERTY VALUE SOURCE users delegation on default # zpool set delegation=off users # zpool get delegation users NAME PROPERTY VALUE SOURCE users delegation off local
By default, the delegation property is enabled.
For more information, see Chapter 9, Oracle Solaris ZFS Delegated Administration and zfs(1M).
Solaris 10 10/08 Release: The ZFS intent log (ZIL) is provided to satisfy POSIX requirements for synchronous transactions. For example, databases often require their transactions to be on stable storage devices when returning from a system call. NFS and other applications can also use fsync() to ensure data stability. By default, the ZIL is allocated from blocks within the main storage pool. In this release, you can decide if you want the ZIL blocks to continue to be allocated from the main storage pool or from a separate log device. Better performance might be possible by using separate intent log devices in your ZFS storage pool, such as with NVRAM or a dedicated disk.
Log devices for the ZFS intent log are not related to database log files.
You can set up a ZFS log device when the storage pool is created or after the pool is created. For examples of setting up log devices, see Creating a ZFS Storage Pool With Log Devices and Adding Devices to a Storage Pool.
You can attach a log device to an existing log device to create a mirrored log device. This operation is identical to attaching a device in a unmirrored storage pool.
Consider the following points when determining whether setting up a ZFS log device is appropriate for your environment:
Any performance improvement seen by implementing a separate log device depends on the device type, the hardware configuration of the pool, and the application workload. For preliminary performance information, see this blog:
http://blogs.oracle.com/perrin/entry/slog_blog_or_blogging_on
Log devices can be unreplicated or mirrored, but RAID-Z is not supported for log devices.
If a separate log device is not mirrored and the device that contains the log fails, storing log blocks reverts to the storage pool.
Log devices can be added, replaced, attached, detached, imported, and exported as part of the larger storage pool. Log devices can be removed starting in the Solaris 10 9/10 release.
The minimum size of a log device is the same as the minimum size of each device in a pool, which is 64 MB. The amount of in-play data that might be stored on a log device is relatively small. Log blocks are freed when the log transaction (system call) is committed.
The maximum size of a log device should be approximately 1/2 the size of physical memory because that is the maximum amount of potential in-play data that can be stored. For example, if a system has 16 GB of physical memory, consider a maximum log device size of 8 GB.
Solaris 10 10/08 Release: You can use the -p option with the zfs create, zfs clone, and zfs rename commands to quickly create a non-existent intermediate dataset, if it doesn't already exist.
In the following example, ZFS datasets (users/area51) are created in the datab storage pool.
# zfs list NAME USED AVAIL REFER MOUNTPOINT datab 106K 16.5G 18K /datab # zfs create -p -o compression=on datab/users/area51
If the intermediate dataset already exists during the create operation, the operation completes successfully.
Properties specified apply to the target dataset, not to the intermediate dataset. For example:
# zfs get mountpoint,compression datab/users/area51 NAME PROPERTY VALUE SOURCE datab/users/area51 mountpoint /datab/users/area51 default datab/users/area51 compression on local
The intermediate dataset is created with the default mount point. Any additional properties are disabled for the intermediate dataset. For example:
# zfs get mountpoint,compression datab/users NAME PROPERTY VALUE SOURCE datab/users mountpoint /datab/users default datab/users compression off default
For more information, see zfs(1M).
Solaris 10 10/08 Release: In this release, ZFS more effectively responds to devices that are removed and can now automatically identify devices that are inserted.
You can replace an existing device with an equivalent device without having to use the zpool replace command.
The autoreplace property controls automatic device replacement. If set to off, device replacement must be initiated by the administrator by using the zpool replace command. If set to on, any new device that is found in the same physical location as a device that previously belonged to the pool is automatically formatted and replaced. The default behavior is off.
The storage pool state REMOVED is provided when a device or hot spare has been physically removed while the system was running. A hot spare device is substituted for the removed device, if available.
If a device is removed and then reinserted, the device is placed online. If a hot spare was activated when the device was reinserted, the hot spare is removed when the online operation completes.
Automatic detection when devices are removed or inserted is hardware-dependent and might not be supported on all platforms. For example, USB devices are automatically configured upon insertion. However, you might have to use the cfgadm -c configure command to configure a SATA drive.
Hot spares are checked periodically to ensure that they are online and available.
For more information, see zpool(1M).
Solaris 10 10/08 Release: You can recursively rename all descendent ZFS snapshots by using the zfs rename -r command. For example:
First, a snapshot of a set of ZFS file systems is created.
# zfs snapshot -r users/home@today # zfs list -t all -r users/home NAME USED AVAIL REFER MOUNTPOINT users/home 2.00G 64.9G 33K /users/home users/home@today 0 - 33K - users/home/mark 1.00G 64.9G 1.00G /users/home/mark users/home/mark@today 0 - 1.00G - users/home/neil 1.00G 64.9G 1.00G /users/home/neil users/home/neil@today 0 - 1.00G -
Then, the snapshots are renamed the following day.
# zfs rename -r users/home@today @yesterday # zfs list -t all -r users/home users/home 2.00G 64.9G 33K /users/home users/home@yesterday 0 - 33K - users/home/mark 1.00G 64.9G 1.00G /users/home/mark users/home/mark@yesterday 0 - 1.00G - users/home/neil 1.00G 64.9G 1.00G /users/home/neil users/home/neil@yesterday 0 - 1.00G -
A snapshot is the only type of dataset that can be renamed recursively.
For more information about snapshots, see Overview of ZFS Snapshots and this blog entry that describes how to create rolling snapshots:
http://blogs.oracle.com/mmusante/entry/rolling_snapshots_made_easy
Solaris 10 10/08 Release: In this release, you can set gzip compression on ZFS file systems, in addition to lzjb compression. You can specify compression as gzip, or gzip-N, where N equals 1 through 9. For example:
# zfs create -o compression=gzip users/home/snapshots # zfs get compression users/home/snapshots NAME PROPERTY VALUE SOURCE users/home/snapshots compression gzip local # zfs create -o compression=gzip-9 users/home/oldfiles # zfs get compression users/home/oldfiles NAME PROPERTY VALUE SOURCE users/home/oldfiles compression gzip-9 local
For more information about setting ZFS properties, see Setting ZFS Properties.
Solaris 10 10/08 Release: As a reliability feature, ZFS file system metadata is automatically stored multiple times across different disks, if possible. This feature is known as ditto blocks.
In this release, you can also store multiple copies of user data, which is also stored per file system by using the zfs set copies command. For example:
# zfs set copies=2 users/home # zfs get copies users/home NAME PROPERTY VALUE SOURCE users/home copies 2 local
Available values are 1, 2, or 3. The default value is 1. These copies are in addition to any pool-level redundancy, such as in a mirrored or RAID-Z configuration.
The benefits of storing multiple copies of ZFS user data are as follows:
Improves data retention by enabling recovery from unrecoverable block read faults, such as media faults (commonly known as bit rot) for all ZFS configurations.
Provides data protection, even when only a single disk is available.
Enables you to select data protection policies on a per-file system basis, beyond the capabilities of the storage pool.
Note - Depending on the allocation of the ditto blocks in the storage pool, multiple copies might be placed on a single disk. A subsequent full disk failure might cause all ditto blocks to be unavailable.
You might consider using ditto blocks when you accidentally create a non-redundant pool and when you need to set data retention policies.
For a detailed description of how storing multiple copies on a system with a single-disk pool or a multiple-disk pool might impact overall data protection, see this blog:
http://blogs.oracle.com/relling/entry/zfs_copies_and_data_protection
For more information about setting ZFS properties, see Setting ZFS Properties.
Solaris 10 8/07 Release: You can use the zpool status -v command to display a list of files with persistent errors. Previously, you had to use the find -inum command to identify the file names from the list of displayed inodes.
For more information about displaying a list of files with persistent errors, see Repairing a Corrupted File or Directory.
Solaris 10 8/07 Release: In this release, you can create a ZFS volume as a Solaris iSCSI target device by setting the shareiscsi property on the ZFS volume. This method is a convenient way to quickly set up a Solaris iSCSI target. For example:
# zfs create -V 2g tank/volumes/v2 # zfs set shareiscsi=on tank/volumes/v2 # iscsitadm list target Target: tank/volumes/v2 iSCSI Name: iqn.1986-03.com.sun:02:984fe301-c412-ccc1-cc80-cf9a72aa062a Connections: 0
After the iSCSI target is created, you can set up the iSCSI initiator. For information about setting up a Solaris iSCSI initiator, see Chapter 14, Configuring Oracle Solaris iSCSI Targets and Initiators (Tasks), in System Administration Guide: Devices and File Systems.
For more information about managing a ZFS volume as an iSCSI target, see Using a ZFS Volume as a Solaris iSCSI Target.
Solaris 10 8/07 Release: In this release, ZFS automatically logs successful zfs and zpool commands that modify pool state information. For example:
# zpool history History for 'newpool': 2007-04-25.11:37:31 zpool create newpool mirror c0t8d0 c0t10d0 2007-04-25.11:37:46 zpool replace newpool c0t10d0 c0t9d0 2007-04-25.11:38:04 zpool attach newpool c0t9d0 c0t11d0 2007-04-25.11:38:09 zfs create newpool/user1 2007-04-25.11:38:15 zfs destroy newpool/user1 History for 'tank': 2007-04-25.11:46:28 zpool create tank mirror c1t0d0 c2t0d0 mirror c3t0d0 c4t0d0
This features enables you or Oracle support personnel to identify the actual ZFS commands that were executed to troubleshoot an error scenario.
You can identify a specific storage pool with the zpool history command. For example:
# zpool history newpool History for 'newpool': 2007-04-25.11:37:31 zpool create newpool mirror c0t8d0 c0t10d0 2007-04-25.11:37:46 zpool replace newpool c0t10d0 c0t9d0 2007-04-25.11:38:04 zpool attach newpool c0t9d0 c0t11d0 2007-04-25.11:38:09 zfs create newpool/user1 2007-04-25.11:38:15 zfs destroy newpool/user1
In this release, the zpool history command does not record user-ID, hostname, or zone-name. However, this information is recorded starting in the Solaris 10 10/08 release. For more information, see ZFS Command History Enhancements (zpool history).
For more information about troubleshooting ZFS problems, see Resolving Problems With ZFS.
Solaris 10 8/07 Release: You can use the xattr property to disable or enable extended attributes for a specific ZFS file system. The default value is on. For a description of ZFS properties, see Introducing ZFS Properties.
Solaris 10 8/07 Release: The new canmount property enables you to specify whether a dataset can be mounted by using the zfs mount command. For more information, see The canmount Property.
Solaris 10 8/07 Release: In addition to the standard native properties that can be used to either export internal statistics or control ZFS file system behavior, ZFS provides user properties. User properties have no effect on ZFS behavior, but you can use them to annotate datasets with information that is meaningful in your environment.
For more information, see ZFS User Properties.
Solaris 10 8/07 Release: In this release, you can set properties when you create a file system, not just after the file system is created.
The following examples illustrate equivalent syntax:
# zfs create tank/home # zfs set mountpoint=/export/zfs tank/home # zfs set quota=50g tank/home # zfs set compression=on tank/home
# zfs create -o mountpoint=/export/zfs -o quota=50g -o compression=on tank/home
Solaris 10 8/07 Release: In this release, you can use various forms of the zfs get command to display information about all datasets if you do not specify a dataset or if you specify all. In previous releases, all dataset information was not retreivable with the zfs get command.
For example:
# zfs get -r -s local all tank/home NAME PROPERTY VALUE SOURCE tank/home atime off local tank/home/jeff atime off local tank/home/mark quota 50G local
Solaris 10 8/07 Release: In this release, you can use the new -F option to the zfs receive command to force a rollback of the file system to the most recent snapshot before the receive is initiated. Using this option might be necessary when the file system is modified after a rollback occurs but before the receive is initiated.
For more information, see Receiving a ZFS Snapshot.
Solaris 10 11/06 Release: When you use the zfs snapshot command to create a file system snapshot, you can use the -r option to recursively create snapshots for all descendent file systems. In addition, you can use the -r option to recursively destroy all descendent snapshots when a snapshot is destroyed.
Recursive ZFS snapshots are created quickly as one atomic operation. The snapshots are created together (all at once) or not created at all. The benefit of such an operation is that the snapshot data is always taken at one consistent time, even across descendent file systems.
For more information, see Creating and Destroying ZFS Snapshots.
Solaris 10 11/06 Release: A redundant RAID-Z configuration can now have either a single- or double-parity configuration, which means that one or two device failures, respectively, can be sustained, without any data loss. You can specify the raidz2 keyword for a double-parity RAID-Z configuration. Or, you can specify the raidz or raidz1 keyword for a single-parity RAID-Z configuration.
For more information, see Creating a RAID-Z Storage Pool or zpool(1M).
Solaris 10 11/06 Release: The ZFS hot spares feature enables you to identify disks that could be used to replace a failed or faulted device in one or more storage pools. Designating a device as a hot spare means that if an active device in the pool fails, the hot spare automatically replaces the failed device. Or, you can manually replace a device in a storage pool with a hot spare.
For more information, see Designating Hot Spares in Your Storage Pool and zpool(1M).
Solaris 10 11/06 Release: The zfs promote command enables you to replace an existing ZFS file system with a clone of that file system. This feature is helpful when you want to run tests on an alternative version of a file system and then make that alternative version the active file system.
For more information, see Replacing a ZFS File System With a ZFS Clone and zfs(1M).
Solaris 10 6/06 Release: You can upgrade your storage pools to a newer version of ZFS to take advantage of the latest features by using the zpool upgrade command. In addition, the zpool status command has been modified to notify you when your pools are running older versions of ZFS.
For more information, see Upgrading ZFS Storage Pools and zpool(1M).
If you want to use the ZFS Administration console on a system with a pool from a previous Solaris release, ensure that you upgrade your pools before using the console. To determine if your pools need to be upgraded, use the zpool status command. For information about the ZFS Administration console, see ZFS Web-Based Management.
Solaris 10 6/06 Release: In this release, the zfs backup and zfs restore commands are renamed to zfs send and zfs receive to more accurately describe their functions. These commands send and receive ZFS data stream representations.
For more information about these commands, see Sending and Receiving ZFS Data.
Solaris 10 6/06 Release: This release includes the zpool import -D command, which enables you to recover pools that were previously destroyed with the zpool destroy command.
For more information, see Recovering Destroyed ZFS Storage Pools.
Solaris 10 6/06 Release: This release includes a ZFS diagnostic engine that is capable of diagnosing and reporting pool failures and device failures. Checksum, I/O, device, and pool errors associated with pool or device failures are also reported.
The diagnostic engine does not include predictive analysis of checksum and I/O errors, nor does it include proactive actions based on fault analysis.
If a ZFS failure occurs, you might see a message similar to the following:
SUNW-MSG-ID: ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major EVENT-TIME: Wed Jun 30 14:53:39 MDT 2010 PLATFORM: SUNW,Sun-Fire-880, CSN: -, HOSTNAME: neo SOURCE: zfs-diagnosis, REV: 1.0 EVENT-ID: 504a1188-b270-4ab0-af4e-8a77680576b8 DESC: A ZFS device failed. Refer to http://sun.com/msg/ZFS-8000-D3 for more information. AUTO-RESPONSE: No automated response will occur. IMPACT: Fault tolerance of the pool may be compromised. REC-ACTION: Run 'zpool status -x' and replace the bad device.
By reviewing the recommended action, which is to follow the more specific directions in the zpool status command, you can quickly identify and resolve the failure.
For an example of recovering from a reported ZFS problem, see Resolving a Missing Device.
Solaris 10 6/06 Release: This release includes the zpool clear command for clearing error counts associated with a device or a pool. Previously, error counts were cleared when a device in a pool was brought online with the zpool online command. For more information, see Clearing Storage Pool Device Errors and zpool(1M).
Solaris 10 6/06 Release: In this release, you can set and display NFSv4 ACLs in two formats: verbose and compact. You can use the chmod command to set either ACL formats. You can use the ls -V command to display the compact ACL format. You can use the ls -v command to display the verbose ACL format.
For more information, see Setting and Displaying ACLs on ZFS Files in Compact Format, chmod(1), and ls(1).
Solaris 10 6/06 Release: A new file system monitoring tool, fsstat, reports file system operations. Activity can be reported by mount point or by file system type. The following example shows general ZFS file system activity:
$ fsstat zfs new name name attr attr lookup rddir read read write write file remov chng get set ops ops ops bytes ops bytes 7.82M 5.92M 2.76M 1.02G 3.32M 5.60G 87.0M 363M 1.86T 20.9M 251G zfs
For more information, see fsstat(1M).
Solaris 10 6/06 Release: A web-based ZFS management tool, the ZFS Administration console, enables you to perform the following administrative tasks:
Create a new storage pool.
Add capacity to an existing pool.
Move (export) a storage pool to another system.
Import a previously exported storage pool to make it available on another system.
View information about storage pools.
Create a file system.
Create a volume.
Create a snapshot of a file system or a volume.
Roll back a file system to a previous snapshot.
You can access the ZFS Administration console through a secure web browser at:
https://system-name:6789/zfs
If you type the appropriate URL and are unable to reach the ZFS Administration console, the server might not be started. To start the server, run the following command:
# /usr/sbin/smcwebserver start
If you want the server to run automatically when the system boots, run the following command:
# /usr/sbin/smcwebserver enable
Note - You cannot use the Solaris Management Console (smc) to manage ZFS storage pools or file systems.