Solaris ZFS Administration Guide

Setting ZFS Quotas and Reservations

You can use the quota property to set a limit on the amount of space a file system can use. In addition, you can use the reservation property to guarantee that some amount of space is available to a file system. Both properties apply to the dataset they are set on and all descendents of that dataset.

That is, if a quota is set on the tank/home dataset, the total amount of space used by tank/home and all of its descendents cannot exceed the quota. Similarly, if tank/home is given a reservation, tank/home and all of its descendents draw from that reservation. The amount of space used by a dataset and all of its descendents is reported by the used property.

The refquota and refreservation properties are available to manage file system space without accounting for space consumed by descendents, such as snapshots and clones.

In this Solaris release, you can set a user or group quota on the amount of space consumed by files that are owned by a particular user or group. The user and group quota properties cannot be set on a volume, on a file system before file system version 4, or on a pool before pool version 15.

Consider the following points to determine which quota and reservations features might better manage your file systems:

The quota and reservation properties are convenient for managing space consumed by datasets.
The refquota and refreservation properties are appropriate for managing space consumed by descendents.
Setting refquota or refreservation higher than quota or reservation has no effect. If you set the quota or refquota properties, operations that try to exceed either value fail. It is possible to a exceed a quota that is greater than refquota. For example, if some snapshot blocks are modified, you might actually exceed the quota before you exceed the refquota.
User and group quotas provide a way to more easily manage disk space with many user accounts, such as in a university environment.

For more information about setting quotas and reservations, see Setting Quotas on ZFS File Systemsand Setting Reservations on ZFS File Systems.

Setting Quotas on ZFS File Systems

ZFS quotas can be set and displayed by using the zfs set and zfs get commands. In the following example, a quota of 10 Gbytes is set on tank/home/bonwick.

# zfs set quota=10G tank/home/bonwick
# zfs get quota tank/home/bonwick
NAME              PROPERTY      VALUE                      SOURCE
tank/home/bonwick quota         10.0G                      local

ZFS quotas also impact the output of the zfs list and df commands. For example:

# zfs list
NAME                   USED  AVAIL  REFER  MOUNTPOINT
tank/home             16.5K  33.5G  8.50K  /export/home
tank/home/bonwick     15.0K  10.0G  8.50K  /export/home/bonwick
tank/home/bonwick/ws  6.50K  10.0G  8.50K  /export/home/bonwick/ws
# df -h /export/home/bonwick
Filesystem             size   used  avail capacity  Mounted on
tank/home/bonwick       10G     8K    10G     1%    /export/home/bonwick

Note that although tank/home has 33.5 Gbytes of space available, tank/home/bonwick and tank/home/bonwick/ws only have 10 Gbytes of space available, due to the quota on tank/home/bonwick.

You cannot set a quota to an amount less than is currently being used by a dataset. For example:

# zfs set quota=10K tank/home/bonwick
cannot set quota for 'tank/home/bonwick': size is less than current used or 
reserved space

You can set a refquota on a dataset that limits the amount of space that the dataset can consume. This hard limit does not include space that is consumed by descendents. For example:

# zfs set refquota=10g students/studentA
# zfs list
NAME                USED  AVAIL  REFER  MOUNTPOINT
profs               106K  33.2G    18K  /profs
students           57.7M  33.2G    19K  /students
students/studentA  57.5M  9.94G  57.5M  /students/studentA
# zfs snapshot students/studentA@today
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                     106K  33.2G    18K  /profs
students                 57.7M  33.2G    19K  /students
students/studentA        57.5M  9.94G  57.5M  /students/studentA
students/studentA@today      0      -  57.5M  -

For additional convenience, you can set another quota on a dataset to help manage the space that is consumed by snapshots. For example:

# zfs set quota=20g students/studentA
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                     106K  33.2G    18K  /profs
students                 57.7M  33.2G    19K  /students
students/studentA        57.5M  9.94G  57.5M  /students/studentA
students/studentA@today      0      -  57.5M  -

In this scenario, studentA might reach the refquota (10 Gbytes) hard limit, but can remove files to recover, even if snapshots exist.

In the above example, the smaller of the two quotas (10 Gbytes versus 20 Gbytes) is displayed in the zfs list output. To see the value of both quotas, use the zfs get command. For example:

# zfs get refquota,quota students/studentA
NAME               PROPERTY  VALUE              SOURCE
students/studentA  refquota  10G                local
students/studentA  quota     20G                local

Setting User or Group Quotas on a ZFS File System

You can set a user or a group quota by using the zfs userquota and zfs groupquota commands as follows:

# zfs create students/compsci
# zfs set userquota@student1=10G students/compsci
# zfs create students/labstaff
# zfs set groupquota@staff=20GB students/labstaff

Display the current user quota or group quota as follows:

# zfs get userquota@student1 students/compsci
NAME              PROPERTY            VALUE               SOURCE
students/compsci  userquota@student1  10G                 local
# zfs get groupquota@staff students/labstaff
NAME               PROPERTY          VALUE             SOURCE
students/labstaff  groupquota@staff  20G               local

You can display general user and group space usage by querying the following properties:

# zfs userspace students/compsci
TYPE        NAME      USED  QUOTA  
POSIX User  root      227M   none  
POSIX User  student1  455M    10G  
# zfs groupspace students/labstaff
TYPE         NAME   USED  QUOTA  
POSIX Group  root   217M   none  
POSIX Group  staff  217M    20G

If you want to identify individual user or group space usage, query the following properties:

# zfs get userused@student1 students/compsci
NAME              PROPERTY           VALUE              SOURCE
students/compsci  userused@student1  455M               local
# zfs get groupused@staff students/labstaff
NAME               PROPERTY         VALUE            SOURCE
students/labstaff  groupused@staff  217M             local

The user and group quota properties are not displayed by using the zfs get all dataset command that displays a listing of all file system properties.

You can remove a user or group quota as follows:

# zfs set userquota@user1=none students/compsci
# zfs set groupquota@staff=none students/labstaff

ZFS user and group quotas provide the following features:

When set on a file system, a user or group quota that is set on a parent file system is not automatically inherited by a descendent file system.
However, the user or group quota is applied when a clone or a snapshot is created from a file system that has a user or group quota. Likewise, a user or group quota is included with the file system when a stream is created by using the zfs send command, even without -R option.
Unprivileged users can only access their own space usage. The root user or a user who has been granted the userused or groupused privilege, can access everyone's user or group space accounting information.
The userquota and groupquota properties cannot be set on ZFS volumes, on a file system prior to file system version 4, or on a pool prior to pool version 15.

Enforcement of user or group quotas might be delayed by several seconds. This delay means that users might exceed their quota before the system notices that they are over quota and refuses additional writes with the EDQUOT error message.

You can use the legacy quota command to review user quotas in an NFS environment, for example, where a ZFS file system is mounted. Without any options, the quota command only displays output if the user's quota is exceeded. For example:

# zfs set userquota@student1=10m students/compsci   
# zfs userspace students/compsci
TYPE        NAME      USED  QUOTA  
POSIX User  root      227M   none  
POSIX User  student1  455M    10M  
# quota student1
Block limit reached on /students/compsci

If you reset the quota and the quota limit is no longer exceeded, you will need to use the quota -v command to review the user's quota. For example:

# zfs set userquota@student1=10GB students/compsci 
# zfs userspace students/compsci
TYPE        NAME      USED  QUOTA  
POSIX User  root      227M   none  
POSIX User  student1  455M    10G  
# quota student1
# quota -v student1
Disk quotas for student1 (uid 201):
Filesystem     usage  quota  limit    timeleft  files  quota  limit    timeleft
/students/compsci
              466029 10485760 10485760

Setting Reservations on ZFS File Systems

A ZFS reservation is an allocation of space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve space for a dataset if that space is not currently available in the pool. The total amount of all outstanding unconsumed reservations cannot exceed the amount of unused space in the pool. ZFS reservations can be set and displayed by using the zfs set and zfs get commands. For example:

# zfs set reservation=5G tank/home/moore
# zfs get reservation tank/home/moore
NAME             PROPERTY     VALUE   SOURCE
tank/home/moore  reservation  5G      local

ZFS reservations can affect the output of the zfs list command. For example:

# zfs list
NAME                   USED  AVAIL  REFER  MOUNTPOINT
tank/home             5.00G  33.5G  8.50K  /export/home
tank/home/moore       15.0K  33.5G  8.50K  /export/home/moore

Note that tank/home is using 5 Gbytes of space, although the total amount of space referred to by tank/home and its descendents is much less than 5 Gbytes. The used space reflects the space reserved for tank/home/moore. Reservations are considered in the used space of the parent dataset and do count against its quota, reservation, or both.

# zfs set quota=5G pool/filesystem
# zfs set reservation=10G pool/filesystem/user1
cannot set reservation for 'pool/filesystem/user1': size is greater than 
available space

A dataset can use more space than its reservation, as long as space is available in the pool that is unreserved and the dataset's current usage is below its quota. A dataset cannot consume space that has been reserved for another dataset.

Reservations are not cumulative. That is, a second invocation of zfs set to set a reservation does not add its reservation to the existing reservation. Rather, the second reservation replaces the first reservation.

# zfs set reservation=10G tank/home/moore
# zfs set reservation=5G tank/home/moore
# zfs get reservation tank/home/moore
NAME             PROPERTY      VALUE                      SOURCE
tank/home/moore  reservation   5.00G                      local

You can set a refreservation to guarantee space for a dataset that does not include space consumed by snapshots and clones. The refreservation reservation is accounted for in the parent datasets' space used, and counts against the parent datasets' quotas and reservations. For example:

# zfs set refreservation=10g profs/prof1
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                    10.0G  23.2G    19K  /profs
profs/prof1                10G  33.2G    18K  /profs/prof1

You can also set a reservation on the same dataset to guarantee dataset space and snapshot space. For example:

# zfs set reservation=20g profs/prof1
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                    20.0G  13.2G    19K  /profs
profs/prof1                10G  33.2G    18K  /profs/prof1

Regular reservations are accounted for in the parent's used space.

In the above example, the smaller of the two quotas (10 Gbytes versus 20 Gbytes) is displayed in the zfs list output. To see the value of both quotas, use the zfs get command. For example:

# zfs get reservation,refreserv profs/prof1
NAME         PROPERTY        VALUE        SOURCE
profs/prof1  reservation     20G          local
profs/prof1  refreservation  10G          local

If refreservation is set, a snapshot is only allowed if enough free pool space exists outside of this reservation to accommodate the current number of referenced bytes in the dataset.