Sun QFS File Systems Design Basics

Sun QFS file systems are multi threaded, advanced storage management systems. To take maximum advantage of the software's capabilities, create multiple file systems whenever possible.

Sun QFS file systems use a linear search method for directory lookups, searching from the beginning of the directory to the end. As the number of files in a directory increases, the search time through the directory also increases. Search times can become excessive when you have directories with thousands of files. These long search times are also evident when you restore a file system. To increase performance and speed up file system dumps and restores, keep the number of files in a directory under 10,000.

The directory name lookup cache (DNLC) feature improves file system performance. This cache stores the directory lookup information for files whose paths are short (30 characters or less), removing the need for directory lookups to be performed on the fly.

Inode Files and File Characteristics

The types of files to be stored in a file system affect file system design. An inode is a 512-byte block of information that describes the characteristics of a file or directory. This information is allocated dynamically within the file system.

Inodes are stored in the .inodes file located under the file system mount point. Like a standard Oracle Solaris OS inode, a Sun QFS file system inode contains the file's POSIX standard inode times: file access, file modification, and inode changed times. A Sun QFS file system inode includes other times as well, as shown in the following table.

Table 1-1 Content of .inode Files

.

Specifying Disk Allocation Units

Disk space is allocated in basic units of online disk storage called disk allocation units (DAUs). Whereas sectors, tracks, and cylinders describe the physical disk geometry, the DAU describes the file system geometry. Choosing the appropriate DAU size and stripe size can improve performance and optimize magnetic disk usage. The DAU setting is the minimum amount of contiguous space that is used when a file is allocated.

DAU Settings and File System Geometry

Sun QFS file systems use an adjustable DAU. You can configure the DAU to tune the file system to the physical disk storage device. This feature minimizes the system overhead caused by read-modify-write operations and is therefore particularly useful for applications that manipulate very large files. For information about how to control the read-modify-write operation, see Increasing File Transfer Performance for Large Files.

Each file system can have its own unique DAU setting even if it is one of several mounted file systems active on a server. The possible DAU settings differ depending on the type of file system you are using. The DAU setting is determined through sammkfs(1M) in Sun QFS and Sun Storage Archive Manager 5.3 Reference Manual when the file system is created. It cannot be changed dynamically.

DAU settings work in conjunction with the device and file system definitions specified in the master configuration (mcf) file. For details about the mcf file, see Chapter 2, About the Master Configuration File and mcf(4) in Sun QFS and Sun Storage Archive Manager 5.3 Reference Manual.

ms and ma File Systems

Two file allocation schemes are available to you:

• An ms file system type – File system data and file system metadata are on the same device

• An ma file system type – File system data and file system metadata are on different devices

For a simple Sun QFS file system, such as one on a single partition, the file system is defined in your mcf file by an Equipment Type value of ms. In the ms file system, the only device type allowed is type md, and both metadata and file data are written to the md devices. By default, the DAU on an md device is 64 kilobytes.

A more complex Sun QFS file system installed on multiple partitions is defined as Equipment Type ma in your mcf file. In an ma file system, metadata is written to mm devices, and data can be written to md, mr, or g XXX devices.

Within an ma file system you can mix devices as follows:

• mm and mr devices

• mm and g XXX devices

• mm, mr, and g XXX devices

• mm and md devices

For more information about these device types, see Chapter 2, About the Master Configuration File.

Dual and Single Allocation Schemes

The md and mm devices use a dual allocation scheme, as follows:

• On md data devices, the small allocation is 4 kilobytes, and the large allocation is a DAU. The default DAU is 64 kilobytes. You can override this default when the file system is initialized by using the -aallocation-unit option to the sammkfs command. The DAU size can be 16, 32, or 64 kilobytes.

• When a file is created on an md device, the system allocates the first eight addresses of the file in the small allocation. If more space is needed, the file system uses one or more large allocations (DAUs) to expand the file. As a result, I/O performance improves for large files while minimizing the disk fragmentation that can result from many small files.

• On mm metadata devices, the small allocation is 4 kilobytes, and the large allocation is 16 kilobytes. The dual allocation scheme enables the file system to write metadata to disk more efficiently and helps minimize disk fragmentation.

Depending on the type of file data stored in the file system, a larger DAU size can improve file system performance significantly. For information about tuning file system performance, see Chapter 7, Advanced File System Topics.

Only ma Sun QFS file systems can include devices that use a single allocation scheme. These file systems consist of separate metadata devices and data devices, as follows:

• The metadata devices can be defined only as Equipment Type mm.

• The data devices can be defined as Equipment Type md, mr, or g XXX. The md devices are limited to DAU sizes of 16 kilobytes, 32 kilobytes, or 64 kilobytes.

  The mr and g XXX devices follow a single allocation scheme. You can mix mr and g XXX devices in a file system, but you cannot mix md devices with either mr or g XXX devices in a file system. The mr and gXXX devices can be set to a minimum DAU allocation of 8 kilobytes for devices in an ma file system. This setting is optimal for workloads with the majority of file sizes at or below 8 kilobytes.

The DAU size for file systems that use mr and g XXX data devices is configurable. The possible DAU sizes that can be used on data devices depend on the Equipment Type value assigned to each data device in the mcf file. The following table shows these DAU sizes.

Table 1-2 Equipment Type Values and DAU Sizes

Data Alignment

Data alignment refers to matching the allocation unit of the RAID controller with the allocation unit of the file system. The optimal file system alignment formula is as follows:

allocation-unit = RAID-stripe-width x number-of-data-disks

For example, suppose a RAID-5 unit has nine disks, with one of the nine being the parity disk, making the number of data disks eight. If the RAID stripe width is 64 kilobytes, then the optimal allocation unit is 64 multiplied by 8, which is 512 kilobytes.

Data files are allocated as striped or round-robin through each striped group (gXXX) or data disk (mr or md) within the same file system.

A mismatched alignment hurts performance because it can cause a read-modify-write operation.

Stripe Widths on Data Disks

Stripe width defaults differ between ms and ma file systems. The stripe width is specified by the -o stripe= n option in the mount command. If the stripe width is set to 0, round-robin allocation is used.

Stripe Widths on ms File Systems

On ms file systems, the stripe width is set at mount time. The following table shows default stripe widths.

Table 1-3 ms File System Default Stripe Widths

For example, if you run sammkfs command with default settings, the default large DAU is 64 kilobytes. If no stripe width is specified when the mount command is issued, the default is used, and the stripe width set at mount time is 2.

Stripe Widths on ma File Systems Not Using Striped Groups

On ma file systems, the stripe width that is set at mount time depends on whether striped groups are configured. A striped group is a collection of devices that are striped as a group. For more information about striped groups, see File Allocation Methods. This section describes stripe widths for Sun QFS file systems that are configured without stripe groups.

If striped groups are not configured, the DAU and stripe width relationships on ma file systems are similar to those for ms file systems. The difference is that DAUs larger than 64 kilobytes are possible and that the DAU is configurable in 8-kilobyte blocks. The maximum DAU size is 65,528 kilobytes.

By default, if no stripe width is specified, the amount of data written to disk is at or near 128 kilobytes. Sun QFS file systems are most efficient if write operations write at least one whole stripe per I/O request. The following table shows the default stripe widths.

Table 1-4 Default Stripe Widths for ma File Systems Not Using Striped Groups

Stripe Widths on ma File Systems Using Striped Groups

If striped groups are configured for your file system, the minimum amount of space allocated is the DAU multiplied by the number of devices in the striped group. The amount of the allocation can be very large with striped groups.

When striped groups are used, data is written to several disk devices at once, as if they were one device. Allocations on striped groups are equal to the DAU size multiplied by the number of elements in the striped group.

The -o stripe= n mount option determines the number of allocations that occur on each stripe group before the allocation moves to a different striped group. If a file system is mounted with -o stripe=0, the allocation is always to one striped group.

By default, the setting is -o stripe=0, which specifies the round-robin allocation method. The setting can be as low as -o stripe=0 (which disables striping) or as high as -o stripe=255. The system sets -o stripe=0 if mismatched striped groups are present, in which case a file can reside on only one striped group.

For more information, see File Allocation Methods.

Stripe Widths on Metadata Disks

You can use the -o mm_stripe= n option to the mount_samfs command to stripe metadata information on the metadata disk. The default stripe width is -o mm_stripe=1, which specifies that the file system write one 16-kilobyte DAU to a metadata disk before switching to the next metadata disk. The small 4-kilobyte DAU is used for metadata disks.

By default, if you have multiple metadata devices, metadata is allocated as specified in the - o mm_stripe= n option to the mount command. The setting can be as low as -o mm_stripe=0, which disables striping, or as high as -o mm_stripe=255.