A simple metadevice is a metadevice built only from slices, and is either used directly or as the basic building block for mirrors and trans metadevices. There are three kinds of simple metadevices: concatenated metadevices, striped metadevices, and concatenated striped metadevices.
In practice, people tend to think of two basic simple metadevices: concatenated metadevices and striped metadevices. (A concatenated stripe is simply a striped metadevice that has been "grown" from its original configuration by concatenating slices.)
Simple metadevices enable you to quickly and simply expand disk storage capacity. The drawback to a simple metadevice is that it does not provide any data redundancy. A mirror or RAID5 metadevice can provide data redundancy. (If a single slice fails on a simple metadevice, data is lost.)
You can use a simple metadevice containing multiple slices for any file system except the following:
Root (/)
/usr
swap
/var
/opt
Any file system accessed during an operating system upgrade or installation
When you mirror root (/), /usr, swap, /var, or /opt, you put the file system into a one-way concatenation (a concatenation of a single slice) that acts as a submirror. This is mirrored by another submirror, which is also a concatenation.
A concatenated metadevice, or concatenation, is a metadevice whose data is organized serially and adjacently across disk slices, forming one logical storage unit.
You would use a concatenated metadevice to get more storage capacity by logically combining the capacities of several slices. You can add more slices to the concatenated metadevice as the demand for storage grows.
A concatenated metadevice enables you to dynamically expand storage capacity and file system sizes online. With a concatenated metadevice you can add slices even if the other slices are currently active.
To increase the capacity of a striped metadevice, you would have to build a concatenated stripe (see "Concatenated Stripe").
A concatenated metadevice can also expand any active and mounted UFS file system without having to bring down the system. In general, the total capacity of a concatenated metadevice is equal to the total size of all the slices in the concatenated metadevice. If a concatenation contains a slice with a state database replica, the total capacity of the concatenation would be the sum of the slices less the space reserved for the replica.
You can also create a concatenated metadevice from a single slice. You could, for example, create a single-slice concatenated metadevice. Later, when you need more storage, you can add more slices to the concatenated metadevice.
Concatenations have names like other metadevices (d0, d1, and so forth). For more information on metadevice naming, see Table 1-4.
When would I create a concatenated metadevice?
To expand the capacity of an existing data set, such as a file system.
Concatenation is good for small random I/O and for even I/O distribution.
What are the limitations to concatenation?
Practically speaking, none. You must use a concatenation to encapsulate root (/), swap, /usr, /opt, or /var when mirroring these file systems.
How large can a concatenated metadevice be?
Up to one Terabyte.
Figure 2-1 illustrates a concatenated metadevice made of three slices (disks).
The data blocks, or chunks, are written sequentially across the slices, beginning with Disk A. Disk A can be envisioned as containing logical chunks 1 through 4. Logical chunk 5 would be written to Disk B, which would contain logical chunks 5 through 8. Logical chunk 9 would be written to Drive C, which would contain chunks 9 through 12. The total capacity of metadevice d1 would be the combined capacities of the three drives. If each drive were 2 Gbytes, metadevice d1 would have an overall capacity of 6 Gbytes.
A striped metadevice, or stripe, is a metadevice that arranges data across two or more slices. Striping alternates equally-sized segments of data across two or more slices, forming one logical storage unit. These segments are interleaved round-robin, so that the combined space is made alternately from each slice, in effect, shuffled like a deck of cards.
Sometimes a striped metadevice is called a "stripe." Other times, "stripe" refers to the component blocks of a striped concatenation. "To stripe" means to spread I/O requests across disks by chunking parts of the disks and mapping those chunks to a virtual device (a metadevice). Striping is also classified as RAID level 0, as is concatenation.
While striping and concatenation both are methods of distributing data across disk slices, striping alternates chunks of data across disk slices, while concatenation distributes data "end-to-end" across disk slices.
For sequential I/O operations on a concatenated metadevice, DiskSuite reads all the blocks on the first slice, then all the blocks of the second slice, and so forth.
For sequential I/O operations on a striped metadevice, DiskSuite reads all the blocks in a segment of blocks (called an interlace) on the first slice, then all the blocks in a segment of blocks on the second slice, and so forth.
On both a concatenation and a striped metadevice, all I/O occurs in parallel.
Why would I create a striped metadevice?
To take advantage of the performance increases that come from accessing data in parallel and to increase capacity. Always use striped metadevices for new file systems or data sets.
Striping enables multiple controllers to access data at the same time (parallel access). Parallel access can increase I/O throughput because all disks in the metadevice are busy most of the time servicing I/O requests.
Striping is good for large sequential I/O and for uneven I/O.
What are the limitations to striping?
An existing file system cannot be directly converted to a striped metadevice. If you need to place a file system on a striped metadevice, you can back up the file system, create a striped metadevice, then restore the file system to the striped metadevice.
When creating a stripe, do not use slices of unequal size, as this will result in unused disk space.
The size, in Kbytes, Mbytes, or blocks, of the logical data chunks in a striped metadevice. Depending on the application, different interlace values can increase performance for your configuration. The performance increase comes from several disk arms doing I/O. When the I/O request is larger than the interlace size, you may get better performance.
What is DiskSuite's default interlace value?
16 Kbytes.
Can I set the interlace value?
Yes, when you create a new striped metadevice, using either the command line or DiskSuite Tool. Once you have created the striped metadevice, you cannot change the interlace value.
Can I set the interlace value on an existing striped metadevice?
No. (Though you could back up the data on it, delete the striped metadevice, create a new striped metadevice with a new interlace value, and then restore the data.)
RAID5 metadevices also use an interlace value. See "RAID5 Metadevices" for more information.
Figure 2-2 shows a striped metadevice built from three slices (disks).
When DiskSuite stripes data from the metadevice to the slices, it writes data from chunk 1 to Disk A, from chunk 2 to Disk B, and from chunk 3 to Disk C. DiskSuite then writes chunk 4 to Disk A, chunk 5 to Disk B, chunk 6 to Disk C, and so forth.
The interlace value sets the size of each chunk. The total capacity of the striped metadevice d2 equals the number of slices multiplied by the size of the smallest slice. (If each slice in the example below were 2 Gbytes, d2 would equal 6 Gbytes.)
A concatenated stripe is a striped metadevice that has been expanded by concatenating additional slices (stripes).
Why would I use a concatenated stripe?
This is the only way to expand an existing striped metadevice.
If you use DiskSuite Tool to drag multiple slices into an existing striped metadevice, you are given the optional of making the slices into a concatenation or a stripe. If you use the metattach(1M) command to add multiple slices to an existing striped metadevice, they must be added as a stripe.
How do I set up the interlace value for a concatenated stripe?
At the stripe level, using either the Stripe Information window in DiskSuite Tool, or the -i option to the metattach(1M) command. Each stripe within the concatenated stripe can have its own interlace value. When you create a concatenated stripe from scratch, if you do not specify an interlace value for a particular stripe, it inherits the interlace value from the stripe before it.
Figure 2-3 illustrates that d10 is a concatenation of three stripes.
The first stripe consists of three slices, Disks A through C, with an interlace of 16 Kbytes. The second stripe consists of two slices Disks D and E, and uses an interlace of 32 Kbytes. The last stripe consists of a two slices, Disks F and G. Because no interlace is specified for the third stripe, it inherits the value from the stripe before it, which in this case is 32 Kbytes. Sequential data chunks are addressed to the first stripe until that stripe has no more space. Chunks are then addressed to the second stripe. When this stripe has no more space, chunks are addressed to the third stripe. Within each stripe, the data chunks are interleaved according to the specified interlace value.
When you create a simple metadevice of more than one slice, any slice except the first skips the first disk cylinder, if the slice starts at cylinder 0. For example, consider this output from the metastat(1M) command:
# metastat d0 d0: Concat/Stripe Size: 3546160 blocks Stripe 0: (interface: 32 blocks) Device Start Block Dbase c1t0d0s0 0 No c1t0d1s0 1520 No c1t0d2s0 1520 No c1t0d2s0 1520 No c1t1d0s0 1520 No c1t1d1s0 1520 No c1t1d2s0 1520 No |
In this example, stripe d0 shows a start block for each slice except the first as block 1520. This is to preserve the disk label in the first disk sector in all of the slices except the first. The metadisk driver must skip at least the first sector of those disks when mapping accesses across the stripe boundaries. Because skipping only the first sector would create an irregular disk geometry, the entire first cylinder of these disks is skipped. This enables higher level file system software (UFS) to optimize block allocations correctly. Thus, DiskSuite protects the disk label from being overwritten, and purposefully skips the first cylinder.
The reason for not skipping the first cylinder on all slices in the concatenation or stripe has to do with UFS. If you create a concatenated metadevice from an existing file system, and add more space to it, you would lose data because the first cylinder is where the data is expected to begin.