DiskSuite Configuration Guidelines
A poorly designed DiskSuite configuration can degrade performance. This section offers tips for getting good performance from DiskSuite.
General Guidelines
-
Disk and controllers - Place drives in a metadevice on separate drive paths. For SCSI drives, this means separate host adaptors. For IPI drives, this means separate controllers. Spreading the I/O load over several controllers improves metadevice performance and availability.
For SPARCstorage Arrays, you should use drives in a mirror on different chassis, if possible. This type of configuration ensures that all mirror data would survive a SPARCstorage Array chassis failure. If you cannot spread drives across different chassis, then use drives in different trays. This enables you to offline submirrors and the tray to be spun down or removed for maintenance while the mirror stays online.
For example, consider a two-way mirror with each submirror composed of a concatenation of three SPARCstorage Array disks. One submirror would consist of three disks in tray 1, and the other submirror would consist of drives in tray 2. Using the command line interface to initialize this configuration would look like this:
# metainit d1 3 1 c0t0d0s2 1 c0t0d1s2 c0t0d2s2 d1: Concat/Stripe is setup # metainit d2 3 1 c0t2d0s2 1 c0t2d1s2 c0t2d2s2 d2: Concat/Stripe is setup # metainit d0 -m d1 d0: Mirror is setup # metattach d0 d2 d0: Component d2 is attached |
Strings t0 and t1 are contained in tray 1, t2 and t3 in tray 2, and t4 and t5 are in tray 3. Hence, in the above commands, to create submirrors in different trays, we use string t0 for one submirror, and t2 for the second submirror.
-
System Files - Never edit or remove the /etc/lvm/mddb.cf or /etc/lvm/md.cf files.
Make sure these files are backed up on a regular basis.
-
After a slice is defined as a metadevice and activated, do not use it for any other purpose.
-
Have a hardcopy of output from the prtvtoc(1M) command in case you need to reformat a bad disk.
State Database Replica Guidelines
-
When a state database replica is placed on a slice that becomes part of a metadevice, the capacity of the metadevice is reduced by the space occupied by the replica(s). The space used by a replica is rounded up to the next cylinder boundary and this space is skipped by the metadevice. However, the default size of a state database replicas is only 1034 blocks, and combining replicas and metadevices in this way is actually a very efficient use of DiskSuite.
Striping Guidelines
-
Stripes are created only from slices, not other metadevices.
-
Do not use slices from disks with different disk geometry.
-
Use slices on the same controller but on different disks. Using stripes that are each on different controllers can increase the number of simultaneous reads and writes that can be performed.
-
Set up a stripe's interlace value to better match the I/O requests made by the system or applications.
-
Do not stripe partitions that are on a single disk. This practice eliminates simultaneous access and causes performance problems.
-
Use the same size disk components. Striping different size disk components results in unused disk space.
-
If you use slices of different sizes when striping, then disk capacity is limited to a multiple of the smallest.
Concatenation Guidelines
-
Concatenations are created only from slices, not other metadevices.
-
Avoid using slices with different disk geometries.
-
When possible, distribute the components of a concatenated metadevice across different controllers and buses.
-
Concatenation uses less CPU cycles than striping. It performs well for small random I/O and for even I/O distribution.
Concatenated Stripe Guidelines
Read the guidelines for stripes and concatenations above.
Mirror Guidelines
-
Disks and controllers - Keep the slices of different submirrors on different disks and controllers. Data protection is diminished considerably if slices of two or more submirrors of the same mirror are on the same disk. Likewise, organize submirrors across separate controllers, because controllers and associated cables tends to fail more often than disks. This practice also improves mirror performance.
-
Same disk - Do not define mirrors on the same disk. Writes to the same drive contend for the same resources, and the failure of the one drive would mean the loss of all data.
-
Read/write performance - Mirroring may improve read performance, but write performance is always degraded. Mirroring improves read performance only in threaded or asynchronous I/O situations. No performance gain results if there is only a single thread reading from the metadevice.
-
Same-size submirrors - Use the same size submirrors. Different size submirrors result in unused disk space.
-
Same-type disks and controllers - Use the same type of disks and controllers in a single mirror. Particularly in old SCSI or SMD storage devices, different models or brands of disk or controller can have widely varying performance. Mixing the different performance levels in a single mirror can cause performance to degrade significantly.
-
Setting read and write policies for submirrors - Experimenting with the mirror read policies can improve performance. For example, the default read mode is to alternate reads in a round-robin fashion among the disks. This is the default because it tends to work best for UFS multi-user, multi-process activity.
In some cases, the geometric read option improves performance by minimizing head motion and access time. This option is most effective when there is only one slice per disk, when only one process at a time is using the slice/file system, and when I/O patterns are highly sequential or when all accesses are read.
To change mirror options, refer to "How to Change a Mirror's Options (DiskSuite Tool)".
-
Mounting mirrors - Only mount the mirror device directly. Do not try and mount a submirror directly, unless it is offline and mounted read-only. Do not mount a slice that is part of a submirror; this could destroy data and crash the system.
-
Mirroring swap - Use swap -l to check for all swap devices. Slices specified as swap must be mirrored separately.
RAID5 Metadevice Guidelines
-
Follow the 20-percent write rule - Because of the complexity of parity calculations, metadevices with greater than about 20 percent writes should probably not be RAID5 metadevice devices. If data redundancy is needed, consider mirroring.
-
Drawbacks to a "slice-heavy" RAID5 metadevice - The more slices a RAID5 metadevice contains, the longer read and write operations will take when a component fails.
-
RAID5 metadevices cannot be mirrored.
-
RAID5 metadevices and striping guidelines - Striping guidelines also apply to RAID5 metadevice configurations. Refer to "Striping Guidelines".
-
Use different controllers - When creating RAID5 devices, use slices across separate controllers, because controllers and associated cables tends to fail more often than disks. This practice also improves mirror performance.
-
Use same-size slices - Use the same size disk slices. Creating a RAID5 metadevice of different size slices results in unused disk space.
-
Interlace value - It is configurable at the time the metadevice is created; thereafter, the value cannot be modified. The default interlace value is 16 Kbytes. This is reasonable for most applications. If the different slices in the RAID5 metadevice reside on different controllers and the accesses to the metadevice are primarily large sequential accesses, then an interlace value of 32 Kbytes might have better performance.
-
Concatenating to a RAID5 metadevice - Concatenating a new slice to an existing RAID5 will have an impact on the overall performance of the metadevice because the data on concatenations is sequential; data is not striped across all components. The original slices of the metadevice have data and parity striped across all slices. This striping is lost for the concatenated slice, although the data is still recoverable from errors because the parity is used during the component I/O.
Concatenated slices also differ in the sense that they do not have parity striped on any of the regions. Thus, the entire contents of the slice are available for data.
Any performance enhancements for large or sequential writes are lost when slice are concatenated.
UFS Logging Guidelines
-
For logging and master devices - Place logging devices and master device that belong to the same trans metadevice on separate drives and controllers.
-
For trans metadevices and shared logging devices - Trans metadevices can share metadevice logging devices. But file systems with the heaviest loads should have separate logs.
-
For small file systems - Small file systems with mostly read operations probably do not need to be logged.
-
For mirroring logging devices - Mirror all logging devices whenever possible. Losing the data in a log because of device errors can leave a file system in an inconsistent state which fsck(1M) may not be able to repair without user intervention.
-
Larger logging devices result in greater concurrency.
Hot Spare Guidelines
-
For hot spares as temporary fixes - Hot spares are not designed to remain a permanent part of your configuration. They need to be replaced with repaired or new slices.
-
For hot spares and state database replicas - Hot spares cannot contain state database replicas.
-
For cross-controller assigning - Ideally, slices added to the hot spare pool should be attached to different controllers. This ensures availability of data due to controller error or failure.
-
For wrong-size hot spares - Do not associate hot spares of the wrong size with submirrors or RAID5 metadevices.
-
For hot spares marked In Use - Make sure that all hot spares within a hot spare pool are not marked In-Use.
-
For one-way mirrors and hot spares - Do not assign a hot spare pool to a submirror in a one-way mirror.
-
Hot spares are used on a first-fit basis - When adding hot spares of different sizes to a hot spare pool, add the smaller slices first.
File System Guidelines
-
Do not mount file systems on a metadevice's underlying slice. If a slice is used for a metadevice of any kind, you must not mount that slice as a file system. If possible, unmount any physical device you intend to use as a metadevice before you activate it. For example, if you create a trans metadevice for a UFS, in the /etc/vfstab file, you would specify the trans metadevice name as the device to mount and fsck.
Labeled Partitions
-
All physical devices must have a disk label, normally created by programs such as install, format, or fmthard. The label can appear on more than one of the logical partitions that are defined in the label. Physical partitions that contain a label should not allow a user to write to the block that contains the label. Normally, this is block 0. UNIX device drivers allow a user to overwrite this label.
Security Considerations
-
DiskSuite does not provide an audit trail for any reconfiguration of metadevices that may be performed on the system. This means that DiskSuite does not support C2 security.
Compatibility
- © 2010, Oracle Corporation and/or its affiliates