C H A P T E R 6 |
Managing Disk Volumes |
This chapter describes redundant array of independent disks (RAID) concepts, how to manage disk volumes, and how to configure hardware mirroring using the SAS controller.
This chapter contains the following sections:
Disk volumes are logical disk devices comprising one or more physical disks or partitions from several different disks.
Once you create a volume, the OS uses and maintains the volume as if it were a single disk. By providing this logical volume management layer, the software overcomes the restrictions imposed by physical disk devices.
Sun's volume management products also provide RAID data redundancy and performance features. RAID is a technology that helps protect against disk and hardware failures. Through RAID technology, volume management software is able to provide high data availability, excellent I/O performance, and simplified administration.
Volume management software lets you create disk volumes. Sun Microsystems offers two different volume management applications for use on the Sun Fire V445 server:
Sun's volume management applications offer the following features:
VERITAS Volume Manager software actively supports multiported disk arrays. It automatically recognizes multiple I/O paths to a particular disk device within an array. Called Dynamic Multipathing (DMP), this capability provides increased reliability by providing a path failover mechanism. If one connection to a disk is lost, VERITAS Volume Manager continues to access the data over the remaining connections. This multipathing capability also provides greater I/O throughput by automatically balancing the I/O load uniformly across multiple I/O paths to each disk device.
A newer alternative to DMP that is also supported by the Sun Fire V445 server is Sun StorEdge Traffic Manager software. Sun StorEdge Traffic Manager is a server-based dynamic path failover software solution, used to improve the overall availability of business applications. Sun StorEdge Traffic Manager (previously known as multiplexed input/output, or MPxIO) is included in the Solaris OS.
The Sun StorEdge Traffic Manager software integrates multiple path I/O capabilities, automatic load balancing, and path failover functions into one package for Sun servers connected to supported Sun StorEdge systems. Sun StorEdge Traffic Manager can provide you with increased system performance and availability for building mission-critical storage area networks (SANs).
The Sun StorEdge Traffic Manager architecture provides the following capabilities:
Sun StorEdge T3, Sun StorEdge 3510, and Sun StorEdge A5x00 storage arrays are all supported by Sun StorEdge Traffic Manager on a Sun Fire V445 server. Supported I/O controllers are single and dual fibre-channel network adapters, including the following:
Note - Sun StorEdge Traffic Manager is not supported for boot disks containing the root (/) file system. You can use hardware mirroring or VERITAS Volume Manager instead. See Creating a Hardware Disk Mirror and About Volume Management Software. |
Refer to the documentation supplied with the VERITAS Volume Manager and Solaris Volume Manager software. For more information about Sun StorEdge Traffic Manager, see your Solaris system administration documentation.
VERITAS Volume Manager and Solstice DiskSuite software support RAID technology to optimize performance, availability, and cost per user. RAID technology reduces recovery time in the event of file system errors, and increases data availability even in the event of a disk failure. There are several levels of RAID configurations that provide varying degrees of data availability with corresponding trade-offs in performance and cost.
This section describes some of the most popular and useful of those configurations, including:
Disk concatenation is a method for increasing logical volume size beyond the capacity of one disk drive by creating one large volume from two or more smaller drives. This lets you create arbitrarily large partitions..
Using this method, the concatenated disks are filled with data sequentially, with the second disk being written to when no space remains on the first, the third when no space remains on the second, and so on.
Disk striping, Integrated Stripe (IS), or RAID 0 is a technique for increasing system throughput by using several disk drives in parallel. In nonstriped disks the OS writes a single block to a single disk. In a striped arrangement, each block is divided and portions of the data are written to different disks simultaneously.
System performance using RAID 0 will be better than using RAID 1, but the possibility of data loss is greater because there is no way to retrieve or reconstruct data stored on a failed disk drive.
Disk mirroring, Integrated Mirror (IM), or RAID 1 is a technique that uses data redundancy - two complete copies of all data stored on two separate disks - to protect against loss of data due to disk failure. One logical volume is duplicated on two separate disks.
Whenever the OS needs to write to a mirrored volume, both disks are updated. The disks are maintained at all times with exactly the same information. When the OS needs to read from the mirrored volume, it reads from whichever disk is more readily accessible at the moment, which can result in enhanced performance for read operations.
RAID 1 offers the highest level of data protection, but storage costs are high, and write performance compared to RAID 0 is reduced since all data must be stored twice.
On the Sun Fire V445 server, you can configure hardware disk mirroring using the SAS controller. This provides higher performance than with conventional software mirroring using volume management software. For more information, see:
In a hot-spares arrangement, one or more disk drives are installed in the system but are unused during normal operation. This configuration is also referred to as hot relocation. Should one of the active drives fail, the data on the failed disk is automatically reconstructed and generated on a hot-spare disk, enabling the entire data set to maintain its availability.
On the Sun Fire V445 server, the SAS controller supports mirroring and striping using the Solaris OS raidctl utility.
A hardware RAID volume created under the raidctl utility behaves slightly differently than one created using volume management software. Under a software volume, each device has its own entry in the virtual device tree, and read/write operations are performed to both virtual devices. Under hardware RAID volumes, only one device appears in the device tree. Member disk devices are invisible to the operating system, and are accessed only by the SAS controller.
In order to perform a disk hot-plug procedure, you must know the physical or logical device name for the drive that you want to install or remove. If your system encounters a disk error, often you can find messages about failing or failed disks in the system console. This information is also logged in the /var/adm/messages file(s).
These error messages typically refer to a failed hard disk drive by its physical device name (such as /devices/pci@1f,700000/scsi@2/sd@1,0) or by its logical device name (such as c1t1d0). In addition, some applications might report a disk slot number (0 through 3).
You can use TABLE 6-1 to associate internal disk slot numbers with the logical and physical device names for each hard disk drive.
Logical Device Name[1] |
||
---|---|---|
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@0,0 |
||
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@1,0 |
||
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@2,0 |
||
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@3,0 |
||
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@4,0 |
||
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@5,0 |
||
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@6,0 |
||
/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1/sd@7,0 |
Perform this procedure to create an internal hardware disk mirror (IM or RAID 1) configuration on your system.
Verify which disk drive corresponds with which logical device name and physical device name. See:
To Create a Hardware Disk Mirror |
1. To verify that a hardware disk mirror does not already exist, type:
The example indicates that no RAID volume exists. In another case:
# raidctl RAID Volume RAID RAID Disk Volume Type Status Disk Status ------------------------------------------------------ c0t4d0 IM OK c0t5d0 OK c0t4d0 OK |
The example indicates a hardware mirror has degraded at disk c1t2d0.
Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed. |
When you create a RAID mirror, the slave drive (in this case, c1t1d0) disappears from the Solaris device tree.
3. To check the status of a RAID mirror, type:
# raidctl RAID RAID RAID Disk Volume Status Disk Status -------------------------------------------------------- c1t0d0 RESYNCING c1t0d0 OK c1t1d0 OK |
The example indicates that the RAID mirror is still resynchronizing with the backup drive.
Note - The process of synchronizing a drive may take up to 60 minutes. |
The example below shows that the RAID mirror is completely restored and online.
# raidctl RAID RAID RAID Disk Volume Status Disk Status ------------------------------------ c1t0d0 OK c1t0d0 OK c1t1d0 OK |
Under RAID 1 (disk mirroring), all data is duplicated on both drives. If a disk fails, replace it with a working drive and restore the mirror. For instructions, see:
For more information about the raidctl utility, see the raidctl(1M) man page.
Due to the volume initialization that occurs on the disk controller when a new volume is created, the volume must be configured and labeled using the format(1M) utility prior to use with the Solaris Operating System (see Configuring and Labeling a Hardware RAID Volume for Use in the Solaris Operating System). Because of this limitation, raidctl(1M) blocks the creation of a hardware RAID volume if any of the member disks currently have a file system mounted.
This section describes the procedure required to create a hardware RAID volume containing the default boot device. Since the boot device always has a mounted file system when booted, an alternate boot medium must be employed, and the volume created in that environment. One alternate medium is a network installation image in single-user mode (refer to the Solaris 10 Installation Guide for information about configuring and using network-based installations).
To Create a Hardware Mirrored Volume of the Default Boot Device |
1. Determine which disk is the default boot device
From the OpenBoot ok prompt, type the printenv command, and if necessary the devalias command, to identify the default boot device. For example:
ok printenv boot-device boot-device = disk ok devalias disk disk /pci@780/pci@0/pci@9/scsi@0/disk@0,0 |
2. Type the boot net -s command.
3. Once the system has booted, use the raidctl(1M) utility to create a hardware mirrored volume, using the default boot device as the primary disk.
See Configuring and Labeling a Hardware RAID Volume for Use in the Solaris Operating System. For example:
# raidctl -c c0t0d0 c0t1d0 Creating RAID volume c0t0d0 will destroy all data on member disks, proceed (yes/no)? yes Volume c0t0d0 created # |
4. Install the volume with the Solaris Operating System using any supported method.
The hardware RAID volume c0t0d0 appears as a disk to the Solaris installation program.
Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed. |
Use this procedure to create a hardware striped (IS or RAID 0) volume.
1. Verify which hard drive corresponds with which logical device name and physical device name.
See About Physical Disk Slot Numbers, Physical Device Names, and Logical Device Names.
To verify the current RAID configuration, type:
The preceding example indicates that no RAID volume exists.
Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed. |
The creation of the RAID volume is interactive, by default. For example:
# raidctl -c -r 0 c0t1d0 c0t2d0 c0t3d0 Creating RAID volume c0t1d0 will destroy all data on member disks, proceed (yes/no)? yes Volume 'c0t1d0' created # |
When you create a RAID striped volume, the other member drives (in this case, c0t2d0 and c0t3d0) disappear from the Solaris device tree.
As an alternative, you can use the -f option to force the creation if you are sure of the member disks, and sure that the data on all other member disks can be lost. For example:
3. To check the status of a RAID striped volume, type:
# raidctl RAID Volume RAID RAID Disk Volume Type Status Disk Status -------------------------------------------------------- c0t1d0 IS OK c0t1d0 OK c0t2d0 OK c0t3d0 OK |
The example shows that the RAID striped volume is online and functioning.
Under RAID 0 (disk striping), there is no replication of data across drives. The data is written to the RAID volume across all member disks in a round-robin fashion. If any one disk is lost, all data on the volume is lost. For this reason, RAID 0 cannot be used to ensure data integrity or availability, but can be used to increase write performance in some scenarios.
For more information about the raidctl utility, see the raidctl(1M) man page.
After a creating a RAID volume using raidctl, use format(1M) to configure and label the volume before attempting to use it in the Solaris Operating System.
The format utility might generate messages about corruption of the current label on the volume, which you are going to change. You can safely ignore these messages.
2. Select the disk name that represents the RAID volume that you have configured.
In this example, c0t2d0 is the logical name of the volume.
3. Type the type command at the format> prompt, then select 0 (zero) to auto configure the volume.
4. Use the partition command to partition, or slice, the volume according to your desired configuration.
See the format(1M) man page for additional details.
5. Write the new label to the disk using the label command.
6. Verify that the new label has been written by printing the disk list using the disk command.
Note that c0t2d0 now has a type indicating it is an LSILOGIC-LogicalVolume.
The volume can now be used in the Solaris Operating System.
Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed. |
Perform this procedure to remove a hardware disk mirror configuration from your system.
Verify which disk drive corresponds with which logical device name and physical device name. See:
To Delete a Hardware Disk Mirror |
1. Determine the name of the mirrored volume. Type:
# raidctl RAID RAID RAID Disk Volume Status Disk Status ------------------------------------ c1t0d0 OK c1t0d0 OK c1t1d0 OK |
In this example, the mirrored volume is c1t0d0.
Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed. |
2. To delete the volume, type:
3. To confirm that you have deleted the RAID array, type:
For more information, see the raidctl(1M) man page.
Verify which disk drive corresponds with which logical device name and physical device name. See:
You need to refer to the following document to perform this procedure:
To Perform a Mirrored Disk Hot-Plug Operation |
1. To confirm a failed disk, type:
# raidctl RAID RAID RAID Disk Volume Status Disk Status ---------------------------------------- c1t1d0 DEGRADED c1t1d0 OK c1t2d0 DEGRADED |
This example indicates that the disk mirror has degraded due to a failure in disk c1t2d0.
Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed. |
2. Remove the disk drive, as described in the Sun Fire V445 Server Service Manual.
There is no need to issue a software command to bring the drive offline when the drive has failed and the OK-to-Remove indicator is lit.
3. Install a new disk drive, as described in the Sun Fire V445 Server Service Manual.
The RAID utility automatically restores the data to the disk.
4. To check the status of a RAID rebuild, type:
# raidctl RAID RAID RAID Disk Volume Status Disk Status ---------------------------------------- c1t1d0 RESYNCING c1t1d0 OK c1t2d0 OK |
This example indicates that RAID volume c1t1d0 is resynchronizing.
If you issue the command again some minutes later, it indicates that the RAID mirror is finished resynchronizing and is back online:
# raidctl RAID RAID RAID Disk Volume Status Disk Status ---------------------------------------- c1t1d0 OK c1t1d0 OK c1t2d0 OK |
For more information, see the raidctl(1M) man page.
Verify which disk drive corresponds with which logical device name and physical device name. See:
Ensure that no applications or processes are accessing the disk drive.
You need to refer to the following document to perform this procedure:
To View the Status of the SCSI Devices |
Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed. |
The -al options return the status of all SCSI devices, including buses and USB devices. (In this example, no USB devices are connected to the system.)
Note that while you can use the Solaris OS cfgadm install_device and cfgadm remove_device commands to perform a disk drive hot-plug procedure, these commands issue the following warning message when you invoke these commands on a bus containing the system disk:
This warning is issued because these commands attempt to quiesce the SAS bus, but the Sun Fire V445 server firmware prevents it. This warning message can be safely ignored in the Sun Fire V445 server, but the following procedure avoids this warning message altogether.
To Perform a Nonmirrored Disk Hot-Plug Operation |
1. To remove the disk drive from the device tree, type:
This example removes c1t3d0 from the device tree. The blue OK-to-Remove indicator lights.
2. To verify that the device has been removed from the device tree, type:
c1t3d0 is now unavailable and unconfigured. The corresponding disk drive OK-to-Remove indicator is lit.
3. Remove the disk drive, as described in the Sun Fire V445 Server Parts Installation and Removal Guide.
The blue OK-to-Remove indicator goes out when you remove the disk drive.
4. Install a new disk drive, as described in the Sun Fire V445 Server Parts Installation and Removal Guide.
5. To configure the new disk drive, type:
The green Activity indicator flashes as the new disk at c1t3d0 is added to the device tree.
6. To verify that the new disk drive is in the device tree, type:
Note that c1t3d0 is now listed as configured.
Copyright © 2007, Sun Microsystems, Inc. All Rights Reserved.