Managing Disks

C H A P T E R 5

Managing Disks

This section describes how to configure and manage hardware RAID volumes on M3000 and M8000/M9000 servers and how to perform an active replacement of a disk drive that is configured as a hardware RAID volume.

RAID technology enables the construction of a logical volume made up of several physical disks to provide data redundancy, improved performance, or both.

The M3000 and M8000/M9000 servers use an on-board SAS controller that supports RAID 1 (mirroring) volumes using the Oracle Solaris OS raidctl(1M) utility. On M3000 servers, the controller is on the MBU. On M8000/M9000 servers, the controller is on a RAID-compatible IOUA. (Not all IOUAs are RAID-compatible. See Requirements for RAID-Capability .)

Volume migration (relocating all RAID volume disk members from one server to another) is not supported. If you must perform this operation, contact a certified service engineer.

This chapter contains these sections:

Requirements for RAID-Capability

Creating a Hardware Mirrored Volume as a Data Disk

Deleting a Hardware RAID Volume

Active Replacement of a Mirrored Disk

Disk Slot Numbers

Related Information

Note - Most examples in this chapter are for M3000 servers. In most cases they are similar to what you might see on M8000/M9000 servers.

Requirements for RAID-Capability

M3000 servers must have a a SPARC64 VII+ processor. You can identify the processor version by executing the showhardconf(8) command. If you see in the command’s output a clock speed of 2.86 GHz, your server has the correct processor.

M8000/M9000 servers must have a RAID-capable IOUA. One way to determine if your M8000/M9000 server has an IOUA that supports RAID is to execute the showhardconf(8) command. If you see in the command’s output “Type:2”, your server has the required IOUA.

To configure and use hardware RAID volumes on M3000 and M8000/M9000 servers, you must have a minimum XSCF firmware with applicable Oracle Solaris OS patches. For the latest information, see the latest SPARC Enterprise M3000/M4000/M5000/M8000/M9000 Servers Product Notes.

Creating a Hardware Mirrored Volume as a Data Disk

Caution - Creating RAID volumes using the on-board SAS controller destroys all data on member disks.

This section describes these procedures:

To Create a Hardware Mirrored Volume as a Data Disk

To Create a Hardware Mirrored Volume as the Default Boot Disk

To Configure a Hardware RAID Volume for the Oracle Solaris OS (Boot Disk / Data Disk)

To Delete a Hardware RAID Volume (Data Disk)

To Perform an Active Replacement of a Mirrored Disk (Boot Disk / Data Disk)

Creating a Hardware Mirrored Volume

The server's SAS controller can configure one RAID volume for each IOUA on M8000/M9000 servers, and up to two RAID volumes on each M3000 server. Prior to mirrored volume creation, ensure that the member disks are available and that RAID volumes do not already exist.

The status of the RAID volume might be:

OPTIMAL - The RAID volume is online and fully synchronized.

SYNC - The data between the primary and secondary member disks in a hardware mirrored volume are synchronizing.

DEGRADED - A member disk is failed or otherwise offline.

FAILED - The RAID volume has failed and should be deleted and reinitialized. This failure can occur when both member disks in a hardware mirrored volume are lost.

The Status column displays the status of each physical disk. Each member disk might be GOOD, indicating that it is online and functioning properly, or it might be FAILED, indicating that the disk has hardware or configuration issues that need to be addressed.

For example, if a failure occurs on a disk of the RAID volume on an M3000 server, output of the raidctl(1M) command would be similar to the following:

# raidctl -l c0t0d0
Volume                  Size    Stripe  Status   Cache  RAID
        Sub                     Size                    Level
                Disk
----------------------------------------------------------------
c0t0d0                  558.8G  N/A     DEGRADED OFF    RAID1
                0.0.0   558.8G          GOOD
                0.1.0   558.8G          FAILED

The above example shows that the RAID volume was degraded due to a failure in the secondary disk, c0t01d0 (0.1.0). See the raidctl(1M) man page for additional details regarding volume and disk status.

Note - The logical device names might appear differently on your system, depending on the server platform and the number and type of add-on disk controllers installed.

The disk controller synchronizes hardware mirrored volumes one at a time. If you create a second hardware mirrored volume before the first hardware mirrored volume completes its synchronization, the first volume's RAID status will indicate SYNC, and the second volume's RAID status will indicate OPTIMAL. Once the first volume has completed, its RAID status changes to OPTIMAL, and the second volume automatically starts synchronizing, with a RAID status of SYNC.

Under RAID 1 (disk mirroring), all data is duplicated on both disk drives. If a disk drive fails, replace it with a working drive and restore the mirror. For instructions, see Active Replacement of a Mirrored Disk.

To Create a Hardware Mirrored Volume as a Data Disk

1. Execute the raidctl(1M) command to verify the available RAID volume and physical device name:

# raidctl
Controller: 0
        Disk: 0.0.0
        Disk: 0.1.0
        Disk: 0.2.0
        Disk: 0.3.0

The preceding display indicates that no RAID volume exists. See Disk Slot Numbers.

2. Execute the raidctl(1M) command with its -c option to create a hardware mirrored volume:

# raidctl -c primary secondary

where primary is the device name of the primary member disk and secondary is the device name of the secondary member disk.

The creation of the RAID volume is interactive, by default. For example:

# raidctl -c c0t2d0 c0t3d0
Creating RAID volume c2t2d0 will destroy all data on member disks, proceed (yes/no)? yes
...
Volume c0t2d0 is created successfully!

As an alternative, you can use the -f option to force the creation, but do not do so unless you are certain about the member disks and that the data on both member disks can be lost. For example:

# raidctl -f -c c0t2d0 c0t3d0
Volume c0t2d0 is created successfully!

When you create a mirrored volume, the secondary disk drive (in this case, c0t3d0) disappears from the Oracle Solaris device tree.

3. To check the status of the mirrored volume, execute the following command.

Output might appear similar to the following:

# raidctl 
Controller: 0
       Volume: c0t2d0
       Disk: 0.0.0
       Disk: 0.1.0
       Disk: 0.2.0
       Disk: 0.3.0

In the preceding example a single volume (c0t2d0) has been enabled.

# raidctl -l c0t2d0
Volume                  Size    Stripe  Status   Cache  RAID
        Sub                     Size                    Level
                Disk
----------------------------------------------------------------
c0t2d0                  558.8G  N/A     SYNC     OFF    RAID1
                0.2.0   558.8G          GOOD
                0.3.0   558.8G          GOOD

The preceding example indicates that the mirrored volume is resynchronizing (SYNC) with the backup drive.

The following example shows that the mirrored volume is synchronized and online (OPTIMAL).

# raidctl -l c0t0d0
Volume                  Size    Stripe  Status   Cache  RAID
        Sub                     Size                    Level
                Disk
----------------------------------------------------------------
c0t2d0                  558.8G  N/A     OPTIMAL  OFF    RAID1
                0.2.0   558.8G          GOOD
                0.3.0   558.8G          GOOD

For more information about the raidctl utility, see the raidctl(1M) man page.

Creating a Hardware Mirrored Volume as the Boot Disk

Due to the volume initialization that occurs on the disk controller when a new RAID volume is created, the RAID volume must be configured and labeled using the format(1M) utility prior to use with the Oracle Solaris (see Configuring a Hardware RAID Volume for the Oracle Solaris OS (Boot Disk / Data Disk)). Because of this limitation, raidctl(1M) blocks the creation of a hardware RAID volume if any of the member disks currently have a file system mounted.

This section describes the procedure required to create a hardware RAID volume as the boot disk. Since the boot disk always has a mounted file system when booted, an alternate boot medium must be employed, and the volume created in that environment. One alternate medium is a network installation image in single-user mode. (Refer to the Oracle Solaris 10 Installation Guide for information about configuring and using network-based installations.)

To Create a Hardware Mirrored Volume as the Default Boot Disk

1. Determine which disk is the boot disk.

From the OpenBoot PROM ok prompt, execute the printenv command and, if necessary, the devalias command, to identify the default boot disk. Note that device paths for devalias vary per server. As in other examples, the output shown in this procedure was produced on an M3000 server.

ok printenv boot-device
boot-device             disk
 
ok devalias disk
disk                     /pci@0,600000/pci@0/pci@0/scsi@0/sd@0,0

2. Execute the following command to boot the Oracle Solaris OS via the network in single-user mode:

ok boot net -s

3. Once the system has booted, use the raidctl(1M) utility to create a hardware mirrored volume, using the default boot disk as the primary disk.

See Creating a Hardware Mirrored Volume. For example:

# raidctl -c c0t0d0 c0t1d0
Creating RAID volume c0t0d0 will destroy all data on member disks, proceed (yes/no)? yes
...
Volume c0t0d0 is created successfully!

4. Install the volume with the Oracle Solaris OS using any supported method.

The hardware RAID volume c0t0d0 appears as a disk to the Oracle Solaris OS installation program.

Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed.

Configuring a Hardware RAID Volume for the Oracle Solaris OS (Boot Disk / Data Disk)

After creating a RAID volume using the raidctl(1M) utility, use the format(1M) utility to configure and label the volume before attempting to use it in the Oracle Solaris OS.

To Configure a Hardware RAID Volume for the Oracle Solaris OS (Boot Disk / Data Disk)

1. Start the format(1M) utility:

# format

The format utility might generate messages about corruption of the current label on the volume, which you are going to change. You can safely ignore these messages.

2. Select the disk name that represents the RAID volume that you have configured.

In this example, c0t2d0 is the logical name of the volume.

# format
Searching for disks...done
 
c0t2d0: configured with capacity of 558.47GB
 
 
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <FUJITSU-MBD2300RC-3702 cyl 46873 alt 2 hd 20 sec 625>
          /pci@0,600000/pci@0/pci@0/scsi@0/sd@0,0
       1. c0t1d0 <FUJITSU-MBD2300RC-3702 cyl 46873 alt 2 hd 20 sec 625>
          /pci@0,600000/pci@0/pci@0/scsi@0/sd@1,0
       2. c0t2d0 <LSILOGIC-LogicalVolume-3000 cyl 65533 alt 2 hd 16 sec 1117>
          /pci@0,600000/pci@0/pci@0/scsi@0/sd@2,0
Specify disk (enter its number): 2
selecting c0t2d0
[disk formatted]
Disk not labeled.  Label it now?

3. Write a new label following the message shown in Step 2:

Disk not labeled.  Label it now? y
 
 
FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        save       - save new disk/partition definitions
        inquiry    - show vendor, product and revision
        volname    - set 8-character volume name
        !<cmd>     - execute <cmd>, then return
        quit
format>

4. Verify that the new label has been written in the disk list displayed by the disk command:

format> disk
 
 
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <FUJITSU-MBD2300RC-3702 cyl 46873 alt 2 hd 20 sec 625>
          /pci@0,600000/pci@0/pci@0/scsi@0/sd@0,0
       1. c0t1d0 <FUJITSU-MBD2300RC-3702 cyl 46873 alt 2 hd 20 sec 625>
          /pci@0,600000/pci@0/pci@0/scsi@0/sd@1,0
       2. c0t2d0 <LSILOGIC-LogicalVolume-3000 cyl 65533 alt 2 hd 16 sec 1117>
          /pci@0,600000/pci@0/pci@0/scsi@0/sd@2,0
Specify disk (enter its number)[2]:

In the example above, verify the information shown for Disk 2.

5. Use the partition command to change partitions according to your requirements.

For more information, see the format(1M) man page.

6. Exit the format utility.

The volume can now be used in the Oracle Solaris OS.

Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed.

Deleting a Hardware RAID Volume

To delete a hardware RAID volume, follow these steps.

To Delete a Hardware RAID Volume (Data Disk)

1. Execute the raidctl(1M) command to verify the available RAID volume and physical device name:

# raidctl
Controller: 0
        Volume:c0t2d0
        Disk: 0.0.0
        Disk: 0.1.0
        Disk: 0.2.0
        Disk: 0.3.0

In this example, the RAID volume is c0t2d0.

Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed.

See Disk Slot Numbers.

2. Execute the following command to delete the volume:

# raidctl -d mirrored-volume

For example:

# raidctl -d c0t2d0
Deleting RAID volume c0t2d0 will destroy all data it contains, proceed (yes/no)? y
Volume c0t2d0 is deleted successfully!

3. Execute the following command to confirm that you have deleted the RAID volume:

# raidctl

For example:

# raidctl
Controller: 0
        Disk: 0.0.0
        Disk: 0.1.0
        Disk: 0.2.0
        Disk: 0.3.0

For more information, see the raidctl(1M) man page.

Active Replacement of a Mirrored Disk

To perform an active replacement of a mirrored disk, follow these steps.

To Perform an Active Replacement of a Mirrored Disk (Boot Disk / Data Disk)

1. Execute the raidctl(1M) command to verify the available RAID volume and physical device name:

# raidctl
Controller: 0
        Volume:c0t0d0
        Disk: 0.0.0
        Disk: 0.1.0
        Disk: 0.2.0
        Disk: 0.3.0

See Disk Slot Numbers.

2. Execute the raidctl(1M) command with its -l option to confirm a failed disk.

If the Status column shows a disk as FAILED, the drive can be removed and a new drive inserted. Upon insertion, the new disk should be GOOD and the volume should be SYNC.

For example:

# raidctl -l c0t0d0
Volume                  Size    Stripe  Status   Cache  RAID
        Sub                     Size                    Level
                Disk
----------------------------------------------------------------
c0t0d0                  558.8G  N/A     DEGRADED OFF    RAID1
                0.0.0   558.8G          GOOD
                0.1.0   558.8G          FAILED

This example indicates that the mirrored volume has degraded due to a failure in the member disk c0t1d0 (0.1.0).

Note - The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed.

3. Remove the disk drive.

For detailed instructions, see the SPARC Enterprise M3000 Server Service Manual or the SPARC Enterprise M8000/M9000 Servers Service Manual.

There is no need to use a software command to bring the drive offline when the drive has failed.

4. Install a new disk drive.

For detaied instructions, see the SPARC Enterprise M3000 Server Service Manual or the SPARC Enterprise M8000/M9000 Servers Service Manual.

The RAID utility automatically restores the data to the disk.

5. Execute the raidctl(1M) command to check the status of a RAID rebuild.

For example:

# raidctl -l c0t0d0
Volume                  Size    Stripe  Status   Cache  RAID
        Sub                     Size                    Level
                Disk
----------------------------------------------------------------
c0t0d0                  558.8G  N/A     SYNC     OFF    RAID1
                0.0.0   558.8G          GOOD
                0.1.0   558.8G          GOOD

This example indicates that RAID volume c0t0d0 is resynchronizing.

If you type the command again once synchronization has completed, it indicates that the RAID mirror is finished resynchronizing and is back online:

# raidctl -l c0t0d0
Volume                  Size    Stripe  Status   Cache  RAID
        Sub                     Size                    Level
                Disk
----------------------------------------------------------------
c0t0d0                  558.8G  N/A     OPTIMAL  OFF    RAID1
                0.0.0   558.8G          GOOD
                0.1.0   558.8G          GOOD

For more information, see the raidctl(1M) man page.

Disk Slot Numbers

To perform a disk active replacement procedure, you must know the physical or logical device name for the drive that you want to install or remove. If your system encounters a disk error, often you can find messages about failing or failed disks in the system console. This information is also logged in the /var/adm/messages files.

These error messages typically refer to a failed disk drive by its physical device name (such as /pci@0,600000/pci@0/pci@0/scsi@0/disk@0) or by its logical device name (such as c0t0d0). In addition, some applications might report a disk slot number (0 through 3).

Device paths vary per server. Slot numbering on M8000/M9000 servers is a complex process. See Appendix A for more information. Slot numbering on M3000 servers is a bit simpler.

The paths in the following table were generated on an M3000 server. You can use the table associate internal disk slot numbers with the logical and physical device names for each disk drive.

TABLE 5-1 Disk Slot Numbers (M3000 Server)
Disk Slot Number	Logical Device Name ^[1]	Physical Device Name
Slot 0	`c0t0d0`	`/pci@0,600000/pci@0/pci@0/scsi@0/disk@0`
Slot 1	`c0t1d0`	`/pci@0,600000/pci@0/pci@0/scsi@0/disk@1`
Slot 2	`c0t2d0`	`/pci@0,600000/pci@0/pci@0/scsi@0/disk@2`
Slot 3	`c0t3d0`	`/pci@0,600000/pci@0/pci@0/scsi@0/disk@3`

Related Information

For additional information on this chapter's topics, see:

Resource	Information
man pages	`raidctl`(1M), `format`(1M), `cfgadm`(1M)
SPARC Enterprise M3000/M4000/M5000/M8000/M9000 Servers Product Notes	Patches
Oracle Solaris 10 Installation Guide	Network installation
SPARC Enterprise M3000 Server Service Manual	Remove and install disk drive
SPARC Enterprise M8000/M9000 Servers Service Manual	Remove and install disk drive

^{1 (TableFootnote) The logical device names might appear differently on your system, depending on the number and type of add-on disk controllers installed.}