Managing Hard Disk Drives

C H A P T E R 11

This chapter discusses the following topics.

Hard Disk Drive Locations

Disk Drive Status and LEDs

EFI Disk Label Overview

Converting EFI Label to SMI (Solaris) Label on the Sun Fire X4540 Disk

About HDtool

Removing a Disk From Service

Correcting Unconfigure Operation Failure

Adding a Disk

Checking Disk Usage

Hard Disk Drive Locations

The Sun Fire X4540 server can contain up to 48 SATA hard disk drives. The hard disk drive locations are numbered sequentially from 0 to 47, starting at the front left corner and incrementing left to right and front to rear (see FIGURE 11-1). The nomenclature for the locations is DISKn, where n is the location number.

FIGURE 11-1 Disk Drive Locations (Boot disks shown in gray)

Diagram showing the locations of the disk drive and boot disks.

Disk Drive Status and LEDs

Each hard disk drive has a sensor that is used to communicate the state of the slot. The hard disk drives use IPMI (Intelligent Platform Management Interface) sensors to convey a slot state, as shown in TABLE 11-1:

TABLE 11-1 Disk Drive Status Sensors
State	Event	Description
0	Device Not Present	The drive bay is empty.
1	Device Installed	The drive is detected. Used by remote management applications.
2	Ready for Device Removal	A drive is unmounted and ready to be physically removed. Controls the OK to Remove LED.
3	Device Faulted	Causes the service processor to illuminate the individual disk drive fault LED.

Inside the Sun Fire X4540 server chassis, each of the 48 hard disk drives has three LEDs, as shown in TABLE 11-2.

The individual LED locator can be used to control both the fault and removal LEDs through an IPMI OEM command. The service processor controls the fault and removal LEDs automatically, based on events in the disk drive sensors.

TABLE 11-2 Disk Drive LED Indicators
LED	Event	Description
Green	Disk drive activity	Controlled by hardware. Does not require SP interaction.
Amber	Hard disk drive failure	Controlled by the SP over the SP-12C bus. Relies on the operating system driver to set appropriately.
Blue	Hard disk drive ready for removal	Controlled by the SP over the SP-12C bus. The operating system determines that the drive has been halted and is ready to be removed.

EFI Disk Label Overview

Extensible Firmware Interface (EFI) is an Intel standard used as a replacement for the PC BIOS. It is responsible for the power-on self-test (POST) process, booting the operating system, and providing an interface between the operating system and the physical hardware. EFIs provides the following capabilities:

System-independent device drivers.

Delegation of networking and memory management issues to the firmware instead of the OS.

EFI selection and loading of the operating system, which removes the need for a boot loader.

The Solaris 10 OS provides support for EFI Labels for disks that are larger than 1 terabyte on systems that run a 64-bit Solaris kernel. The Extensible Firmware Interface GUID Partition Table (EFI GPT) disk label provides support for physical disks and virtual disk volumes.

You can download the EFI specification at:

http://www.intel.com/technology/efi/main_specification.htm

You can use the format -e command to apply an EFI label to a disk if the system is running the appropriate Solaris release. However, you should review the important information in Restrictions of the EFI Disk Label before attempting to apply an EFI label.

For additional information about EFI disk labels, managing disks with EFI labels, EFI disk label restrictions, and troubleshooting problems with EFI disk labels, refer to the Solaris 10 Systems Administration Guide at:

http://docs.sun.com

Converting EFI Label to SMI (Solaris) Label on the Sun Fire X4540 Disk

The ZFS system uses EFI labels to label whole disks that are added to a ZFS storage pool. If you convert an EFI label on disk that is part of a ZFS storage pool, you can erase the data on that disk.

Caution - Do not attempt to convert an EFI label to an SMI label using theformat(1M) command.

To convert an EFI disk label to an SMI (Solaris) label, delete the EFI fdisk partition, then create a new Solaris fdisk partition.

1. Use fdisk to delete the EFI fdisk partition.

CODE EXAMPLE 11-1 EFI to SMI Disk Label Conversion
# `fdisk /dev/rdsk/c0t7d0p0` Total disk size is 30400 cylinders Cylinder size is 16065 (512 byte) blocks Cylinders Partition Status Type Start End Length % ========= ====== ============ ===== === ====== === 1 EFI 0 30400 30401 100 SELECT ONE OF THE FOLLOWING: 1. Create a partition 2. Specify the active partition 3. Delete a partition 4. Change between Solaris and Solaris2 Partition IDs 5. Exit (update disk configuration and exit) 6. Cancel (exit without updating disk configuration) Enter Selection: `3` Specify the partition number to delete (or enter 0 to exit): `1` Are you sure you want to delete partition 1? This will make all files and programs in this partition inaccessible (type "y" or "n"). `y` Total disk size is 30400 cylinders Cylinder size is 16065 (512 byte) blocks Cylinders Partition Status Type Start End Length % ========= ====== ============ ===== === ====== === WARNING: no partitions are defined! (The partition is now deleted. The menu reappears, as shown in Step 2.)

CODE EXAMPLE 11-1 EFI to SMI Disk Label Conversion

 # fdisk /dev/rdsk/c0t7d0p0
            Total disk size is 30400 cylinders
            Cylinder size is 16065 (512 byte) blocks
 
                                              Cylinders
     Partition   Status    Type          Start   End   Length    %
     =========   ======    ============  =====   ===   ======   ===
         1                 EFI               0  30400    30401    100
 
 
SELECT ONE OF THE FOLLOWING:
  1. Create a partition
  2. Specify the active partition
  3. Delete a partition
  4. Change between Solaris and Solaris2 Partition IDs
  5. Exit (update disk configuration and exit)
  6. Cancel (exit without updating disk configuration)
Enter Selection: 3
 
Specify the partition number to delete (or enter 0 to exit): 1
 
Are you sure you want to delete partition 1? This will make all files and programs in this partition inaccessible (type "y" or "n"). y
 
 
            Total disk size is 30400 cylinders
            Cylinder size is 16065 (512 byte) blocks
 
                                              Cylinders
     Partition   Status    Type          Start   End   Length    %
     =========   ======    ============  =====   ===   ======   ===
WARNING: no partitions are defined!

(The partition is now deleted. The menu reappears, as shown in Step 2.)

2. Create the new partition.

(Continued from previous display)
 
SELECT ONE OF THE FOLLOWING:
  1. Create a partition
  2. Specify the active partition
  3. Delete a partition
  4. Change between Solaris and Solaris2 Partition IDs
  5. Exit (update disk configuration and exit)
  6. Cancel (exit without updating disk configuration)
Enter Selection: 1
Select the partition type to create:
  1=SOLARIS2  2=UNIX        3=PCIXOS     4=Other
  5=DOS12     6=DOS16       7=DOSEXT     8=DOSBIG
  9=DOS16LBA  A=x86 Boot    B=Diagnostic C=FAT32
  D=FAT32LBA  E=DOSEXTLBA   F=EFI        0=Exit? 
 
Specify the percentage of disk to use for this partition
(or type "c" to specify the size in cylinders). 100
 
Should this become the active partition? If yes, it  will be activated each time the computer is reset or turned on.
 
Please type "y" or "n". y
 
            Total disk size is 30400 cylinders
            Cylinder size is 16065 (512 byte) blocks
 
                                              Cylinders
     Partition   Status    Type          Start   End   Length    %
     =========   ======    ============  =====   ===   ======   ===
         1       Active    Solaris2          1  30399    30399    100
 
SELECT ONE OF THE FOLLOWING:
  1. Create a partition
  2. Specify the active partition
  3. Delete a partition
  4. Change between Solaris and Solaris2 Partition IDs
  5. Exit (update disk configuration and exit)
  6. Cancel (exit without updating disk configuration)
Enter Selection: 5

3. Verify that Solaris2 fdisk has been created on the same disk.

# fdisk /dev/rdsk/c0t7d0p0
 
            Total disk size is 30400 cylinders
            Cylinder size is 16065 (512 byte) blocks
 
                                              Cylinders
     Partition   Status    Type          Start   End   Length    %
     =========   ======    ============  =====   ===   ======   ===
         1       Active    Solaris2          1  30399    30399    100
 
 
SELECT ONE OF THE FOLLOWING:
  1. Create a partition
  2. Specify the active partition
  3. Delete a partition
  4. Change between Solaris and Solaris2 Partition IDs
  5. Exit (update disk configuration and exit)
  6. Cancel (exit without updating disk configuration)
Enter Selection: 5

The above display confirms that the Solaris2 fdsik has been created.

For additional information about converting EFI and SMI disk labels, refer to the Solaris 10 Systems Administration Guide at:

http://docs.sun.com

About HDtool

You can use the HDtool utility to view the hard drive configuration. The following example shows the Sun Fire X4540 disks in the HDtool display.

CODE EXAMPLE 11-2 HDtool Sample
---------------------SunFireX4540-------Rear-------------------- 3: 7: 11: 15: 19: 23: 27: 31: 35: 39: 43: 47: c0t3 c0t7 c1t3 c1t7 c2t3 c2t7 c3t3 c3t7 c4t3 c4t7 c5t3 c5t7 ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ 2: 6: 10: 14: 18: 22: 26: 30: 34: 38: 42: 46: c0t2 c0t6 c1t2 c1t6 c2t2 c2t6 c3t2 c3t6 c4t2 c4t6 c5t2 c5t6 ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ 1: 5: 9: 13: 17: 21: 25: 29: 33: 37: 41: 45: c0t1 c0t5 c1t1 c1t5 c2t1 c2t5 c3t1 c3t5 c4t1 c4t5 c5t1 c5t5 ^b+ ^++ ^b+ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ 0: 4: 8: 12: 16: 20: 24: 28: 32: 36: 40: 44: c0t0 c0t4 c1t0 c1t4 c2t0 c2t4 c3t0 c3t4 c4t0 c4t4 c5t0 c5t4 ^b+ ^++ ^b+ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ -------------------SunFireX4540------Front-------------*----

CODE EXAMPLE 11-2 HDtool Sample

---------------------SunFireX4540-------Rear--------------------
 
 3:    7:   11:   15:   19:   23:   27:   31:   35:   39:   43:   47:
c0t3  c0t7  c1t3  c1t7  c2t3  c2t7  c3t3  c3t7  c4t3  c4t7  c5t3  c5t7
^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++
 2:    6:   10:   14:   18:   22:   26:   30:   34:   38:   42:   46:
c0t2  c0t6  c1t2  c1t6  c2t2  c2t6  c3t2  c3t6  c4t2  c4t6  c5t2  c5t6
^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++
 1:    5:    9:   13:   17:   21:   25:   29:   33:   37:   41:   45:
c0t1  c0t5  c1t1  c1t5  c2t1  c2t5  c3t1  c3t5  c4t1  c4t5  c5t1  c5t5
^b+   ^++   ^b+   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++
 0:    4:    8:   12:   16:   20:   24:   28:   32:   36:   40:   44:
c0t0  c0t4  c1t0  c1t4  c2t0  c2t4  c3t0  c3t4  c4t0  c4t4  c5t0  c5t4
^b+   ^++   ^b+   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++   ^++
-------*-----------*-SunFireX4540---*---Front----*---------*----

For additional information about installing and using HDtool, refer to the x64 Servers Utilities Reference Manual at:

http://docs.sun.com

Removing a Disk From Service

Caution - You must follow these steps before removing a disk from service. Failure to follow the procedure can corrupt your data or render your file system inoperable.

1. Assume you know that the logical disk node is c4t0d0. Type the following command:

# cfgadm -al| grep c4t0d0

The physical slot is displayed, showing where the disk is connected. For example, this hard disk is attached to SATA controller 2, and port 0:

c0::dsk/c0t4d0 disk         connected    configured   unknown

2. Unconfigure the disk to turn on Blue "Ready to Remove" LEDbefore removal.

To unconfigure the disk, you must suspend activity on the SATA device. For example, type the following command:

# cfgadm -c unconfigure c0::dsk/c0t4d0

3. Verify that the disk has been unconfigured by typing the following command:

# cfgadm -al | grep c0t4d0

The following information shows that the disk has been unconfigured:

c0::dsk/c0t4d0 disk         connected    unconfigured unknown

Note - The blue LEDs indicate that the disks that are safe to remove.

4. Remove the disk from the chassis.

Note - If the process of unconfiguring the disk failed, the disk might be in use by ZFS, UFS, or some other entity. See Checking Disk Usage.

Correcting Unconfigure Operation Failure

If a disk unconfigure operation fails, check to see if the system is in the correct state, and that a utility is not using the disk. See Checking Disk Usage.

When unconfiguring a disk that is part of a ZFS storage pool, follow these guidelines:

Disks can be replaced or detached in a ZFS mirrored storage pool.

Disks can only be replaced in a ZFS RAID-Z storage pool.

For more information about detaching or replacing disks in a storage pool, refer to the ZFS Administration Guide, (819-5461).

Adding a Disk

This process assumes you have physically inserted a disk and now want to bring the inserted disk online.

If you are replacing a bootable disk that is mirrored with the Solaris Volume Manager software, use the Solaris Volume Manager to enable the disk. For additional information, refer to the Solaris Volume Manager Administration Guide (819-2789).

To add a disk:

1. Determine the attachment point by typing the following command:

# cfgadm -al > cfgadm_snapshot

Note - Before you insert the disk, predetermine which attachment point the disk is being inserted into . Refer to FIGURE 8-4 for a list of disk drives.

2. Insert the disk.

3. Type the following command:

# cfgadm -al > cfgadm_snapshot_2

4. Compare the two files by typing the following command:

# diff cfgadm_snapshot cfgadm_snapshot_2

Information similar to the following is displayed:

7c7
< c0::dsk/c0t4d0 disk         connected    unconfigured unknown
....
> c0::dsk/c0t4d0 disk         connected    configured   unknown

From this information you determine that the inserted drive uses SATA port 4 on controller 0.

5. Remove the two temporary files by typing the following command:

# rm cfgadm_snapshot cfgadm_snapshot_2

6. The Solaris OS configures the disk automatically. (S10 05/08 or later)

Note - The S10 U5 support hot-plug feature, "cfgadm -c configure c5::dsk/c5t7d0" is optional for adding a disk into system.

7. Verify that the blue LED on the disk turns off after one minute.

If the blue LED does not turn off after one minute, you can have the OS reenumerate device nodes and links by typing: # devfsadm -c.

Checking Disk Usage

To determine if ZFS, UFS, or another utility is using the disk:

1. To determine whether the disk is in use, type the following command:

# cfgadm | grep c4t0d0

The following information is displayed:

 sata2/0::dsk/c4t0d0            disk         connected    configured   ok

2. Identify if ZFS is using a disk by typing the following command:

# zpool status | grep c4t0d0

The following example shows that ZFS is using the disk:

c4t0d0    ONLINE       0     0     0

Note - The disk must detached from the pool or replaced with another disk before the disk can be unconfigured. For more information about detaching or replacing disks in a storage pool, see the ZFS Administration Guide, (819-5461).

Note - If the disk cannot be moved offline from the ZFS pool, then you can either destroy or export the pool to remove the disk. See the ZFS Administration Guide, (819-5461) for more details.

3. To stop ZFS from using the disk, type the following command:

# zpool export <pool>

4. To verify that ZFS is no longer using the disk, type the following command:

# cfgadm -c unconfigure c0::dsk/c0t4d0
# cfgadm -al | grep c0t4d0

The following message is displayed:

c0::dsk/c0t4d0 disk         connected    unconfigured unknown

5. You can now remove the hard drive.