This section provides the procedures for maintaining a StorEdge MultiPack enclosure. The following table lists these procedures.
Table 4-2 Task Map:Maintaining a StorEdge MultiPack Enclosure
Task |
For Instructions, Go To... |
---|---|
Add a disk drive Use the cfgadm(1M), devfsadm(1M), scgdevs(1M), and scdidadm(1M) commands to add a disk drive to an enclosure. | |
Replace a disk drive | |
Remove a disk drive | |
Add an enclosure to an existing cluster Use the nvedit(1M), shutdown(1M), scswitch(1M), and scdidadm commands to add an enclosure to an existing cluster. | |
Replace an enclosure in an existing cluster Use this procedure to replace an enclosure with a new enclosure. This procedure assumes that you want to retain the disk drives in the enclosure you want to replace. | |
Remove an enclosure from an existing cluster Use the cfgadm, devfsadm, and scdidadm commands to remove an enclosure without replacing it with another enclosure. |
Use this procedure to add a disk drive to an existing cluster. Perform the steps in this procedure in conjunction with the procedures in Sun Cluster 3.0 System Administration Guide and your server hardware manual.
For conceptual information on quorum, quorum devices, global devices, and device IDs, see Sun Cluster 3.0 Concepts.
Locate an empty disk slot in the enclosure for the disk drive you want to add.
Identify the empty slots either by observing the disk drive LEDs on the front of the enclosure, or by removing the left side cover of the unit. The target address IDs corresponding to the slots appear on the middle partition of the drive bay.
Install the disk drive.
For more information, see the documentation that shipped with your enclosure.
On all nodes attached to the enclosure, configure the disk drive.
# cfgadm -c configure cN # devfsadm |
On all nodes, ensure that entries for the disk drive have been added to the /dev/rdsk directory.
# ls -l /dev/rdsk |
If needed, use the format(1M) command or the fmthard(1M) command to partition the disk drive.
From any node, update the global device namespace.
If a volume management daemon such as vold is running on your node, and you have a CD-ROM drive connected to the node, a device busy error might be returned even if no disk is in the drive. This error is an expected behavior.
# scgdevs |
On all nodes, verify that a device ID (DID) has been assigned to the disk drive.
# scdidadm -l |
As shown in "Example--Adding a StorEdge MultiPack Disk Drive", the DID 35 assigned to the new disk drive might not be in sequential order in the disk array.
Perform volume management administration to add the new disk drive to the configuration.
For more information, see your Solstice DiskSuiteTM or VERITAS Volume Manager documentation.
If you want this new disk drive to be a quorum device, add the quorum device.
Refer to Sun Cluster 3.0 System Administration Guide for the procedure on adding a quorum device.
The following example shows how to apply the procedure for adding a StorEdge MultiPack enclosure disk drive.
# scdidadm -l 16 phys-circinus-3:/dev/rdsk/c2t0d0 /dev/did/rdsk/d16 17 phys-circinus-3:/dev/rdsk/c2t1d0 /dev/did/rdsk/d17 18 phys-circinus-3:/dev/rdsk/c2t2d0 /dev/did/rdsk/d18 19 phys-circinus-3:/dev/rdsk/c2t3d0 /dev/did/rdsk/d19 ... 26 phys-circinus-3:/dev/rdsk/c2t12d0 /dev/did/rdsk/d26 30 phys-circinus-3:/dev/rdsk/c1t2d0 /dev/did/rdsk/d30 31 phys-circinus-3:/dev/rdsk/c1t3d0 /dev/did/rdsk/d31 32 phys-circinus-3:/dev/rdsk/c1t10d0 /dev/did/rdsk/d32 33 phys-circinus-3:/dev/rdsk/c0t0d0 /dev/did/rdsk/d33 34 phys-circinus-3:/dev/rdsk/c0t6d0 /dev/did/rdsk/d34 8190 phys-circinus-3:/dev/rmt/0 /dev/did/rmt/2 # cfgadm -c configure c1 # devfsadm # scgdevs Configuring DID devices Could not open /dev/rdsk/c0t6d0s2 to verify device id. Device busy Configuring the /dev/global directory (global devices) obtaining access to all attached disks reservation program successfully exiting # scdidadm -l 16 phys-circinus-3:/dev/rdsk/c2t0d0 /dev/did/rdsk/d16 17 phys-circinus-3:/dev/rdsk/c2t1d0 /dev/did/rdsk/d17 18 phys-circinus-3:/dev/rdsk/c2t2d0 /dev/did/rdsk/d18 19 phys-circinus-3:/dev/rdsk/c2t3d0 /dev/did/rdsk/d19 ... 26 phys-circinus-3:/dev/rdsk/c2t12d0 /dev/did/rdsk/d26 30 phys-circinus-3:/dev/rdsk/c1t2d0 /dev/did/rdsk/d30 31 phys-circinus-3:/dev/rdsk/c1t3d0 /dev/did/rdsk/d31 32 phys-circinus-3:/dev/rdsk/c1t10d0 /dev/did/rdsk/d32 33 phys-circinus-3:/dev/rdsk/c0t0d0 /dev/did/rdsk/d33 34 phys-circinus-3:/dev/rdsk/c0t6d0 /dev/did/rdsk/d34 35 phys-circinus-3:/dev/rdsk/c2t13d0 /dev/did/rdsk/d35 8190 phys-circinus-3:/dev/rmt/0 /dev/did/rmt/2 # |
Use this procedure to replace a StorEdge MultiPack enclosure disk drive. "Example--Replacing a StorEdge MultiPack Disk Drive" shows you how to apply this procedure. Perform the steps in this procedure in conjunction with the procedures in Sun Cluster 3.0 System Administration Guide and your server hardware manual. Use the procedures in your server hardware manual to identify a failed disk drive.
For conceptual information on quorum, quorum devices, global devices, and device IDs, see Sun Cluster 3.0 Concepts.
Identify the disk drive that needs replacement.
If the disk error message reports the drive problem by device ID (DID), use the scdidadm -l command to determine the Solaris logical device name. If the disk error message reports the drive problem by the Solaris physical device name, use your Solaris documentation to map the Solaris physical device name to the Solaris logical device name. Use this Solaris logical device name and DID throughout this procedure.
# scdidadm -l deviceID |
Determine if the disk drive you want to replace is a quorum device.
# scstat -q |
If the disk drive you want to replace is a quorum device, remove the quorum device before you proceed. Otherwise, proceed to Step 3.
Refer to Sun Cluster 3.0 System Administration Guide for procedures on replacing a quorum device and putting a quorum device into maintenance state.
If possible, back up the metadevice or volume.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Perform volume management administration to remove the disk drive from the configuration.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Identify the failed disk drive's physical DID.
Use this physical DID in Step 12 to verify that the failed disk drive has been replaced with a new disk drive.
# scdidadm -o diskid -l cNtXdY |
If you are using Solstice DiskSuite as your volume manager, save the disk partitioning for use when partitioning the new disk drive.
If you are using VERITAS Volume Manager, proceed to Step 7.
# prtvtoc /dev/rdsk/cNtXdYsZ > filename |
Do not save this file under /tmp because you will lose this file when you reboot. Instead, save this file under /usr/tmp.
Replace the failed disk drive.
For more information, see the documentation that shipped with your enclosure.
On one node attached to the enclosure, run the devfsadm(1M) command to probe all devices and to write the new disk drive to the /dev/rdsk directory.
Depending on the number of devices connected to the node, the devfsadm command can take at least five minutes to complete.
# devfsadm |
If you are using Solstice DiskSuite as your volume manager, from any node connected to the enclosure, partition the new disk drive, using the partitioning you saved in Step 6.
If you are using VERITAS Volume Manager, proceed to Step 10.
# fmthard -s filename /dev/rdsk/cNtXdYsZ |
One at a time, shut down and reboot the nodes connected to the enclosure.
# scswitch -S -h nodename # shutdown -y -g 0 -i 6 |
For more information, see Sun Cluster 3.0 System Administration Guide.
From any node connected to the disk drive, update the DID database.
# scdidadm -R deviceID |
From any node, confirm that the failed disk drive has been replaced by comparing the new physical DID to the physical DID in Step 5.
If the new physical DID is different from the physical DID in Step 5, you successfully replaced the failed disk drive with a new disk drive.
# scdidadm -o diskid -l cNtXdY |
On all connected nodes, upload the new information to the DID driver.
If a volume management daemon such as vold is running on your node, and you have a CD-ROM drive connected to the node, a device busy error might be returned even if no disk is in the drive. This error is an expected behavior.
# scdidadm -ui |
Perform volume management administration to add the disk drive back to its diskset or disk group.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
If you want this new disk drive to be a quorum device, add the quorum device.
Refer to Sun Cluster 3.0 System Administration Guide for the procedure on adding a quorum device.
The following example shows how to apply the procedure for replacing a StorEdge MultiPack enclosure disk drive.
# scdidadm -l d20 20 phys-schost-2:/dev/rdsk/c3t2d0 /dev/did/rdsk/d20 # scdidadm -o diskid -l c3t2d0 5345414741544520393735314336343734310000 # prtvtoc /dev/rdsk/c3t2d0s2 > /usr/tmp/c3t2d0.vtoc ... # devfsadm # fmthard -s /usr/tmp/c3t2d0.vtoc /dev/rdsk/c3t2d0s2 # scswitch -S -h node1 # shutdown -y -g 0 -i 6 ... # scdidadm -R d20 # scdidadm -o diskid -l c3t2d0 5345414741544520393735314336363037370000 # scdidadm -ui |
Use this procedure to remove a disk drive from a StorEdge MultiPack enclosure. Perform the steps in this procedure in conjunction with the procedures in Sun Cluster 3.0 System Administration Guide and your server hardware manual.
For conceptual information on quorum, quorum devices, global devices, and device IDs, see Sun Cluster 3.0 Concepts.
Determine if the disk drive you want to remove is a quorum device.
# scstat -q |
If the disk drive you want to remove is a quorum device, remove the quorum device before you proceed. Otherwise, proceed to Step 2.
Refer to Sun Cluster 3.0 System Administration Guide for procedures on removing a quorum device and putting a quorum device into maintenance state.
Perform volume management administration to remove the disk drive from the configuration.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Identify the disk drive that needs to be removed and the slot that the disk drive needs to be removed from.
If the disk error message reports the drive problem by DID, use the scdidadm -l command to determine the Solaris device name.
# scdidadm -l deviceID # cfgadm -al |
Remove the disk drive.
For more information, see the documentation that shipped with your enclosure.
On all nodes, remove references to the disk drive.
# cfgadm -c unconfigure cN::dsk/cNtXdY # devfsadm -C # scdidadm -C |
Use this procedure to install a StorEdge MultiPack enclosure in an existing cluster. Perform the steps in this procedure in conjunction with the procedures in Sun Cluster 3.0 Installation Guide and your server hardware manual.
For conceptual information on multi-initiator SCSI and device IDs, see Sun Cluster 3.0 Concepts.
Ensure that each device in the SCSI chain has a unique SCSI address.
The default SCSI address for host adapters is 7. Reserve SCSI address 7 for one host adapter in the SCSI chain. This procedure refers to the host adapter you choose for SCSI address 7 as the host adapter on the second node. To avoid conflicts, in Step 7 you will change the scsi-initiator-id of the remaining host adapter in the SCSI chain to an available SCSI address. This procedure refers to the host adapter with an available SCSI address as the host adapter on the first node. Depending on the device and configuration settings of the device, either SCSI address 6 or 8 is usually available.
Even though a slot in the enclosure might not be in use, you should avoid setting the scsi-initiator-id for the first node to the SCSI address for that disk slot. This precaution minimizes future complications if you install additional disk drives.
For more information, see the OpenBoot 3.x Command Reference Manual and the labels inside the storage device.
Shut down and power off the first node.
# scswitch -S -h nodename # shutdown -y -g 0 |
For more information, see Sun Cluster 3.0 System Administration Guide.
Install the host adapters in the node.
For the procedure on installing host adapters, see the documentation that shipped with your host adapters and nodes.
Connect the single-ended SCSI cable between the node and the enclosures, as shown in Figure 4-2.
Make sure that the entire SCSI bus length to each enclosure is less than 6 m. This measurement includes the cables to both nodes, as well as the bus length internal to each enclosure, node, and host adapter. Refer to the documentation that shipped with the enclosure for other restrictions regarding SCSI operation.
Temporarily install a single-ended terminator on the SCSI IN port of the second enclosure, as shown in Figure 4-2.
Connect the enclosures to different power sources.
Power on the node and the enclosures.
Find the paths to the host adapters.
{0} ok show-disks a) /pci@1f,4000/pci@4/SUNW,isptwo@4/sd b) /pci@1f,4000/pci@2/SUNW,isptwo@4/sd |
Identify and record the two controllers that will be connected to the storage devices, and record these paths. Use this information to change the SCSI addresses of these controllers in the nvramrc script. Do not include the /sd directories in the device paths.
Edit the nvramrc script to set the scsi-initiator-id for the host adapters on the first node.
For a list of nvramrc editor and nvedit keystroke commands, see Appendix B, NVRAMRC Editor and NVEDIT Keystroke Commands.
The following example sets the scsi-initiator-id to 6. The OpenBoot PROM Monitor prints the line numbers (0:, 1:, and so on).
Insert exactly one space after the first double quote and before scsi-initiator-id.
{0} ok nvedit 0: probe-all 1: cd /pci@1f,4000/pci@4/SUNW,isptwo@4 2: 6 " scsi-initiator-id" integer-property 3: device-end 4: cd /pci@1f,4000/pci@2/SUNW,isptwo@4 5: 6 " scsi-initiator-id" integer-property 6: device-end 7: install-console 8: banner <Control C> {0} ok |
Store the changes.
The changes you make through the nvedit command are done on a temporary copy of the nvramrc script. You can continue to edit this copy without risk. After you complete your edits, save the changes. If you are not sure about the changes, discard them.
To store the changes, type:
{0} ok nvstore {0} ok |
To discard the changes, type:
{0} ok nvquit {0} ok |
Verify the contents of the nvramrc script you created in Step 9.
If the contents of the nvramrc script are incorrect, use the nvedit command to make corrections.
{0} ok printenv nvramrc nvramrc = probe-all cd /pci@1f,4000/pci@4/SUNW,isptwo@4 6 " scsi-initiator-id" integer-property device-end cd /pci@1f,4000/pci@2/SUNW,isptwo@4 6 " scsi-initiator-id" integer-property device-end install-console banner {0} ok |
Instruct the OpenBoot PROM Monitor to use the nvramrc script.
{0} ok setenv use-nvramrc? true use-nvramrc? = true {0} ok |
Boot the first node and wait for it to join the cluster.
{0} ok boot -r |
For more information, see Sun Cluster 3.0 System Administration Guide.
On all nodes, verify that the DIDs have been assigned to the disk drives in the enclosure.
# scdidadm -l |
Shut down the second node.
# scswitch -S -h nodename # shutdown -y -g 0 |
Power off the second node.
Install the host adapters in the second node.
For the procedure on installing host adapters, see the documentation that shipped with your nodes.
Connect the enclosures to the host adapters by using single-ended SCSI cables.
Without allowing the node to boot, power on the second node. If necessary, abort the system to continue with OpenBoot PROM Monitor tasks.
Verify that the second node sees the new host adapters and disk drives.
{0} ok show-disks |
Verify that the scsi-initiator-id for the host adapter on the second node is set to 7.
Use the show-disks command to find the paths to the host adapters connected to these enclosures. Select each host adapter's device tree node, and display the node's properties to confirm that the scsi-initiator-id for each host adapter is set to 7.
{0} ok cd /pci@1f,4000/pci@4/SUNW,isptwo@4 {0} ok .properties ... scsi-initiator-id 00000007 ... {0} ok cd /pci@1f,4000/pci@2/SUNW,isptwo@4 {0} ok .properties ... scsi-initiator-id 00000007 |
Boot the second node and wait for it to join the cluster.
{0} ok boot -r |
On all nodes, verify that the DIDs have been assigned to the disk drives in the enclosure.
# scdidadm -l |
Perform volume management administration to add the disk drives in the array to the volume management configuration.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Use this procedure to replace a StorEdge MultiPack enclosure. This procedure assumes that you want to retain the disk drives in the enclosure you want to replace and retain the references to these same disk drives.
If you want to replace your disk drives, see "How to Replace a StorEdge MultiPack Disk Drive".
If possible, back up the metadevices or volumes that reside in the disk array.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Perform volume management administration to remove the disk array from the configuration.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Disconnect the SCSI cables from the enclosure, removing them in the order shown in Figure 4-4.
Power off, and disconnect the enclosure from the AC power source.
For more information, see the documentation that shipped with your enclosure and the labels inside the lid of the enclosure.
Connect the new enclosure to an AC power source.
Refer to the documentation that shipped with the enclosure as well as the labels inside the lid of the enclosure.
Connect the SCSI cables to the new enclosure by reversing the steps shown in Figure 4-4.
Move the disk drives one at time from the old enclosure to the same slot in the new enclosure.
Power on the enclosure.
On all nodes attached to the enclosure, run the devfsadm(1M) command.
# devfsadm |
One at a time, shut down and reboot the nodes connected to the enclosure.
# scswitch -S -h nodename # shutdown -y -g 0 -i 6 |
For more information on shutdown(1M), see Sun Cluster 3.0 System Administration Guide.
Perform volume management administration to add the disk array to the configuration.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Use this procedure to remove a StorEdge MultiPack enclosure from a cluster.This procedure assumes that you want to remove the references to the disk drives in the enclosure.
Perform volume management administration to remove the enclosure from the configuration.
For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.
Disconnect the SCSI cables from the enclosure, removing them in the order shown in Figure 4-5.
Power off, and disconnect the enclosure from the AC power source.
For more information, see the documentation that shipped with the enclosure and the labels inside the lid of the enclosure.
Remove the enclosure.
Identify the disk drives you need to remove.
# cfgadm -al |
On all nodes, remove references to the disk drives in the enclosure you removed in Step 4.
# cfgadm -c unconfigure cN::dsk/cNtXdY # devfsadm -C # scdidadm -C |
If needed, remove any host adapters from the nodes.
For the procedure on removing host adapters, see the documentation that shipped with your nodes.