SPARC: How to Add a Storage Array to an Existing SPARC Based Cluster (Sun Cluster 3.0-3.1 With SCSI JBOD Storage Device Manual for Solaris OS)

Sun Cluster 3.0-3.1 With SCSI JBOD Storage Device Manual for Solaris OS

SPARC: How to Add a Storage Array to an Existing SPARC Based Cluster

This procedure contains instructions for adding storage arrays to an operational cluster. If you need to install storage arrays to a new cluster, use the procedure in SPARC: How to Install a Storage Array in a New SPARC Based Cluster or x86: How to Install a Storage Array in a New X86 Based Cluster.

Adding a storage array enables you to alter your storage pool. You might want to perform this procedure in the following scenarios.

You need to increase your storage pool.
You need to upgrade to a higher-quality or to a larger storage array.

To upgrade storage arrays, remove the old storage array and then add the new storage array.

To replace a storage array with the same type of storage array, see How to Replace the Chassis.

This procedure defines Node A as the node with which you begin working. Node B is the remaining node.

Before You Begin

This procedure relies on the following prerequisites and assumptions.

Your cluster is operational and all nodes are powered on.
Your nodes are not configured with dynamic reconfiguration functionality.

If your nodes are configured for dynamic reconfiguration, see the Sun Cluster system administration documentation, and skip steps that instruct you to shut down the node.

Steps

Install all software that is specific to the storage array or to any new host adapters.

Install the software and patches to all nodes that will connect to the new storage array.

PatchPro is a patch-management tool that eases the selection and download of patches required for installation or maintenance of Sun Cluster software. PatchPro provides an Interactive Mode tool especially for Sun Cluster. The Interactive Tool makes the installation of patches easier. PatchPro's Expert Mode tool helps you to maintain your configuration with the latest set of patches. Expert Mode is especially useful for obtaining all of the latest patches, not just the high availability and security patches.

To access the PatchPro tool for Sun Cluster software, go to http://www.sun.com/PatchPro/, click Sun Cluster, then choose either Interactive Mode or Expert Mode. Follow the instructions in the PatchPro tool to describe your cluster configuration and download the patches.

For third-party firmware patches, see the SunSolve^SM Online site at http://sunsolve.ebay.sun.com.

Move all resource groups and device groups off Node A.
# scswitch -S -h from-node

If you need to install host adapters in Node A, perform the following steps.
1. Shut down and power off Node A.
  
  For the procedure about how to shut down and power off a node, see the Sun Cluster system administration documentation.
2. Install host adapters in Node A.
  
  For the procedure about how to install host adapters, see your host adapters and server documentation.

Connect the storage array to the host adapters on Node A.
- If necessary, terminate the ports that will connect to Node B.
  - If you have a Netra^TM D130 array, always terminate the ports that connect to Node B.
  - If you have a StorEdge 3310 or 3320 SCSI array, terminate the ports that connect to Node B when using a split-bus configuration.
- If your storage array is a StorEdge 3310 or 3320 SCSI array, do not power on the storage array until the storage array is cabled to Node A.
For cabling diagrams, see Chapter 3, Cabling Diagrams.

Note –
Ensure that the bus length does not exceed SCSI bus-length specifications. This measurement includes the cables to both nodes, as well as the bus length that is internal to each storage array, node, and the host adapter. For more information about SCSI bus-length limitations, see your hardware documentation.

Ensure that each device in the SCSI chain has a unique SCSI address.

To avoid SCSI-chain conflicts, the following steps instruct you to reserve SCSI address 7 for one host adapter in the SCSI chain and change the other host adapter's global scsi-initiator-id to an available SCSI address. Then the steps instruct you to change the scsi-initiator-id for local devices back to 7.

Note –

A slot in the storage array might not be in use. However, do not set the scsi-initiator-id to a SCSI address for that disk slot. This precaution minimizes future complications if you install additional disk drives.

If necessary, power on the storage devices.

For the procedure about powering on a storage device, see the service manual that shipped with your storage device.

Note –
If necessary, halt the nodes so that you can perform OpenBoot^TM PROM (OBP) Monitor tasks at the ok prompt.

If necessary, power on Node A, but do not allow it to boot. If necessary, halt the system to continue with OBP Monitor tasks.

Set the scsi-initiator-id for Node A to 6.

{1} ok setenv scsi-initiator-id 6
scsi-initiator-id = 6

Find the paths to the host adapters that connect to the local disk drives.
{0} ok show-disks
Use this information to change the SCSI addresses in the nvramrc script. Do not include the /sd directories in the device paths.

Edit the nvramrc script to set the scsi-initiator-id for the local devices on the first node to 7.

For a full list of commands, see the OpenBoot 2.x Command Reference Manual.

Caution –

Insert exactly one space after the first double quote and before scsi-initiator-id.

{0} ok nvedit
 0: probe-all 
 1: cd /pci@1f,4000/scsi@2
 2: 7 encode-int " scsi-initiator-id" property
 3: device-end 
 4: cd /pci@1f,4000/scsi@3 
 5: 7 encode-int " scsi-initiator-id" property 
 6: device-end
 7: install-console
 8: banner[Control C] 
{0} ok

Store the changes.

The changes you make through the nvedit command are recorded on a temporary copy of the nvramrc script. You can continue to edit this copy without risk. After you complete your edits, save the changes. If you are not sure about the changes, discard them.
- To store the changes, type the following command:
  {0} ok nvstore {1} ok
- To discard the changes, type the following command:
  {0} ok nvquit {1} ok

Verify the contents of the nvramrc script that you created, as shown in the following example.

If the contents of the nvramrc script are incorrect, use the nvedit command to make corrections.

{0} ok printenv nvramrc
nvramrc =             probe-all
                      cd /pci@1f,4000/scsi@2 
                      7 " scsi-initiator-id" integer-property
                      device-end 
                      cd /pci@1f,4000/scsi@3
                      7 " scsi-initiator-id" integer-property
                      device-end 
                      install-console
                      banner
{1} ok

Instruct the OpenBoot PROM (OBP) Monitor to use the nvramrc script, as shown in the following example.
{0} ok setenv use-nvramrc? true use-nvramrc? = true {1} ok

Perform a reconfiguration boot on Node A to create the new Solaris device files and links.

If necessary, format and label the disks.

On Node A, verify that the device IDs have been assigned to the disk drives in the storage array.

# scdidadm -l

Move all resource groups and device groups off Node B.
# scswitch -S -h from-node

If you need to install host adapters in Node B, perform the following steps.
1. Shut down Node B.
  
  For the procedure about how to shut down and power off a node, see the Sun Cluster system administration documentation.
2. Install the host adapters in Node B.
  
  For the procedure about how to install a host adapter, see your host adapter and server documentation.
3. Power on and boot Node B.

Connect the storage array to the host adapters on Node B.

If you added port terminators in Step 4, remove the terminator ports and connect the storage array to Node B.

For cabling diagrams, see Chapter 3, Cabling Diagrams.

Note –
Ensure that the bus length does not exceed bus-length specifications. This measurement includes the cables to both nodes, as well as the bus length that is internal to each storage array, node, and the host adapter. For more information about bus-length limitations, see your hardware documentation.

Verify that the scsi-initiator-id is set correctly on the second node.
1. If necessary, power on the second node, but do not allow it to boot. If necessary, halt the system to continue with OBP Monitor tasks.
2. Verify that the scsi-initiator-id for each host adapter on the second node is set to 7.
  
  Use the show-disks command to find the paths to the host adapters that are connected to these enclosures. Select each host adapter's device tree node, and display the node's properties to confirm that the scsi-initiator-id for each host adapter is set to 7.
  {0} ok cd /pci@6,4000/pci@3/scsi@5 {0} ok .properties scsi-initiator-id 00000007 ...

Perform a reconfiguration boot to create the new Solaris device files and links.

On Node B, verify that the device IDs have been assigned to the disk drives in the storage array.
# scdidadm -L

Perform volume management administration to add the disk drives in the storage array to the volume management configuration.

For more information, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

Next Steps

If needed, finish setting up your storage arrays, including partitions. If you are using Solstice DiskSuite^TM/Solaris Volume Manager as your volume manager, save the disk-partitioning information. You might need disk-partitioning information if you replace a failed disk drive in the future.

Caution –

Do not save disk-partitioning information in /tmp because you will lose this file when you reboot. Instead, save this file in /usr/tmp.