Maintaining Storage Systems

This section contains the procedures for maintaining a storage system in a running cluster. Table 1-3 lists these procedures.

Table 1-3 Task Map: Maintaining Storage Systems

StorEdge 6320 System FRUs

The following is a list of administrative tasks that require no cluster-specific procedures. See the Sun StorEdge 6320 System Reference and Service Manual for the following procedures.

• Adding a disk drive.

• Replacing a failed disk drive.

• Removing a disk drive.

• Replacing an Ethernet cable.

• Replacing an Ethernet hub in a cabinet.

• Replacing the power cable on the storage array.

• Replacing a power and cooling unit (PCU).

• Replacing a storage service processor.

• Replacing a storage service processor accessory tray.

• Replacing a service processor panel.

• Replacing a USB relay panel.

• Replacing a USB flash disk.

• Extending a dual-controller configuration.

• Removing a dual-controller configuration.

• Replacing a chassis.

• Replacing a controller.

• Replacing a storage array-to-switch component in a dual-controller configuration.

  A storage array-to-switch component includes all of the following components:

  • Fiber-optic cable that connects an FC switch to a storage array

  • FC switch GBIC or an SFP that connects an FC switch to a storage array

  • FC switch that connects an FC switch to a storage array

  • FC switch power cord

  • Interconnect cables that connect two storage arrays

  • Interconnect cards

How to Upgrade Storage Array Firmware

Use this procedure to upgrade storage array firmware in a running cluster. Storage array firmware includes controller firmware, unit interconnect card (UIC) firmware, EPROM firmware, and disk drive firmware.

device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

1. Stop all I/O to the storage arrays you are upgrading.
2. Apply the controller, disk drive, and loop-card firmware patches by using the arrays' GUI tools.

   For the list of required patches, see the Sun StorEdge 6320 System Reference and Service Manual. For the procedure about how to apply firmware patches, see the firmware patch README file. For the procedure about how to verify the firmware level, see the Sun StorEdge 6320 System Reference and Service Manual.

   For specific instructions, see your storage array's documentation.

3. Confirm that all storage arrays that you upgraded are visible to all nodes.

   # luxadm probe

4. Restart all I/O to the storage arrays.

   You stopped I/O to these storage arrays in Step 1.

How to Remove a Storage System

Use this procedure to permanently remove a storage system from a running cluster.

This procedure defines Node N as the node that is connected to the storage system you are removing and the node with which you begin working.

---

Caution - During this procedure, you lose access to the data that resides on the storage system that you are removing.

---

1. If necessary, back up all database tables, data services, and volumes that are associated with each partner group that is affected.
2. Remove references to the volumes that reside on the storage system that you are removing.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

3. Disconnect the cables that connected Node N to the FC switches in your storage system.
4. On all nodes, remove the obsolete Solaris links and device IDs (DIDs).

   # devfsadm -C
   # cldevice clear

5. Repeat Step 3 through Step 4 for each node that is connected to the storage system.

Replacing a Node-to-Switch Component

Use this procedure to replace a node-to-switch component that has failed or that you suspect might be contributing to a problem.

This procedure defines Node A as the node that is connected to the node-to-switch component that you are replacing. This procedure assumes that, except for the component you are replacing, your cluster is operational.

Ensure that you are following the appropriate instructions:

• If your cluster uses multipathing, see How to Replace a Node-to-Switch Component in a Cluster That Uses Multipathing.

• If your cluster does not use multipathing, see How to Replace a Node-to-Switch Component in a Cluster Without Multipathing.

How to Replace a Node-to-Switch Component in a Cluster That Uses Multipathing

1. If your configuration is active-passive, and if the active path is the path that needs a component replaced, make that path passive.
2. Replace the component.

   Refer to your hardware documentation for any component-specific instructions.

3. (Optional) If your configuration is active-passive and you changed your configuration in Step 1, switch your original data path back to active.

How to Replace a Node-to-Switch Component in a Cluster Without Multipathing

Before You Begin

This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.

1. Become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.
2. If the physical data path has failed, do the following:
   1. Replace the component.
   2. Fix the volume manager error that was caused by the failed data path.
   3. (Optional) If necessary, return resource groups and device groups to this node.

   You have completed this procedure.

3. If the physical data path has not failed, determine the resource groups and device groups that are running on Node A.

   # clresourcegroup status -n NodeA
   # cldevicegroup status -n NodeA

   -n NodeA

   The node for which you are determining resource groups and device groups.

4. Move all resource groups and device groups to another node.

   # clnode evacuate nodename

5. Replace the node-to-switch component.

   Refer to your hardware documentation for any component-specific instructions.

6. (Optional) Restore the device groups to the original node.

   Do the following for each device group that you want to return to the original node.

   # cldevicegroup switch -n nodename devicegroup1[ devicegroup2 ...]

   -n nodename

   The node to which you are restoring device groups.

   devicegroup1[ devicegroup2 …]

   The device group or groups that you are restoring to the node.

7. (Optional) Restore the resource groups to the original node.

   Do the following for each resource group that you want to return to the original node.

   # clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]

   nodename

   For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

   resourcegroup1[ resourcegroup2 …]

   The resource group or groups that you are returning to the node or nodes.

How to Replace a Chassis

Use this procedure to replace a storage array chassis in a running cluster. This procedure assumes that you want to retain all FRUs other than the chassis and the backplane. To replace the chassis, you must replace both the chassis and the backplane. These components are manufactured as one part.

---

Caution - You must be an Oracle service provider to perform this procedure. If you need to replace a storage array chassis or a storage array midplane, contact your Oracle service provider.

---

1. Detach the submirrors on the storage array that is connected to the chassis and the midplane that you are replacing. Detach the submirrors to stop all I/O activity to this storage array.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

2. Are the storage arrays in your partner–group configuration redundant as a result of host-based mirroring?

   • If yes, proceed to Step 3.

   • If no, shut down the cluster.

     For the procedure about how to shut down a cluster, see your Oracle Solaris Cluster system administration documentation.

3. Replace the chassis and the midplane.

   For the procedure about how to replace a storage array chassis and a storage array midplane, see the Sun StorEdge 6320 System Reference and Service Manual.

4. Did you shut down the cluster in Step 2?

   • If no, proceed to Step 5.

   • If yes, boot the cluster back into cluster mode.

     For the procedure about how to boot a cluster, see your Oracle Solaris Cluster system administration documentation.

5. Reattach the submirrors that you detached in Step 1 to resynchronize the submirrors.

   ---

   Caution - The world wide numbers (WWNs) change as a result of this procedure. You must reconfigure your volume manager software to recognize the new WWNs.

   ---

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

How to Replace a Host Adapter

Use this procedure to replace a failed host adapter in a running cluster. This procedure defines Node A as the node with the failed host adapter that you are replacing.

Before You Begin

This procedure relies on the following prerequisites and assumptions.

• Except for the failed host adapter, your cluster is operational and all nodes are powered on.

• Your nodes are not configured with dynamic reconfiguration functionality.

  If your nodes are configured for dynamic reconfiguration and you are using two entirely separate hardware paths to your shared data, see the Oracle Solaris Cluster 3.3 Hardware Administration Manual and skip steps that instruct you to shut down the cluster.

  You cannot replace a single, dual-port HBA that has quorum configured on that storage path by using DR. Follow all steps in the procedure. For the details on the risks and limitations of this configuration, see Configuring Cluster Nodes With a Single, Dual-Port HBA in Oracle Solaris Cluster 3.3 Hardware Administration Manual.

  Exceptions to this restriction include three-node or larger cluster configurations where no storage device has a quorum device configured.

This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.

1. Become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.
2. Determine the resource groups and device groups that are running on Node A.

   Record this information because you use this information in Step 10 and Step 11 of this procedure to return resource groups and device groups to Node A.

   # clresourcegroup status -n NodeA 
   # cldevicegroup status -n NodeA

   -n NodeA

   The node for which you are determining resource groups and device groups.

3. Move all resource groups and device groups off Node A.

   # clnode evacuate nodename

4. Shut down Node A.

   For the full procedure about how to shut down and power off a node, see Chapter 3, Shutting Down and Booting a Cluster, in Oracle Solaris Cluster System Administration Guide.

5. Power off Node A.
6. Replace the failed host adapter.

   To remove and add host adapters, see the documentation that shipped with your nodes.

7. If you need to upgrade the node's host adapter firmware, boot Node A into noncluster mode by adding -x to your boot instruction. Proceed to Step 8.

   If you do not need to upgrade firmware, skip to Step 9.

8. Upgrade the host adapter firmware on Node A.

   The Oracle Enterprise Manager Ops Center 2.5 software helps you patch and monitor your data center assets. Oracle Enterprise Manager Ops Center 2.5 helps improve operational efficiency and ensures that you have the latest software patches for your software. Contact your Oracle representative to purchase Oracle Enterprise Manager Ops Center 2.5.

   Additional information for using the Oracle patch management tools is provided in Oracle Solaris Administration Guide: Basic Administration on the Oracle Technology Network. Refer to the version of this manual for the Oracle Solaris OS release that you have installed.

   If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Oracle Solaris Cluster System Administration Guide to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.

   For required firmware, see the Oracle Technology Network.

9. Boot Node A into cluster mode.

   For more information about how to boot nodes, see Chapter 3, Shutting Down and Booting a Cluster, in Oracle Solaris Cluster System Administration Guide.

10. (Optional) Restore the device groups to the original node.

    Do the following for each device group that you want to return to the original node.

    # cldevicegroup switch -n nodename devicegroup1[ devicegroup2 ...]

    -n nodename

    The node to which you are restoring device groups.

    devicegroup1[ devicegroup2 …]

    The device group or groups that you are restoring to the node.

11. (Optional) Restore the resource groups to the original node.

    Do the following for each resource group that you want to return to the original node.

    # clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]

    nodename

    For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

    resourcegroup1[ resourcegroup2 …]

    The resource group or groups that you are returning to the node or nodes.