Maintaining Sun StorEdge T3 or T3+ Storage Array Components

This section contains the procedures about how to maintain a storage array. The following table lists these procedures. This section does not include a procedure about how to add a disk drive and a procedure about how to remove a disk drive, because a storage array only operates when fully configured.

If you remove any field-replaceable unit (FRU) for an extended period of time, thermal complications might result. To prevent this complication, the storage array is designed so that an orderly shutdown occurs. This shutdown occurs when you remove a component for longer than 30 minutes. Therefore, a replacement part must be immediately available before you start an FRU replacement procedure. You must replace an FRU within 30 minutes. If you do not replace the FRU within this time, the storage array, and all attached storage array, shut down and power off.

When you upgrade firmware on a storage device or on an enclosure, redefine the stripe size of a LUN, or perform other LUN operations, a device ID might change unexpectedly. When you perform a check of the device ID configuration by running the cldevice check or scdidadm -c command, the following error message appears on your console if the device ID changed unexpectedly.

device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

To fix device IDs that report this error, run the cldevice repair or scdidadm -R command for each affected device.

Sun StorEdge T3 and T3+ Array FRUs

The following is a list of administrative tasks that require no cluster-specific procedures. See the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual for the following procedures.

• Replacing an Ethernet cable.

• Replacing a power cable on the storage array.

• Replacing a power and cooling unit (PCU).

• Replacing a unit interconnect card (UIC).

• Adding a disk drive.

• Removing a disk drive.

Upgrading a Sun StorEdge T3 array controller to a Sun StorEdge T3+ array controller requires no cluster-specific procedures. See the Sun StorEdge T3 Array Controller Upgrade Manual for the following procedures.

How to Replace a Disk Drive

Use this procedure to replace a failed disk drive in a storage array in a running cluster.

Sun storage documentation uses the following terms:

• Logical volume

• Logical device

• Logical unit number (LUN)

This manual uses logical volume to refer to all such logical constructs.

Before You Begin

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands, in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

1. Become superuser or assume a role that provides solaris.cluster.read RBAC authorization.

2. If the failed disk drive affect the storage array logical volume's availability, If yes, use volume manager commands to detach the submirror or plex.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

3. If the logical volume (in Step 1) is configured as a quorum device, choose another volume to configure as the quorum device. Then remove the old quorum device.

   To determine whether the LUN is configured as a quorum device, use one of the following commands.

   • If you are using Sun Cluster 3.2, use the following command:

     # clquorum show

   • If you are using Sun Cluster 3.1, use the following command:

     # scstat -q

   For procedures about how to add and remove quorum devices, see your Sun Cluster system administration documentation.

4. Replace the failed disk drive.

   For instructions, refer to the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

5. (Optional) If the new disk drive is part of a logical volume that you want to be a quorum device, add the quorum device.

   To add a quorum device, see your Sun Cluster system administration documentation.

6. If you detached a submirror or plex in Step 1, use volume manager commands to reattach the submirror or plex.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

Replacing a Node-to-Switch Component

Use this procedure to replace a node-to-switch component that has failed or that you suspect might be contributing to a problem.

Node-to-switch components that are covered by this procedure include the following components:

• Node-to-switch fiber-optic cables

• Gigabit interface converters (GBICs) or small form-factor pluggables (SFPs) on an FC switch

• FC switches

To replace a host adapter, see How to Replace a Host Adapter.

This procedure defines Node A as the node that is connected to the node-to-switch component that you are replacing. This procedure assumes that, except for the component you are replacing, your cluster is operational.

Ensure that you are following the appropriate instructions:

• If your cluster uses multipathing, see How to Replace a Node-to-Switch Component in a Cluster That Uses Multipathing.

• If your cluster does not use multipathing, see How to Replace a Node-to-Switch Component in a Cluster Without Multipathing.

How to Replace a Node-to-Switch Component in a Cluster That Uses Multipathing

1. If your configuration is active-passive, and if the active path is the path that needs a component replaced, make that path passive.

2. Replace the component.

   Refer to your hardware documentation for any component-specific instructions.

3. (Optional) If your configuration is active-passive and you changed your configuration in Step 1, switch your original data path back to active.

How to Replace a Node-to-Switch Component in a Cluster Without Multipathing

Before You Begin

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands, in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

1. Become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.

2. If the physical data path has failed, do the following:

   1. Replace the component.

   2. Fix the volume manager error that was caused by the failed data path.

   3. (Optional) If necessary, return resource groups and device groups to this node.

   You have completed this procedure.

3. If the physical data path has not failed, determine the resource groups and device groups that are running on Node A.

   • If you are using Sun Cluster 3.2, use the following commands:

     # clresourcegroup status -n NodeA # cldevicegroup status -n NodeA

     -n NodeA

     The node for which you are determining resource groups and device groups.

   • If you are using Sun Cluster 3.1, use the following command:

     # scstat

4. Move all resource groups and device groups to another node.

   • If you are using Sun Cluster 3.2, use the following command:

     # clnode evacuate nodename

   • If you are using Sun Cluster 3.1, use the following command:

     # scswitch -S -h nodename

5. Replace the node-to-switch component.

   Refer to your hardware documentation for any component-specific instructions.

6. (Optional) Restore the device groups to the original node.

   Do the following for each device group that you want to return to the original node.

   • If you are using Sun Cluster 3.2, use the following command:

     # cldevicegroup switch -n nodename devicegroup1[ devicegroup2 ...]

     -n nodename

     The node to which you are restoring device groups.

     devicegroup1[ devicegroup2 …]

     The device group or groups that you are restoring to the node.

   • If you are using Sun Cluster 3.1, use the following command:

     # scswitch -z -D devicegroup -h nodename

7. (Optional) Restore the resource groups to the original node.

   Do the following for each resource group that you want to return to the original node.

   • If you are using Sun Cluster 3.2, use the following command:

     # clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]

     nodename

     For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

     resourcegroup1[ resourcegroup2 …]

     The resource group or groups that you are returning to the node or nodes.

   • If you are using Sun Cluster 3.1, use the following command:

     # scswitch -z -g resourcegroup -h nodename

How to Replace a Hub/Switch or Hub/Switch-to-Array Component in a Single-Controller Configuration

Use this procedure to replace a hub/switch or the following hub/switch-to-array components for an array in a single-controller configuration. To replace these components for an array in a partner-group configuration, see How to Replace an FC Switch or Storage Array-to-Switch Component in a Partner-Group Configuration. You can use storage array in single-controller configuration with FC switches when you create a SAN.

• Fiber-optic cable that connects a hub/switch to a storage array

• FC hub/switch GBIC or an SFP that connects a hub/switch to a storage array

• FC hub/switch

• FC hub/switch power cord

• Media interface adapter (MIA) on a StorEdge T3 array

  This component does not apply to StorEdge T3+ arrays.

1. Detach the submirrors on the storage array. This array is connected to the hub/switch-to-array fiber-optic cable that you are replacing. Detach the submirrors to stop all I/O activity to this storage array.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

2. Replace the hub/switch or hub/switch-to-array component.

   • For the procedure about how to replace a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

   • For the procedure about how to replace an FC hub/switch GBIC or an SFP, an FC hub/switch, or an FC hub/switch power cord, see the documentation that shipped with your FC hub/switch hardware.

   • For the procedure about how to replace an MIA, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

   • If you are replacing FC switches in a SAN, follow the hardware installation and SAN configuration instructions in the documentation that shipped with your switch hardware.

     ---

     Note –

     If you are replacing an FC switch and you intend to save the switch configuration for restoration to the replacement switch, do not connect the cables to the replacement switch until after you recall the Fabric configuration to the replacement switch. For more information about how to save and recall switch configurations, see the documentation that shipped with your switch hardware.

     ---

     ---

     Note –

     Before you replace an FC switch, be sure that the probe_timeout parameter of your data service software is set to more than 90 seconds. If you increase the value of the probe_timeout parameter to more than 90 seconds, you avoid unnecessary resource group restarts. Resource group restarts occur when one of the FC switches is powered off.

     ---

3. Reattach the submirrors to resynchronize the submirrors.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

How to Replace an FC Switch or Storage Array-to-Switch Component in a Partner-Group Configuration

Use this procedure to replace components for a storage array in a partner-group configuration in a running cluster. To replace components for arrays in a single-controller configuration, see How to Replace a Hub/Switch or Hub/Switch-to-Array Component in a Single-Controller Configuration. Use this procedure to replace the following storage array-to-switch components.

• Fiber-optic cable that connects an FC switch to a storage array.

• GBIC or an SFP on an FC switch, connecting to a storage array.

• FC switch.

• Media Interface Adapter (MIA) on a Sun StorEdge T3 storage array. This component does not apply to Sun StorEdge T3+ storage arrays.

• Interconnect cables between two interconnected storage arrays of a partner group.

1. Access the storage array. This storage array is connected to the FC switch or component that you are replacing.

2. View the controller status for the two storage arrays in the partner group.

   For more information, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.

3. If you replacing an FC switch or component that is attached to a storage array controller that is ONLINE, disable the controller.

   If the controller is already DISABLED, skip to Step 5.

   For more information, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.

4. Verify that the controller's state has been changed to DISABLED.

5. Replace the component by using the following references.

   • For the procedure about how to replace a fiber-optic cable between a storage array and an FC switch, see the documentation that shipped with your switch hardware.

   • For the procedure about how to replace a GBIC or an SFP on an FC switch, see the documentation that shipped with your FC switch hardware.

   • For the procedure about how to replace an FC switch, see the documentation that shipped with your switch hardware.

     ---

     Note –

     If you are replacing an FC switch and you intend to save the switch IP configuration for restoration to the replacement switch, wait to connect the cables to the replacement switch. Connect the cables to the replacement switch after you recall the Fabric configuration to the replacement switch. For more information about how to save and recall switch configurations, see the documentation that shipped with your switch hardware.

     ---

     ---

     Note –

     Before you replace an FC switch, be sure that the probe_timeout parameter of your data service software is set to more than 90 seconds. If you increase the value of the probe_timeout parameter to more than 90 seconds, you avoid unnecessary resource group restarts. Resource group restarts occur when one of the FC switches is powered off.

     ---

   • For the procedure about how to replace an MIA, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

   • For the procedure about how to replace interconnect cables, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

6. If necessary, access the storage array. This storage array is the storage array in the partner group that is still online.

7. Enable the storage array that you disabled in Step 3.

   For more information, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.

8. Verify that the controller's state has been changed to ONLINE.

How to Replace a Storage Array Controller

1. Detach the submirrors on the storage array. This array is connected to the controller that you are replacing. Detach the submirrors to stop all I/O activity to this storage array.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

2. Replace the controller.

   For the procedure about how to replace a storage array controller, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

3. Reattach the submirrors to resynchronize the submirrors.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

How to Replace a Chassis

Use this procedure to replace a storage array chassis. This procedure assumes that you are retaining all FRUs other than the chassis and the midplane. To replace the chassis, you must replace both the chassis and the midplane because these components are manufactured as one part.

You must be a Sun service provider to perform this procedure. If you need to replace a chassis, contact your Sun service provider.

1. Detach the submirrors on the storage array. This array is connected to the chassis that you are replacing. Detach the submirrors to stop all I/O activity to this storage array.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

2. Replace the chassis and the midplane.

   For the procedure about how to replace a storage array chassis, see the Sun StorEdge T3 and T3+ Array Field Service Manual.

3. Reattach the submirrors to resynchronize the submirrors.

   ---

   Caution –

   The world wide names (WWNs) change as a result of this procedure. You must reconfigure your volume manager software to recognize the new WWNs.

   ---

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

How to Replace a Host Adapter

Use this procedure to replace a failed host adapter in a running cluster. This procedure defines Node A as the node with the failed host adapter that you are replacing.

Before You Begin

This procedure relies on the following prerequisites and assumptions.

• Except for the failed host adapter, your cluster is operational and all nodes are powered on.

• Your nodes are not configured with dynamic reconfiguration functionality.

  If your nodes are configured for dynamic reconfiguration and you are using two entirely separate hardware paths to your shared data, see the Sun Cluster Hardware Administration Manual for Solaris OS and skip steps that instruct you to shut down the cluster.

  You cannot replace a single, dual-port HBA that has quorum configured on that storage path by using DR. Follow all steps in the procedure. For the details on the risks and limitations of this configuration, see Configuring Cluster Nodes With a Single, Dual-Port HBA in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

  Exceptions to this restriction include three-node or larger cluster configurations where no storage device has a quorum device configured.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands, in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

1. Become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.

2. Determine the resource groups and device groups that are running on Node A.

   Record this information because you use this information in Step 10 and Step 11 of this procedure to return resource groups and device groups to Node A.

   • If you are using Sun Cluster 3.2, use the following commands:

     # clresourcegroup status -n NodeA # cldevicegroup status -n NodeA

     -n NodeA

     The node for which you are determining resource groups and device groups.

   • If you are using Sun Cluster 3.1, use the following command:

     # scstat

3. Move all resource groups and device groups off Node A.

   • If you are using Sun Cluster 3.2, use the following command:

     # clnode evacuate nodename

   • If you are using Sun Cluster 3.1, use the following command:

     # scswitch -S -h nodename

4. Shut down Node A.

   For the full procedure about how to shut down and power off a node, see Chapter 3, Shutting Down and Booting a Cluster, in Sun Cluster System Administration Guide for Solaris OS.

5. Power off Node A.

6. Replace the failed host adapter.

   To remove and add host adapters, see the documentation that shipped with your nodes.

7. If you need to upgrade the node's host adapter firmware, boot Node A into noncluster mode by adding -x to your boot instruction. Proceed to Step 8.

   If you do not need to upgrade firmware, skip to Step 9.

8. Upgrade the host adapter firmware on Node A.

   If you use the Solaris 8, Solaris 9, or Solaris 10 Operating System, Sun Connection Update Manager keeps you informed of the latest versions of patches and features. Using notifications and intelligent needs-based updating, Sun Connection helps improve operational efficiency and ensures that you have the latest software patches for your Sun software.

   You can download the Sun Connection Update Manager product for free by going to http://www.sun.com/download/products.xml?id=4457d96d.

   Additional information for using the Sun patch management tools is provided in Solaris Administration Guide: Basic Administration at http://docs.sun.com. Refer to the version of this manual for the Solaris OS release that you have installed.

   If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Sun Cluster System Administration Guide for Solaris OS to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.

   For a list of patches that affect Sun Cluster, see the Sun Cluster Wiki Patch Klatch.

   For required firmware, see the Sun System Handbook.

9. Boot Node A into cluster mode.

   For more information about how to boot nodes, see Chapter 3, Shutting Down and Booting a Cluster, in Sun Cluster System Administration Guide for Solaris OS.

10. (Optional) Restore the device groups to the original node.

    Do the following for each device group that you want to return to the original node.

    • If you are using Sun Cluster 3.2, use the following command:

      # cldevicegroup switch -n nodename devicegroup1[ devicegroup2 ...]

      -n nodename

      The node to which you are restoring device groups.

      devicegroup1[ devicegroup2 …]

      The device group or groups that you are restoring to the node.

    • If you are using Sun Cluster 3.1, use the following command:

      # scswitch -z -D devicegroup -h nodename

11. (Optional) Restore the resource groups to the original node.

    Do the following for each resource group that you want to return to the original node.

    • If you are using Sun Cluster 3.2, use the following command:

      # clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]

      nodename

      For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

      resourcegroup1[ resourcegroup2 …]

      The resource group or groups that you are returning to the node or nodes.

    • If you are using Sun Cluster 3.1, use the following command:

      # scswitch -z -g resourcegroup -h nodename

How to Remove a Storage Array in a Single-Controller Configuration

Use this procedure to remove a storage array and its submirrors from a running cluster. This procedure provides the flexibility to remove the host adapters from the nodes for the storage array that you are removing. To remove a partner group from the cluster, see How to Remove a Partner Group.

When you upgrade firmware on a storage device or on an enclosure, redefine the stripe size of a LUN, or perform other LUN operations, a device ID might change unexpectedly. When you perform a check of the device ID configuration by running the cldevice check or scdidadm -c command, the following error message appears on your console if the device ID changed unexpectedly.

device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

To fix device IDs that report this error, run the cldevice repair or scdidadm -R command for each affected device.

This procedure defines Node A as the node with which you begin working. Node B is another node in the cluster.

During this procedure, you lose access to the data that resides on the storage array that you are removing.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands, in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

Before You Begin

To perform this procedure, become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify role-based access control (RBAC) authorization.

1. Back up all database tables, data services, and volumes that are associated with the storage array. This storage array is the storage array you are removing.

2. Detach the submirrors from the storage array that you are removing. Detach the submirrors to stop all I/O activity to the storage array.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

3. Remove the references to the LUN(s) from any diskset or disk group.

   For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

4. Determine the resource groups and device groups that are running on Node B.

   Record this information because you use this information in Step 18 and Step 19 of this procedure to return resource groups and device groups to these nodes.

   • If you are using Sun Cluster 3.2, use the following commands:

     # clresourcegroup status + # cldevicegroup status +

   • If you are using Sun Cluster 3.1, use the following command:

     # scstat

5. Shut down Node A.

   For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

6. If the storage array that you are removing is the last storage array that is connected to Node A, disconnect the fiber-optic cable between Node A and the FC hub/switch that is connected to this storage array. Afterward, disconnect the fiber-optic cable between the FC hub/switch and this storage array.

   If this is not the last storage array. skip this step.

   For the procedure about how to remove a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

   ---

   Note –

   If you are use your storage array in a SAN-configured cluster, you must keep two FC switches configured in parallel. This configuration maintains cluster availability. See SAN Solutions in a Sun Cluster Environment in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS for more information.

   ---

7. If you want to remove the host adapter from Node A, power off the node.

   If you do not want to remove host adapters, skip to Step 10.

8. Remove the host adapter from Node A.

   For the procedure about how to remove host adapters, see the documentation that shipped with your nodes.

9. Without enabling the node to boot, power on Node A.

   For more information, see your Sun Cluster system administration documentation.

10. Boot Node A into cluster mode.

    For more information about how to boot nodes, see your Sun Cluster system administration documentation.

11. Shut down Node B.

    For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

12. If the storage array that you are removing is the last storage array that is connected to the FC hub/switch, disconnect the fiber-optic cable that connects this FC hub/switch and Node B.

    For the procedure about how to remove a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

    ---

    Note –

    If you are use your storage array in a SAN-configured cluster, you must keep two FC switches configured in parallel. This configuration maintains cluster availability. See SAN Solutions in a Sun Cluster Environment in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS for more information.

    ---

13. If you want to remove the host adapter from Node B, power off the node.

    If you do not want to remove host adapters, skip to Step 16.

14. Remove the host adapter fromNode B.

    For the procedure about how to remove host adapters, see the documentation that shipped with your nodes.

15. Without enabling the node to boot, power on Node B

    For more information, see your Sun Cluster system administration documentation.

16. Boot Node B into cluster mode.

    For more information about how to boot nodes, see your Sun Cluster system administration documentation.

17. On all nodes, update the /devices and /dev entries.

    • If you are using Sun Cluster 3.2, use the following command:

      # devfsadm -C # cldevice clear

    • If you are using Sun Cluster 3.1, use the following command:

      # devfsadm -C # scdidadm -C

18. (Optional) Restore the device groups to the original node.

    Perform the following step for each device group you want to return to the original node.

    • If you are using Sun Cluster 3.2, use the following command:

      # cldevicegroup switch -n nodenamedevicegroup1[ devicegroup2 …]

      -n nodename

      The node to which you are restoring device groups.

      devicegroup1[ devicegroup2 …]

      The device group or groups that you are restoring to the node.

    • If you are using Sun Cluster 3.1, use the following command:

      # scswitch -z -D devicegroup -h nodename

19. (Optional) Restore the resource groups to the original node.

    Perform the following step for each resource group you want to return to the original node.

    • If you are using Sun Cluster 3.2, use the following command:

      # clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]

      nodename

      For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

      resourcegroup1[ resourcegroup2 …]

      The resource group or groups that you are returning to the node or nodes.

    • If you are using Sun Cluster 3.1, use the following command:

      # scswitch -z -g resourcegroup -h nodename

How to Remove a Partner Group

Use this procedure to permanently remove storage array partner groups and their submirrors from a running cluster. To remove a storage array in single-controller configuration, see How to Remove a Storage Array in a Single-Controller Configuration.

When you upgrade firmware on a storage device or on an enclosure, redefine the stripe size of a LUN, or perform other LUN operations, a device ID might change unexpectedly. When you perform a check of the device ID configuration by running the cldevice check or scdidadm -c command, the following error message appears on your console if the device ID changed unexpectedly.

device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

To fix device IDs that report this error, run the cldevice repair or scdidadm -R command for each affected device.

This procedure defines Node A as the cluster node that you begin working with. Node B is another node in the cluster.

During this procedure, you lose access to the data that resides on each storage array partner group that you are removing.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands, in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

Before You Begin

To perform this procedure, become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.

1. If necessary, back up all database tables, data services, and volumes associated with each partner group that you are removing.

2. If necessary, detach the submirrors from each storage array or partner group that you are removing. Detach the submirrors to stop all I/O activity to the storage array or partner group.

   For more information, see your Solstice DiskSuite or Veritas Volume Manager documentation.

3. Remove references to each LUN. Each LUN belongs to the storage array or partner group that you are removing.

   For more information, see your Solstice DiskSuite or Veritas Volume Manager documentation.

4. Determine the resource groups and device groups that are running on all nodes.

   Record this information because you use this information in Step 19 and Step 20 of this procedure to return resource groups and device groups to these nodes.

   • If you are using Sun Cluster 3.2, use the following commands:

     # clresourcegroup status + # cldevicegroup status +

   • If you are using Sun Cluster 3.1, use the following command:

     # scstat

5. Shut down Node A.

   For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

6. Disconnect from both storage arrays the fiber-optic cables connecting to the FC switches, then the Ethernet cable(s).

7. If any storage array that you are removing is the last storage array connected to an FC switch that is on Node A, disconnect the fiber-optic cable between Node A and the FC switch that was connected to this storage array.

   If no array that you are removing is the last array connected to the node, skip to Step 11.

   For the procedure about how to remove a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

   ---

   Note –

   If you are using your storage arrays in a SAN-configured cluster, you must keep two FC switches configured in parallel. This configuration maintains cluster availability. See SAN Considerations for more information.

   ---

8. If you want to remove the host adapters from Node A, power off the node.

   If you do not want to remove host adapters, skip to Step 11.

9. Remove the host adapters from Node A.

   For the procedure about how to remove host adapters, see the documentation that shipped with your host adapter and nodes.

10. Without enabling the node to boot, power on Node A

    For more information, see your Sun Cluster system administration documentation.

11. Boot Node A into cluster mode.

    For more information on booting nodes, see your Sun Cluster system administration documentation.

12. Shut down Node B.

    For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

13. If any storage array that you are removing is the last storage array connected to an FC switch that is on Node B, disconnect the fiber-optic cable that connects this FC switch to Node B.

    If array that you are removing is the last array connected to the node, , proceed to Step 14.

    For the procedure about how to remove a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

    ---

    Note –

    If you are using your storage arrays in a SAN-configured cluster, you must keep two FC switches configured in parallel. This configuration maintains cluster availability. See SAN Considerations for more information.

    ---

14. If you want to remove the host adapters from Node B, power off the node.

    If you do not want to remove host adapters, skip to Step 17.

15. Remove the host adapters from Node B.

    For the procedure about how to remove host adapters, see the documentation that shipped with your nodes.

16. Without enabling the node to boot, power on Node B

    For more information, see your Sun Cluster system administration documentation.

17. Boot Node B into cluster mode.

    For more information about how to boot nodes, see Chapter 3, Shutting Down and Booting a Cluster, in Sun Cluster System Administration Guide for Solaris OS.

18. On all nodes, update the /devices and /dev entries.

    • If you are using Sun Cluster 3.2, use the following command:

      # devfsadm -C # cldevice clear

    • If you are using Sun Cluster 3.1, use the following command:

      # devfsadm -C # scdidadm -C

19. (Optional) Restore the device groups to the original node.

    Perform the following step for each device group you want to return to the original node.

    • If you are using Sun Cluster 3.2, use the following command:

      # cldevicegroup switch -n nodename devicegroup1[ devicegroup2 …]

      -n nodename

      The node to which you are restoring device groups.

      devicegroup1[ devicegroup2 …]

      The device group or groups that you are restoring to the node.

    • If you are using Sun Cluster 3.1, use the following command:

      # scswitch -z -D devicegroup -h nodename

20. (Optional) Restore the resource groups to the original node.

    Perform the following step for each resource group you want to return to the original node.

    • If you are using Sun Cluster 3.2, use the following command:

      # clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]

      nodename

      For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

      resourcegroup1[ resourcegroup2 …]

      The resource group or groups that you are returning to the node or nodes.

    • If you are using Sun Cluster 3.1, use the following command:

      # scswitch -z -g resourcegroup -h nodename