test

Sun Cluster 3.0-3.1 With Sun StorEdge 6320 System Manual for Solaris OS

Chapter 1 Installing and Maintaining a Sun StorEdge 6320 System

This chapter contains the procedures about how to install, configure, and maintain a SunTM StorEdgeTM 6320 system. These procedures are specific to a SunTM Cluster environment.

This chapter contains the following main topics:

For detailed information about storage system architecture, features, and configuration utilities, see the Sun StorEdge 6320 System Reference and Service Manual and the Sun StorEdge 6320 System Installation Guide.

Installing Storage Systems

This section contains the procedures listed in Table 1–1

Table 1–1 Task Map: Installing Storage Systems

Task 

Information 

Install storage systems in a new cluster, before the OS and Sun Cluster software is installed. 

How to Install Storage Systems in a New Cluster

Add storage systems to an existing cluster.  

How to Add Storage Systems to an Existing Cluster

You can install your storage system in several different configurations. Evaluate your needs. Determine which configuration is best for your situation. See the Sun StorEdge 6320 System Installation Guide, and Installing Storage Arrays in Sun Cluster 3.0-3.1 With Sun StorEdge 6120 Array Manual for Solaris OS.

ProcedureHow to Install Storage Systems in a New Cluster

Use this procedure to install a storage system before you install the Solaris operating environment and Sun Cluster software on your nodes. To add a storage system to an existing cluster, use the procedure in How to Add Storage Systems to an Existing Cluster.

Steps

  1. If necessary, install host adapters in the nodes to be connected to the storage system.

    For the procedure about how to install host adapters, see the documentation that shipped with your host adapters and nodes.

  2. (StorEdge 6320SL storage system ONLY) Install the Fibre Channel (FC) switch for the storage system if you do not have a switch installed.

    Note –

    In a StorEdge 6320SL storage system, the customer provides the switch.

    For the procedure about how to install an FC switch, see the documentation that shipped with your FC switch hardware.

  3. Unpack, place, and level the storage system.

    For instructions, see the Sun StorEdge 6320 System Installation Guide.

  4. Install the system power cord and the system grounding strap.

    For instructions, see the Sun StorEdge 6320 System Installation Guide.

  5. (StorEdge 6320SL storage system ONLY) Connect the storage arrays to the FC switches by using fiber-optic cables.

    Caution – Caution –

    Do not connect the switch's Ethernet port to the storage system's private LAN.

    For the procedure about how to cable the storage system, see the Sun StorEdge 6320 System Installation Guide.

  6. Power on the storage system and the nodes.

    For instructions about how to power on the storage system, see the Sun StorEdge 6320 System Installation Guide. For instructions about how to power on a node, see the documentation that shipped with your node hardware.

  7. Configure the service processor.

    For more information, see the Sun StorEdge 6320 System Installation Guide.

  8. Create a volume.

    For the procedure about how to create a volume, see the Sun StorEdge 6320 System Reference and Service Manual.

  9. (Optional) Specify initiator groups for the volume.

    For the procedure about how to specify initiator groups, see the Sun StorEdge 6320 System Reference and Service Manual.

  10. If necessary, reconfigure the storage system's FC switches to ensure that all nodes can access each storage array.

    The following configurations might prevent some nodes from accessing each storage array in the cluster.

    • Zone configuration

    • Multiple clusters that use same switch

    • Unconfigured ports or misconfigured ports

  11. On all nodes, install the Solaris operating system and apply the required Solaris patches for Sun Cluster software and storage array support.

    For the procedure about how to install the Solaris operating environment, see How to Install Solaris Software in Sun Cluster Software Installation Guide for Solaris OS.

  12. Install any required patches or software for Sun StorEdge Traffic Manager software support to nodes and enable multipathing.

    For the procedure about how to install the Sun StorEdge Traffic Manager software, see How to Install Sun Multipathing Software in Sun Cluster Software Installation Guide for Solaris OS.

  13. Configure the STMS paths.


    cfgadm -c configure controllerinstance

    For the procedure about how to configure STMS paths, see the Sun StorEdge Traffic Manager Installation and Configuration Guide.

  14. Update the Solaris device files and links.


    # devfsadm
    Note –

    You can wait for the devfsadm daemon to automatically update the Solaris device files and links, or you can run the devfsadm command to immediately update the Solaris device files and links.

  15. Confirm that all storage arrays that you installed are visible to all nodes. 


    # luxadm probe
See Also

To continue with Sun Cluster software installation tasks, see your Sun Cluster software installation documentation.

How to Add Storage Systems to an Existing Cluster

Use this procedure to add a new storage system to a running cluster. To install systems to a new Sun Cluster configuration that is not running, use the procedure in How to Install Storage Systems in a New Cluster.

This procedure defines Node N as the node to be connected to the storage system you are adding and the node with which you begin working.

ProcedureHow to Perform Initial Configuration Tasks on the Storage Array

Steps

  1. (StorEdge 6320SL storage system ONLY) Install the Fibre Channel (FC) switch for the storage system if you do not have a switch installed.

    Note –

    In a StorEdge 6320SL storage system, the customer provides the switch.

    For the procedure about how to install an FC switch, see the documentation that shipped with your FC switch hardware.

  2. Configure the service processor.

    For more information, see the Sun StorEdge 6320 System Installation Guide.

  3. Create a volume.

    For the procedure about how to create a volume, see the Sun StorEdge 6320 System Reference and Service Manual.

  4. (Optional) Specify initiator groups for the volume.

    For the procedure about how to specify initiator groups, see the Sun StorEdge 6320 System Reference and Service Manual.

  5. Unpack, place, and level the storage system.

    For instructions, see the Sun StorEdge 6320 System Installation Guide.

  6. Install the system power cord and the system grounding strap.

    For instructions, see the Sun StorEdge 6320 System Installation Guide.

  7. (StorEdge 6320SL storage system ONLY) Connect the storage arrays to the FC switches by using fiber-optic cables.

    Caution – Caution –

    Do not connect the switch's Ethernet port to the storage system's private LAN.

    For the procedure about how to cable the storage system, see the Sun StorEdge 6320 System Installation Guide.

  8. Power on the new storage system.

    Note –

    The storage arrays in your system might require several minutes to boot.

    For the procedure about how to power on the storage system, see the Sun StorEdge 6320 System Installation Guide.

  9. If necessary, reconfigure the storage system's FC switches to ensure that all nodes can access each storage array.

    The following configurations might prevent some nodes from accessing each storage array in the cluster:

    • Zone configuration

    • Multiple clusters that use the same switch

    • Unconfigured ports or misconfigured ports

ProcedureHow to Connect the Node to the FC Switches

Steps

  1. Determine the resource groups and device groups that are running on all nodes.

    Record this information because you use this information in Step 20 of this procedure to return resource groups and device groups to these nodes.


    # scstat

  2. Move all resource groups and device groups off Node N.


    # scswitch -S -h from-node

  3. Do you need to install one or more host adapters in Node N?

  4. Is the host adapter that you that are installing the first host adapter on Node N?

    • If no, skip to Step 6.

    • If yes, determine whether the required drivers for the host adapter are already installed on this node. For the required packages, see the documentation that shipped with your host adapters.

  5. Are the required support packages already installed?

    • If yes, proceed to Step 6.

    • If no, install the packages.

    The support packages are located in the Product directory of the Solaris CD-ROM.

  6. Shut down and power off Node N.

    For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

  7. Install one or more host adapters in Node N.

    For the procedure about how to install host adapters, see the documentation that shipped with your host adapters and nodes.

  8. Power on and boot Node N into noncluster mode.

    For more information about how to boot nodes, see your Sun Cluster system administration documentation.

  9. If necessary, upgrade the host adapter firmware on Node N.

    PatchPro is a patch-management tool that eases the selection and download of patches required for installation or maintenance of Sun Cluster software. PatchPro provides an Interactive Mode tool especially for Sun Cluster. The Interactive Tool makes the installation of patches easier. PatchPro's Expert Mode tool helps you to maintain your configuration with the latest set of patches. Expert Mode is especially useful for obtaining all of the latest patches, not just the high availability and security patches.

    To access the PatchPro tool for Sun Cluster software, go to http://www.sun.com/PatchPro/, click Sun Cluster, then choose either Interactive Mode or Expert Mode. Follow the instructions in the PatchPro tool to describe your cluster configuration and download the patches.

    For third-party firmware patches, see the SunSolveSM Online site at http://sunsolve.ebay.sun.com.

  10. If necessary, install a GBIC or an SFP in the FC switch or the storage array.

    For the procedure about how to install a GBIC or an SFP, see the documentation that shipped with your FC switch hardware or the Sun StorEdge 6320 System Installation Guide.

  11. Connect a fiber-optic cable between the FC switch and Node N.

    For the procedure about how to install a fiber-optic cable, see the Sun StorEdge 6320 System Installation Guide.

  12. Install the required Solaris patches for storage array support on Node N.

    PatchPro is a patch-management tool that eases the selection and download of patches required for installation or maintenance of Sun Cluster software. PatchPro provides an Interactive Mode tool especially for Sun Cluster. The Interactive Tool makes the installation of patches easier. PatchPro's Expert Mode tool helps you to maintain your configuration with the latest set of patches. Expert Mode is especially useful for obtaining all of the latest patches, not just the high availability and security patches.

    To access the PatchPro tool for Sun Cluster software, go to http://www.sun.com/PatchPro/, click Sun Cluster, then choose either Interactive Mode or Expert Mode. Follow the instructions in the PatchPro tool to describe your cluster configuration and download the patches.

    For third-party firmware patches, see the SunSolveSM Online site at http://sunsolve.ebay.sun.com.

  13. Perform a reconfiguration boot on Node N to create the new Solaris device files and links.


    # boot -r

  14. Are any devices that are associated with the volume you created at an unconfigured state?


    # cfgadm -al | grep disk

    • If no, proceed to Step 15.

    • If yes, configure the STMS paths.


      cfgadm -c configure controllerinstance

      For the procedure about how to configure STMS paths, see the Sun StorEdge Traffic Manager Installation and Configuration Guide.

      Note –

      You need to reboot if the cfgadm command does not configure the unconfigured devices that are associated with the volume you are creating. See the Sun StorEdge Traffic Manager Installation and Configuration Guide for more information.

  15. Update the Solaris device files and links.


    # devfsadm
    Note –

    You can wait for the devfsadm daemon to automatically update the Solaris device files and links, or you can run the devfsadm command to immediately update the Solaris device files and links.

  16. On Node N, update the paths to the DID instances.


    # scgdevs

  17. If necessary, label the new storage array logical volume.

    For the procedure about how to label a logical volume, see the Sun StorEdge 6320 System Reference and Service Manual.

  18. (Optional) On Node N, verify that the device IDs (DIDs) are assigned to the new storage array.


    # scdidadm -l

  19. Repeat Step 2 through Step 18 for each remaining node that you plan to connect to the storage array.

  20. (Optional) Return the resource groups and device groups that you identified in Step 1 to the original nodes.


    # scswitch -z -g resource-group -h nodename
# scswitch -z -D device-group-name -h nodename

    For more information, see your Sun Cluster system administration documentation.

  21. Perform volume management administration to incorporate the new volumes into the cluster.

    For more information, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

Next Steps

The best way to enable multipathing for a cluster is to install the multipathing software and enable multipathing before installing the Sun Cluster software and establishing the cluster. For this procedure, see How to Install Sun Multipathing Software in Sun Cluster Software Installation Guide for Solaris OS. If you need to add multipathing software to an established cluster, see How to Install Sun Multipathing Software in Sun Cluster Software Installation Guide for Solaris OS and follow the troubleshooting steps to clean up the device IDs.

Configuring Storage Systems

This section contains the procedures about how to configure a storage system in a running cluster. Table 1–2 lists these procedures.

Table 1–2 Task Map: Configuring Storage Arrays

Task 

Information 

Create a volume. 

How to Create a Logical Volume

Remove a volume. 

How to Remove a Logical Volume

The following is a list of administrative tasks that require no cluster-specific procedures. See the Sun StorEdge 6320 System Reference and Service Manual for the following procedures.

ProcedureHow to Create a Logical Volume

Use this procedure to create a logical volume from unassigned storage capacity.

Note –

Sun storage documentation uses the following terms:

This manual uses logical volume to refer to all such logical constructs.

Before You Begin

This procedure relies on the following prerequisites and assumptions.

Steps

  1. Follow the instructions in your storage device's documentation to create and map the logical volume. For a URL to this storage documentation, see Related Documentation.

    • Completely set up the logical volume. When you are finished, the volume must be created, mapped, mounted, and initialized.

    • If necessary, partition the volume.

    • To allow multiple clusters and nonclustered nodes to access the storage device, create initiator groups by using LUN masking.

  2. Are you using multipathing?

  3. Are any devices that are associated with the volume you created at an unconfigured state?


    # cfgadm -al | grep disk

    • If no, proceed to Step 4.

    • If yes, configure the Traffic Manager paths on each node that is connected to the storage device.


      cfgadm -o force_update -c configure controllerinstance

      For the procedure about how to configure Traffic Manager paths, see the Sun StorEdge Traffic Manager Installation and Configuration Guide.

  4. On one node that is connected to the storage device, use the format command to label the new logical volume.

  5. From any node in the cluster, update the global device namespace. 


    # scgdevs
    Note –

    You might have a volume management daemon such as vold running on your node, and have a CD-ROM drive connected to the node. Under these conditions, a device busy error might be returned even if no disk is in the drive. This error is expected behavior. You can safely ignore this error message.

  6. To manage this volume with volume management software, use the appropriate Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager commands to update the list of devices on all nodes that are attached to the new volume that you created.

    For more information, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

See Also

ProcedureHow to Remove a Logical Volume

Use this procedure to remove a logical volume. This procedure defines Node A as the node with which you begin working.

Note –

Sun storage documentation uses the following terms:

This manual uses logical volume to refer to all such logical constructs.

Before You Begin

This procedure relies on the following prerequisites and assumptions.

Steps

  1. Identify the logical volume that you are removing.

    Refer to your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation for more information.

  2. (Optional) Migrate all data off the logical volume that you are removing. Alternatively, back up that data.

  3. Check if the logical volume that you are removing is a quorum device.


    # scstat -q

    If yes, choose and configure another device as the quorum device. Then remove the old quorum device.

    For procedures about how to add and remove quorum devices, see Chapter 5, Administering Quorum, in Sun Cluster System Administration Guide for Solaris OS.

  4. If you are using volume management software, use that software to update the list of devices on all nodes that are attached to the logical volume that you are removing.

    For instructions about how to update the list of devices, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

  5. If you are using volume management software, run the appropriate Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager commands to remove the logical volume from any diskset or disk group.

    For more information, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

    Note –

    Volumes that were managed by VERITAS Volume Manager must be completely removed from VERITAS Volume Manager control before you can delete them from the Sun Cluster environment. After you delete the volume from any disk group, use the following commands on both nodes to remove the volume from VERITAS Volume Manager control. 


    # vxdisk offline Accessname
# vxdisk rm Accessname
    Accessname

    Disk access name

  6. If you are using multipathing, unconfigure the volume in Sun StorEdge Traffic Manager. 


    # cfgadm -o force_update -c unconfigure Logical_Volume

  7. Access the storage device and remove the logical volume.

    For the procedure about how to remove the volume, see your storage documentation. For a list of storage documentation, see Related Documentation.

  8. Determine the resource groups and device groups that are running on all nodes.

    Record this information because you use it in Step 13 of this procedure to return resource groups and device groups to these nodes. 


    # scstat

  9. Move all resource groups and device groups off Node A. 


    # scswitch -s -h from-node

  10. Shut down and reboot Node A.

    For the procedure about how to shut down and power off a node, see Chapter 3, Shutting Down and Booting a Cluster, in Sun Cluster System Administration Guide for Solaris OS.

  11. On Node A, remove the paths to the logical volume that you removed. Remove obsolete device IDs. 


    # devfsadm -C
# scdidadm -C

  12. For each additional node that is connected to the shared storage that hosted the logical volume, repeat Step 8 to Step 11.

  13. (Optional) Return the resource groups and device groups that you identified in Step 8 to all cluster nodes.

Maintaining Storage Systems

This section contains the procedures for maintaining a storage system in a running cluster. Table 1–3 lists these procedures.

Table 1–3 Task Map: Maintaining Storage Systems

Task 

Information 

Remove a storage system. 

How to Remove a Storage System

Upgrade storage array firmware. 

How to Upgrade Storage Array Firmware

Replace a node-to-switch component. 

  • Node-to-switch fiber-optic cable

  • FC host adapter

  • FC switch

  • GBIC or SFP

How to Replace a Node-to-Switch Component in a Cluster Without Multipathing

Replace a node's host adapter. 

How to Replace a Host Adapter

Add a node to the storage array.

Sun Cluster system administration documentation 

Remove a node from the storage array.

Sun Cluster system administration documentation 

StorEdge 6320 System FRUs

The following is a list of administrative tasks that require no cluster-specific procedures. See the Sun StorEdge 6320 System Reference and Service Manual for the following procedures.

ProcedureHow to Upgrade Storage Array Firmware

Use this procedure to upgrade storage array firmware in a running cluster. Storage array firmware includes controller firmware, unit interconnect card (UIC) firmware, EPROM firmware, and disk drive firmware.

Note –

When you upgrade firmware on a storage device or on an enclosure, redefine the stripe size of a LUN, or perform other LUN operations, a device ID might change unexpectedly. When you perform a check of the device ID configuration by running the scdidadm -c command, the following error message appears on your console if the device ID changed unexpectedly.


device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

To fix device IDs that report this error, run the scdidadm -R command for each affected device.

Steps

  1. Stop all I/O to the storage arrays you are upgrading.

  2. Apply the controller, disk drive, and loop-card firmware patches.

    For the list of required patches, see the Sun StorEdge 6320 System Reference and Service Manual. For the procedure about how to apply firmware patches, see the firmware patch README file. For the procedure about how to verify the firmware level, see the Sun StorEdge 6320 System Reference and Service Manual.

  3. Confirm that all storage arrays that you upgraded are visible to all nodes. 


    # luxadm probe

  4. Restart all I/O to the storage arrays.

    You stopped I/O to these storage arrays in Step 1.

ProcedureHow to Remove a Storage System

Use this procedure to permanently remove a storage system from a running cluster.

This procedure defines Node N as the node that is connected to the storage system you are removing and the node with which you begin working.

Caution – Caution –

During this procedure, you lose access to the data that resides on the storage system that you are removing.

Steps

  1. If necessary, back up all database tables, data services, and volumes that are associated with each partner group that is affected.

  2. Remove references to the volumes that reside on the storage system that you are removing.

    For more information, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

  3. Disconnect the cables that connected Node N to the FC switches in your storage system.

  4. On all nodes, remove the obsolete Solaris links and device IDs (DIDs).


    # devfsadm -C
# scdidadm -C

  5. Repeat Step 3 through Step 4 for each node that is connected to the storage system.

Replacing a Node-to-Switch Component

Use this procedure to replace a node-to-switch component that has failed or that you suspect might be contributing to a problem.

Note –

Node-to-switch components that are covered by this procedure include the following components:

For the procedure about how to replace a host adapter, see How to Replace a Host Adapter.

This procedure defines Node A as the node that is connected to the node-to-switch component that you are replacing. This procedure assumes that, except for the component you are replacing, your cluster is operational.

Ensure that you are following the appropriate instructions:

ProcedureHow to Replace a Node-to-Switch Component in a Cluster That Uses Multipathing

Steps

  1. Is your configuration active-passive?

    If yes, and the active path is the path that needs a component replaced, make that path passive.

  2. Replace the component.

    Refer to your hardware documentation for any component-specific instructions.

  3. (Optional) If your configuration is active-passive and you changed your configuration in Step 1, switch your original data path back to active.

ProcedureHow to Replace a Node-to-Switch Component in a Cluster Without Multipathing

Steps

  1. Check if the physical data path failed.

    If no, proceed to Step 2.

    If yes:

    1. Replace the component.

      Refer to your hardware documentation for any component-specific instructions.

    2. Fix the volume manager error that was caused by the failed data path.

    3. (Optional) If necessary, return resource groups and device groups to this node.

    You have completed this procedure.

  2. Determine the resource groups and device groups that are running on Node A. 


    # scstat

  3. Move all resource groups and device groups to another node. 


    # scswitch -s -h from-node

  4. Replace the node-to-switch component.

    Refer to your hardware documentation for any component-specific instructions.

  5. (Optional) If necessary, return the resource groups and device groups that you identified in Step 2 to Node A. 


    # scswitch -z -g resource-group -h nodename
# swswitch -z -D device-group -h nodename

ProcedureHow to Replace a Host Adapter

Use this procedure to replace a failed host adapter in a running cluster. This procedure defines Node A as the node with the failed host adapter that you are replacing.

Before You Begin

This procedure relies on the following prerequisites and assumptions.

Steps

  1. Determine the resource groups and device groups that are running on Node A.

    Record this information because you use this information in Step 9 of this procedure to return resource groups and device groups to Node A.


    # scstat

  2. Move all resource groups and device groups off Node A.


    # scswitch -S -h nodename

  3. Shut down Node A.

    For the full procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

  4. Power off Node A.

  5. Replace the failed host adapter.

    For the procedure about how to remove and add host adapters, see the documentation that shipped with your nodes.

  6. Do you need to upgrade the node's host adapter firmware?

    • If yes, boot Node A into noncluster mode. Proceed to Step 7.

      For more information about how to boot nodes, see your Sun Cluster system administration documentation.

    • If no, proceed to Step 8.

  7. Upgrade the host adapter firmware on Node A.

    PatchPro is a patch-management tool that eases the selection and download of patches required for installation or maintenance of Sun Cluster software. PatchPro provides an Interactive Mode tool especially for Sun Cluster. The Interactive Tool makes the installation of patches easier. PatchPro's Expert Mode tool helps you to maintain your configuration with the latest set of patches. Expert Mode is especially useful for obtaining all of the latest patches, not just the high availability and security patches.

    To access the PatchPro tool for Sun Cluster software, go to http://www.sun.com/PatchPro/, click Sun Cluster, then choose either Interactive Mode or Expert Mode. Follow the instructions in the PatchPro tool to describe your cluster configuration and download the patches.

    For third-party firmware patches, see the SunSolveSM Online site at http://sunsolve.ebay.sun.com.

  8. Boot Node A into cluster mode.

    For more information about how to boot nodes, see your Sun Cluster system administration documentation.

  9. Return the resource groups and device groups you identified in Step 1 to Node A.


    # scswitch -z -g resource-group -h nodename
# scswitch -z -D device-group-name -h nodename

    For more information, see your Sun Cluster system administration documentation.