test

Sun Cluster 3.0 12/01 Hardware Guide

Maintaining a StorEdge A3500/A3500FC System

This section contains the procedures for maintaining a StorEdge A3500/A3500FC system in a Sun Cluster environment. Some maintenance tasks listed in Table 7-2 are performed the same as in a non-cluster environment, so the task's procedures are referenced rather than contained in this section. Table 7-2 lists the procedures for maintaining the StorEdge A3500/A3500FC system.

Table 7-2 Tasks: Maintaining a StorEdge A3500/A3500FC System

Task 

For Instructions, Go To 

A3500/A3500FC system/controller module procedures: 

Add a StorEdge A3500/A3500FC system to a running cluster. 

"How to Add a StorEdge A3500/A3500FC System to a Running Cluster"

Remove a StorEdge A3500/A3500FC system from a running cluster. 

"How to Remove a StorEdge A3500/A3500FC System From a Running Cluster"

Replace a failed StorEdge A3500/A3500FC controller module or restore an offline controller module. 

"How to Replace a Failed Controller or Restore an Offline Controller "

Upgrade StorEdge A3500/A3500FC controller module firmware and NVSRAM file. 

"How to Upgrade Controller Module Firmware in a Running Cluster"

Replace a power cord to a StorEdge A3500/A3500FC controller module. 

 

Shut down the cluster, then follow the same procedure used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge A3500/A3500FC Controller Module Guide for replacement procedures

Replace a StorEdge A3500/A3500FC controller module fan canister. 

Follow the same procedure that is used in a non-cluster environment. 

Sun StorEdge A3500/A3500FC Controller Module Guide

Replace the StorEdge A3500/A3500FC controller module card cage. 

Shut down the cluster, then follow the same procedure that is used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge A3500/A3500FC Controller Module Guide for replacement procedures

Replace the entire StorEdge A3500/A3500FC controller module assembly. 

Shut down the cluster, then follow the same procedure that is used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge A3500/A3500FC Controller Module Guide for replacement procedures

Cable/hub/connector procedures: 

Replace a SCSI cable from the controller module to the disk array. 

Follow the same procedure that is used in a non-cluster environment. 

Note: You might encounter I/O errors when replacing this cable. These errors are temporary and should disappear when the new cable is securely in place. You might have to use your volume management recovery procedure to recover from these I/O errors. 

Sun StorEdge A3500/A3500FC Controller Module Guide

 

Sun StorEdge RAID Manager User's Guide

Sun StorEdge RAID Manager Release Notes

Replace a StorEdge A3500-to-host SCSI cable. 

Follow the same procedure that is used in a non-cluster environment. 

Sun StorEdge A3500/A3500FC Controller Module Guide

 

Sun StorEdge RAID Manager User's Guide

Sun StorEdge RAID Manager Release Notes

Replace a StorEdge A3500FC-to-host (or hub) fiber-optic cable. 

Follow the same procedure that is used in a non-cluster environment. 

Sun StorEdge A3500/A3500FC Controller Module Guide

 

Sun StorEdge RAID Manager User's Guide

Sun StorEdge RAID Manager Release Notes

Replace a StorEdge FC100 hub.  

 

Follow the same procedure that is used in a non-cluster environment. 

Sun StorEdge A3500/A3500FC Controller Module Guide

 

Sun StorEdge FC-100 Hub Installation and Service Manual

Replace a StorEdge Network FC Switch-8 or Switch-16 

 

(Applies to SAN-configured clusters only.) 

See "StorEdge A3500FC Array SAN Considerations" for SAN information.

Replace a StorEdge FC100 hub gigabit interface converter (GBIC) that connects cables to the host or hub.  

 

Follow the same procedure that is used in a non-cluster environment. 

Sun StorEdge A3500/A3500FC Controller Module Guide

Sun StorEdge FC-100 Hub Installation and Service Manual

Replace a GBIC on a node. 

 

Follow the same procedure that is used in a non-cluster environment. 

Sun StorEdge A3500/A3500FC Controller Module Guide

Cabinet/power subassembly procedures: 

Replace the power supply fan canister.  

Follow the same procedure that is used in a non-cluster environment.  

Sun StorEdge A3500/A3500FC Controller Module Guide

Replace a DC-power or battery harness. 

Shut down the cluster, then follow the same procedure that is used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge A3500/A3500FC Controller Module Guide for replacement procedures

Replace the battery unit. 

Shut down the cluster, then follow the same procedure that is used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge A3500/A3500FC Controller Module Guide for replacement procedures

Replace the power supply housing. 

Shut down the cluster, then follow the same procedure that is used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge A3500/A3500FC Controller Module Guide

Replace a power cord to the cabinet power distribution unit. 

 

Shut down the cluster, then follow the same procedure that is used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge Expansion Cabinet Installation and Service Manual for replacement procedures.

 

D1000 Disk Array Procedures: 

Add a disk drive. 

"How to Add a Disk Drive in a Running Cluster"

Replace a disk drive. 

"How to Replace a Failed Disk Drive in a Running Cluster"

Remove a disk drive. 

"How to Remove a Disk Drive From a Running Cluster"

Upgrade disk drive firmware. 

"How to Upgrade Disk Drive Firmware in a Running Cluster"

Replace a power cord to a StorEdge D1000 disk array. 

 

Shut down the cluster, then follow the same procedure that is used in a non-cluster environment. 

Sun Cluster 3.0 12/01 System Administration Guide for procedures on shutting down a cluster

 

Sun StorEdge A1000 and D1000 Installation, Operations, and Service Manual for replacement procedures

Node/host adapter procedures: 

Replace a host adapter in a node. 

 

Follow the procedure for your type of controller module (StorEdge A3500 or StorEdge A3500FC). 

"How to Replace a Host Adapter in a Node (Connected to a StorEdge A3500 System)"

 

 

"How to Replace a Host Adapter in a Node (Connected to a StorEdge A3500FC System)"

 

 

How to Add a StorEdge A3500/A3500FC System to a Running Cluster

Use this procedure to add a StorEdge A3500/A3500FC system to a running cluster.

  1. Install the RAID Manager software.

    For the procedure on installing RAID Manager software, see the Sun StorEdge RAID Manager Installation and Support Guide.

    Note -

    RAID Manager 6.22 or a compatible version is required for clustering with Sun Cluster 3.0.

    Note -

    For the most current list of software, firmware, and patches that are required for the StorEdge A3x00/A3500FC controller module, refer to EarlyNotifier 20029, "A1000/A3x00/A3500FC Software/Firmware Configuration Matrix." This document is available online to Sun service providers and to customers with SunSolve service contracts at the SunSolve site: http://sunsolve.sun.com.

  2. Install any StorEdge A3500/A3500FC system patches.

    For the location of patches and installation instructions, see the Sun Cluster 3.0 12/01 Release Notes.

  3. Set the Rdac parameters in the /etc/osa/rmparams file:


    Rdac_RetryCount=1
Rdac_NoAltOffline=TRUE

  4. Power on the StorEdge A3500/A3500FC system.

    For the procedure on powering on the StorEdge A3500/A3500FC system, see the Sun StorEdge A3500/A3500FC Controller Module Guide.

  5. Depending on which type of system you are adding:

    • If you are adding a StorEdge A3500 system, go to Step 6.

    • If you are adding a StorEdge A3500FC system, set the loop ID of the controller module by installing jumpers to the appropriate pins on the rear of the controller module.

      For diagrams and information about setting FC-AL ID settings, see the Sun StorEdge A3500/A3500FC Controller Module Guide.

  6. Are you installing new host adapters to your nodes for connection to the StorEdge A3500/A3500FC system?

    • If not, go to Step 8.

    • If you are installing new host adapters, shut down and power off the first node.


      # scswitch -S -h nodename
# shutdown -y -g0 -i0

    For the full procedure on shutting down and powering off a node, see the Sun Cluster 3.0 12/01 System Administration Guide.

  7. Install the host adapters in the first node.

    For the procedure on installing host adapters, see the documentation that shipped with your host adapters and nodes.

  8. Cable the StorEdge A3500/A3500FC system to the first node. Depending on which type of system you are adding:

    • If you are adding a StorEdge A3500 system, connect the differential SCSI cable between the node and the controller module as shown in Figure 7-3. Make sure that the entire SCSI bus length to each enclosure is less than 25 m. This measurement includes the cables to both nodes, as well as the bus length internal to each enclosure, node, and host adapter.

    • If you are installing a StorEdge A3500FC system, see Figure 7-4 for a sample StorEdge A3500FC cabling connection. The example shows the first node that is connected to a StorEdge A3500FC controller module.

      For more sample configurations, see the Sun StorEdge A3500/A3500FC Hardware Configuration Guide.

      For the procedure on installing the cables, see the Sun StorEdge A3500/A3500FC Controller Module Guide.

    Note -

    Cabling procedures are different if you are using your StorEdge A3500FC arrays to create a SAN by using two Sun StorEdge Network FC Switch-8 or Switch-16 switches and Sun SAN Version 3.0 release software. (StorEdge A3500 arrays are not supported by the Sun SAN 3.0 release at this time.) See "StorEdge A3500FC Array SAN Considerations" for more information.

    Figure 7-3 Sample StorEdge A3500 Cabling

    Graphic

    Figure 7-4 Sample StorEdge A3500FC Cabling (1st Node Attached)

    Graphic

  9. Did you power off the first node to install a host adapter?

    • If not, go to Step 10.

    • If you did power off the first node, power it and the StorEdge A3500 system on, but do not allow the node to boot. If necessary, halt the system to continue with OpenBoot PROM (OBP) Monitor tasks.

  10. Depending on which type of controller module you are adding, do the following:

    • If you are installing a StorEdge A3500FC controller module, go to Step 15.

    • If you are adding a StorEdge A3500 controller module, find the paths to the SCSI host adapters. 


      {0} ok show-disks
...b) /sbus@6,0/QLGC,isp@2,10000/sd...d) /sbus@2,0/QLGC,isp@2,10000/sd...

      Identify and record the two controllers that are to be connected to the disk arrays, and record these paths. Use this information to change the SCSI addresses of these controllers in the nvramrc script in Step 11. Do not include the sd directories in the device paths.

  11. Edit the nvramrc script to change the scsi-initiator-id for the host adapters on the first node.

    The default SCSI address for host adapters is 7. Reserve SCSI address 7 for one host adapter in the SCSI chain. This procedure refers to the host adapter that has SCSI address 7 as the host adapter on the "second node."

    To avoid conflicts, change the scsi-initiator-id of the remaining host adapter in the SCSI chain to an available SCSI address. This procedure refers to the host adapter that has an available SCSI address as the host adapter on the "first node."

    For a partial list of nvramrc editor and nvedit keystroke commands, see Appendix B, NVRAMRC Editor and NVEDIT Keystroke Commands of this guide. For a full list of commands, see the OpenBoot 3.x Command Reference Manual.

    The following example sets the scsi-initiator-id to 6. The OpenBoot PROM Monitor prints the line numbers (0:, 1:, and so on).

    Note -

    Insert exactly one space after the quotation mark and before scsi-initiator-id. 


    {0} ok nvedit 
0: probe-all
1: cd /sbus@6,0/QLGC,isp@2,10000 
2: 6 " scsi-initiator-id" integer-property 
3: device-end 
4: cd /sbus@2,0/QLGC,isp@2,10000
5: 6 " scsi-initiator-id" integer-property 
6: device-end 
7: install-console 
8: banner <Control C>
{0} ok

  12. Store the changes.

    The changes you make through the nvedit command are recorded on a temporary copy of the nvramrc script. You can continue to edit this copy without risk. After you have completed your edits, save the changes. If you are not sure about the changes, discard them.

    • To store the changes, type:


      {0} ok nvstore
{0} ok

    • To discard the changes, type:


      {0} ok nvquit
{0} ok

  13. Verify the contents of the nvramrc script you created in Step 11, as shown in the following example.

    If the contents of the nvramrc script are incorrect, use the nvedit command to make corrections.


    {0} ok printenv nvramrc
nvramrc =             probe-all
                      cd /sbus@6,0/QLGC,isp@2,10000
                      6 " scsi-initiator-id" integer-property
                      device-end 
                      cd /sbus@2,0/QLGC,isp@2,10000
                      6 " scsi-initiator-id" integer-property
                      device-end 
                      install-console
                      banner
{0} ok

  14. Instruct the OpenBoot PROM Monitor to use the nvramrc script: 


    {0} ok setenv use-nvramrc? true
use-nvramrc? = true
{0} ok

  15. Did you power off the first node to install a host adapter?

    • If not, go to Step 21.

    • If you powered off the first node, boot it now and wait for it to join the cluster.


      {0} ok boot -r

    For more information on booting nodes, see the Sun Cluster 3.0 12/01 System Administration Guide.

  16. Are you installing new host adapters to the second node for connection to the StorEdge A3500/A3500FC system?

    • If not, go to Step 21.

    • If you are installing new host adapters, shut down and power off the second node.


      # scswitch -S -h nodename
# shutdown -y -g0 -i0

    For the procedure on shutting down and powering off a node, see the Sun Cluster 3.0 12/01 System Administration Guide.

  17. Install the host adapters in the second node.

    For the procedure on installing host adapters, see the documentation that shipped with your nodes.

  18. Cable the StorEdge A3500/A3500FC system to your node. Depending on which type of controller module you are adding, do the following:

    • If you are adding a StorEdge A3500 controller module, connect the differential SCSI cable between the node and the controller module as shown in Figure 7-3. Make sure that the entire SCSI bus length to each enclosure is less than 25 m. This measurement includes the cables to both nodes, as well as the bus length internal to each enclosure, node, and host adapter.

    • If you are installing a StorEdge A3500FC controller module, see Figure 7-5 for a sample StorEdge A3500FC cabling connection. The example shows two nodes that are connected to a StorEdge A3500FC controller module.

      For more sample configurations, see the Sun StorEdge A3500/A3500FC Hardware Configuration Guide.

      For the procedure on installing the cables, see the Sun StorEdge A3500/A3500FC Controller Module Guide.

    Note -

    Cabling procedures are different if you are using your StorEdge A3500FC arrays to create a SAN by using two Sun StorEdge Network FC Switch-8 or Switch-16 switches and Sun SAN Version 3.0 release software. (StorEdge A3500 arrays are not supported by the Sun SAN 3.0 release at this time.) See "StorEdge A3500FC Array SAN Considerations" for more information.

    Figure 7-5 Sample StorEdge A3500FC Cabling (2nd Node Attached)

    Graphic

  19. Did you power off the second node to install a host adapter?

    • If not, go to Step 21.

    • If you did power off the second node, power it and the StorEdge A3500/A3500FC system on, but do not allow the node to boot. If necessary, halt the system to continue with OpenBoot PROM (OBP) Monitor tasks.

  20. Verify that the second node recognizes the new host adapters and disk drives.

    If the node does not recognize the new hardware, check all hardware connections and repeat installation steps you performed in Step 17.


    {0} ok show-disks
...b) /sbus@6,0/QLGC,isp@2,10000/sd...d) /sbus@2,0/QLGC,isp@2,10000/sd...{0} ok

  21. Depending on which type of controller module you are adding, do the following:

    • If you are installing a StorEdge A3500FC controller module, go to Step 26.

    • If you are adding a StorEdge A3500 controller module, verify that the scsi-initiator-id for the host adapters on the second node is set to 7.

      Use the show-disks command to find the paths to the host adapters that are connected to these enclosures. Select each host adapter's device tree node, and display the node's properties to confirm that the scsi-initiator-id for each host adapter is set to 7.


      {0} ok cd /sbus@6,0/QLGC,isp@2,10000
{0} ok .properties
scsi-initiator-id        00000007 
...

  22. Did you power off the second node to install a host adapter?

    • If not, go to Step 26.

    • If you powered off the second node, boot it now and wait for it to join the cluster.


      {0} ok boot -r

    For more information, see the Sun Cluster 3.0 12/01 System Administration Guide.

  23. Check the StorEdge A3500/A3500FC controller module NVSRAM file revision, and if necessary, install the most recent revision.

    For the NVSRAM file revision number and boot level, see the Sun StorEdge RAID Manager Release Notes. For the procedure on upgrading the NVSRAM file, see the Sun StorEdge RAID Manager User's Guide.

  24. Check the StorEdge A3500/A3500FC controller module firmware revision, and, if necessary, install the most recent firmware revision.

    For the revision number and boot level of the StorEdge A3500/A3500FC controller module firmware, see the Sun StorEdge RAID Manager Release Notes. For the procedure on upgrading the StorEdge A3500/A3500FC controller firmware, see "How to Upgrade Controller Module Firmware in a Running Cluster".

  25. One at a time, boot each node into cluster mode.


    # reboot

  26. On one node, verify that the DIDs have been assigned to the StorEdge A3500/A3500FC LUNs for all nodes that are attached to the StorEdge A3500/A3500FC system:


    # scdidadm -L

  27. Verify that the controller module is set to active/active mode (if it is not, set it to active/active).

    For more information on controller modes, see the Sun StorEdge RAID Manager Installation and Support Guide and the Sun StorEdge RAID Manager User's Guide.

Where to Go From Here

To create a LUN from disk drives that are unassigned, see "How to Create a LUN".

To upgrade StorEdge A3500/A3500FC controller module firmware, see "How to Upgrade Controller Module Firmware in a Running Cluster".

How to Remove a StorEdge A3500/A3500FC System From a Running Cluster

Use this procedure to remove a StorEdge A3500/A3500FC system from a running cluster.

Caution - Caution -

This procedure removes all data that is on the StorEdge A3500/A3500FC system you remove.

  1. Migrate any Oracle Parallel Server (OPS) tables, data services, or volumes off the StorEdge A3500/A3500FC system.

  2. Halt all activity to the StorEdge A3500/A3500FC controller module.

    See the RAID Manager User's Guide and your operating system documentation for instructions.

  3. Does a volume manager manage any of the LUNs on the StorEdge A3500/A3500FC controller module you are removing?

    • If not, go to Step 4.

    • If a volume manager does manage the LUN, run the appropriate Solstice DiskSuite or VERITAS Volume Manager commands to remove the LUN from any diskset or disk group. For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation. See the following paragraph for additional VERITAS Volume Manager commands that are required.

      LUNs that were managed by VERITAS Volume Manager must be completely removed from VERITAS Volume Manager control before you can delete them. To remove the LUNs, after you delete the LUN from any disk group, use the following commands: 


      # vxdisk offline cNtXdY
# vxdisk rm cNtXdY

  4. Disconnect all cables from the StorEdge A3500/A3500FC system and remove the hardware from your cluster.

  5. From one node, delete the LUN.

    For the procedure on deleting a LUN, see the Sun StorEdge RAID Manager User's Guide.

  6. Remove the paths to the LUN(s) you are deleting:


    # rm /dev/rdsk/cNtXdY*
# rm /dev/dsk/cNtXdY*

# rm /dev/osa/dev/dsk/cNtXdY*
# rm /dev/osa/dev/rdsk/cNtXdY*

  7. Use the lad command to determine the alternate paths to the LUN(s) you are deleting.

    The RAID Manager software creates two paths to the LUN in the /dev/osa/dev/rdsk directory. Substitute the cNtXdY number from the other controller module in the disk array to determine the alternate path.

    For example, with this configuration:


    # lad
c0t5d0 1T93600714 LUNS: 0 1
c1t4d0 1T93500595 LUNS: 2

    The alternate paths would be the following. 


    /dev/osa/dev/dsk/c1t4d1*
/dev/osa/dev/rdsk/c1t4d1*

  8. Remove the alternate paths to the LUN(s) you are deleting:


    # rm /dev/osa/dev/dsk/cNtXdY*
# rm /dev/osa/dev/rdsk/cNtXdY*

  9. On all cluster nodes, remove references to the StorEdge A3500/A3500FC system:


    # scdidadm -C

  10. Are you are removing the last StorEdge A3500FC controller module from a hub or FC switch in your cluster?

    • If not, go to Step 11.

    • If you are removing the last StorEdge A3500FC controller module from a particular hub or FC switch, remove the hub or FC switch hardware and cables from your cluster.

    Note -

    If you are using your StorEdge A3500FC arrays in a SAN-configured cluster, you must keep two FC switches configured in parallel to maintain cluster availability. See "StorEdge A3500FC Array SAN Considerations" for more information.

  11. Remove any unused host adapter from nodes that were attached to the StorEdge A3500/A3500FC system:

    1. Shut down and power off the first node from which you are removing a host adapter:


      # scswitch -S -h nodename
# shutdown -y -g0 -i0

      For the procedure on shutting down and powering off a node, see the Sun Cluster 3.0 12/01 System Administration Guide.

    2. Remove the host adapter from the first node.

      See the documentation that came with your node hardware for removal instructions.

    3. Boot the node and wait for it to rejoin the cluster.


      # boot -r

    4. Repeat Step a through Step c for the second node that was attached to the StorEdge A3500/A3500FC system.

  12. Switch the cluster back online.


    # scswitch -Z

  13. Are you removing the last StorEdge A3500/A3500FC system from your cluster?

    • If not, you are finished with this procedure.

    • If you are removing the last StorEdge A3500/A3500FC system from your cluster, remove StorEdge A3500/A3500FC software packages. For the procedure on removing software packages, see the documentation that shipped with your StorEdge A3500/A3500FC system.

How to Replace a Failed Controller or Restore an Offline Controller

Use this procedure to replace a StorEdge A3500/A3500FC controller, or to restore an offline controller.

For conceptual information on SCSI reservations and failure fencing, see the Sun Cluster 3.0 12/01 Concepts.

Note -

Replacement and cabling procedures are different from the following procedure if you are using your StorEdge A3500FC arrays to create a SAN by using a Sun StorEdge Network FC Switch-8 or Switch-16 and Sun SAN Version 3.0 release software. (StorEdge A3500 arrays are not supported by the Sun SAN 3.0 release at this time.) See "StorEdge A3500FC Array SAN Considerations" for more information.

  1. On both nodes, to prevent LUNs from automatic assignment to the controller that is being brought online, set the System_LunReDistribution parameter in the /etc/raid/rmparams file to false.

    Caution - Caution -

    You must set the System_LunReDistribution parameter in the /etc/raid/rmparams file to false so that no LUNs are assigned to the controller being brought online. After you verify in Step 5 that the controller has the correct SCSI reservation state, you can balance LUNs between both controllers.

    For the procedure on modifying the rmparams file, see the Sun StorEdge RAID Manager Installation and Support Guide.

  2. Restart the RAID Manager daemon:


    # /etc/init.d/amdemon stop
# /etc/init.d/amdemon start

  3. Do you have a failed controller?

    • If your controller module is offline, but does not have a failed controller, go to Step 4.

    • If you have a failed controller, replace the failed controller with a new controller, but do not bring the controller online.

      For the procedure on replacing StorEdge A3500/A3500FC controllers, see the Sun StorEdge A3500/A3500FC Controller Module Guide and the Sun StorEdge RAID Manager Installation and Support Guide for additional considerations.

  4. On one node, use the RAID Manager GUI's Recovery application to restore the controller online.

    Note -

    You must use the RAID Manager GUI's Recovery application to bring the controller online. Do not use the Redundant Disk Array Controller Utility (rdacutil) because it ignores the value of the System_LunReDistribution parameter in the /etc/raid/rmparams file.

    For information on the Recovery application, see the Sun StorEdge RAID Manager User's Guide. If you have problems with bringing the controller online, see the Sun StorEdge RAID Manager Installation and Support Guide.

  5. On one node that is connected to the StorEdge A3500/A3500FC system, verify that the controller has the correct SCSI reservation state.

    Run the scdidadm(1M) repair option (-R) on LUN 0 of the controller you want to bring online:


    # scdidadm -R /dev/dsk/cNtXdY

  6. Set the controller to active/active mode and assign LUNs to it.

    For more information on controller modes, see the Sun StorEdge RAID Manager Installation and Support Guide and the Sun StorEdge RAID Manager User's Guide.

  7. Reset the System_LunReDistribution parameter in the /etc/raid/rmparams file to true.

    For the procedure on changing the rmparams file, see the Sun StorEdge RAID Manager Installation and Support Guide.

  8. Restart the RAID Manager daemon:


    # /etc/init.d/amdemon stop
# /etc/init.d/amdemon start

How to Upgrade Controller Module Firmware in a Running Cluster

Use this procedure to upgrade firmware in a StorEdge A3500/A3500FC controller module, in a running cluster. Depending on which firmware you are upgrading, you must use either the online or offline method, as described in the Sun StorEdge RAID Manager User's Guide.

  1. Are you upgrading the NVSRAM firmware file?

    • If you are not upgrading the NVSRAM file, you can use the online method.

      Upgrade the firmware by using the online method, as described in the Sun StorEdge RAID Manager User's Guide. No special steps are required for a cluster environment.

    • If you are upgrading the NVSRAM file, you must use the offline method, using one of the following two procedures:

      -- If the data on your StorEdge A3500/A3500FC controller module is mirrored on another controller module, use the procedure that is described in Step 2.

      -- If the data on your StorEdge A3500/A3500FC controller module is not mirrored on another controller module, use the procedure that is described in Step 3.

  2. Use this step if you are upgrading the NVSRAM and other firmware files on a controller module that has its data mirrored:

    1. Halt all activity to the StorEdge A3500/A3500FC controller module.

      For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.

    2. Update the firmware files by using the offline method, as described in the RAID Manager User's Guide.

    3. Restore all activity to the StorEdge A3500/A3500FC controller module.

      For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.

      This step completes the firmware upgrade.

  3. Use this step if you are upgrading the NVSRAM and other firmware files on a controller module that does not have its data mirrored:

    1. Shut down the entire cluster.


      # scshutdown -y -g0

      For the full procedure on shutting down a cluster, see the Sun Cluster 3.0 12/01 System Administration Guide.

    2. Boot one node that is attached to the controller module into non-cluster mode.


      ok boot -x

      For the full procedure on booting a node into non-cluster mode, see the Sun Cluster 3.0 12/01 System Administration Guide.

    3. Update the firmware files using the offline method, as described in the RAID Manager User's Guide.

    4. Reboot both nodes into cluster mode.


      ok boot

      For the full procedure on booting nodes into the cluster, see the Sun Cluster 3.0 12/01 System Administration Guide.

      This step completes the firmware upgrade.

How to Add a Disk Drive in a Running Cluster

Use this procedure to add a disk drive to a StorEdge A3500/A3500FC controlled disk array that is in a running cluster.

Caution - Caution -

If the disk drive you are adding was previously owned by another controller module, preformat the disk drive to wipe clean the old DacStore information before adding it to this disk array.

  1. Install the new disk drive to the disk array.

    For the procedure on installing a disk drive, see the Sun StorEdge D1000 Storage Guide.

  2. Allow the disk drive to spin up (approximately 30 seconds).

  3. Run Health Check to ensure that the new disk drive is not defective.

    For instructions on running Recovery Guru and Health Check, see the Sun StorEdge RAID Manager User's Guide.

  4. Fail the new drive, then revive the drive to update DacStore on the drive.

    For instructions on failing drives and manual recovery procedures, see the Sun StorEdge RAID Manager User's Guide.

  5. Repeat Step 1 through Step 4 for each disk drive you are adding.

Where to Go From Here

To create LUNs for the new drives, see "How to Create a LUN" for more information.

How to Replace a Failed Disk Drive in a Running Cluster

Use this procedure to replace a failed disk drive in a running cluster.

  1. Does replacing the disk drive affects any LUN's availability?

    • If not, go to Step 2.

    • If the replacement does affect LUN availability, remove the LUN(s) from volume management control. For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.

  2. Replace the disk drive in the disk array.

    For the procedure on replacing a disk drive, see the Sun StorEdge D1000 Storage Guide.

  3. Run Health Check to ensure that the new disk drive is not defective.

    For instructions on running Recovery Guru and Health Check, see the Sun StorEdge RAID Manager User's Guide.

  4. Does the failed drive belong to a drive group?

    • If the drive does not belong to a device group, go to Step 5.

    • If the drive is part of a device group, reconstruction is started automatically. If reconstruction does not start automatically for any reason, then select Reconstruct from the Manual Recovery application. Do not select Revive. When reconstruction is complete, go to Step 6.

  5. Fail the new drive, then revive the drive to update DacStore on the drive.

    For instructions on failing drives and manual recovery procedures, see the Sun StorEdge RAID Manager User's Guide.

  6. If you removed LUNs from volume management control in Step 1, return the LUN(s) to volume management control.

    For more information, see your Solstice DiskSuite or VERITAS Volume Manager documentation.

How to Remove a Disk Drive From a Running Cluster

Use this procedure to remove a disk drive from a running cluster.

  1. Remove the logical unit number (LUN) that is associated with the disk drive you are removing.

    For the procedure on removing a LUN, see "How to Delete a LUN".

  2. Remove the disk drive from the disk array.

    For the procedure on removing a disk drive, see the Sun StorEdge D1000 Storage Guide.

    Caution - Caution -

    After you remove the disk drive, install a dummy drive to maintain proper cooling.

How to Upgrade Disk Drive Firmware in a Running Cluster

Note -

Only qualified service personnel should perform disk drive firmware updates. If you need to upgrade drive firmware, contact your local Sun solution center or Sun service provider.

How to Replace a Host Adapter in a Node (Connected to a StorEdge A3500 System)

Note -

This section describes the procedure for replacing a failed host adapter in a running node that is attached to a StorEdge A3500 SCSI-based system. For the same procedure for a cluster node that is attached to an A3500FC system, see "How to Replace a Host Adapter in a Node (Connected to a StorEdge A3500FC System)".

In the following procedure, node 1's host adapter on SCSI bus A needs replacement but node 2 remains in service.

Note -

Several steps in this procedure require that you halt I/O activity. To halt I/O activity, take the controller module offline by using the RAID Manager GUI's manual recovery procedure in the Sun StorEdge RAID Manager User's Guide.

  1. Without powering off the node, shut down node 1.


    # scswitch -S -h nodename
# shutdown -y -g0 -i0

    For the procedure on shutting down a node, see the Sun Cluster 3.0 12/01 System Administration Guide.

  2. From node 2, halt I/O activity to SCSI bus A.

    See the RAID Manager User's Guide for instructions.

  3. From the controller module end of the SCSI cable, disconnect the SCSI bus A cable that connects the StorEdge A3500 controller module to node 1, then replace this cable with a differential SCSI terminator.

  4. Restart I/O activity on SCSI bus A.

    See the RAID Manager User's Guide for instructions.

  5. Does servicing the failed host adapter affect SCSI bus B?

    • If SCSI bus B is not affected, go to Step 9.

    • If SCSI bus B is affected, continue with Step 6.

  6. From node 2, halt I/O activity to the StorEdge A3500 controller module on SCSI bus B.

    See the RAID Manager User's Guide for instructions.

  7. From the controller module end of the SCSI cable, disconnect the SCSI bus B cable that connects the StorEdge A3500 controller module to node 1 and replace this cable with a differential SCSI terminator.

  8. Restart I/O activity on SCSI bus B.

    See the RAID Manager User's Guide for instructions.

  9. Power off node 1.

  10. Replace node 1's host adapter.

    See the documentation that came with your node hardware for instructions.

  11. Power on node 1, but do not allow it to boot. If necessary, halt the system.

  12. From node 2, halt I/O activity to the StorEdge A3500 controller module on SCSI bus A.

    See the RAID Manager User's Guide for instructions.

  13. Remove the differential SCSI terminator from SCSI bus A, then reinstall the SCSI cable to connect the StorEdge A3500 controller module to node 1.

  14. Restart I/O activity on SCSI bus A.

    See the RAID Manager User's Guide for instructions.

  15. Did you install a differential SCSI terminator to SCSI bus B in Step 7?

    • If not, skip to Step 18.

    • If you did install a SCSI terminator to SCSI bus B, halt I/O activity on SCSI bus B, then continue with Step 16.

  16. Remove the differential SCSI terminator from SCSI bus B, then reinstall the SCSI cable to connect the StorEdge A3500 controller module to node 1.

  17. Restart I/O activity on SCSI bus B.

    See the RAID Manager User's Guide for instructions.

  18. Bring the StorEdge A3500 controller module back online.

    See the RAID Manager User's Guide for instructions.

  19. Rebalance all logical unit numbers (LUNs).

    See the RAID Manager User's Guide for instructions.

  20. Boot node 1 into cluster mode.


    {0} ok boot

How to Replace a Host Adapter in a Node (Connected to a StorEdge A3500FC System)

Note -

This section describes the procedure for replacing a failed host adapter in a node that is attached to a StorEdge A3500FC fiber-optic-based system. For the same procedure for a cluster node that is attached to a StorEdge A3500 system, see "How to Replace a Host Adapter in a Node (Connected to a StorEdge A3500 System)".

Note -

Several steps in this procedure require that you halt I/O activity. To halt I/O activity, take the controller module offline by using the RAID Manager GUI's manual recovery procedure in the Sun StorEdge RAID Manager User's Guide.

  1. Move all Sun Cluster data services off of the node in which you are replacing a host adapter.

    See the Sun Cluster 3.0 12/01 Data Services Installation and Configuration Guide for instructions.

  2. Halt all I/O activity on the affected StorEdge A3500FC controller module.

    See the RAID Manager User's Guide for instructions.

  3. Shut down and power off the node in which you are replacing a host adapter.


    # scswitch -S -h nodename
# shutdown -y -g0 -i0

    For the procedure on shutting down and powering off a node, see the Sun Cluster 3.0 12/01 System Administration Guide.

  4. Disconnect the fiber-optic cable from the host adapter that you are replacing.

  5. Replace the host adapter in the node.

    See the documentation that came with your node hardware for instructions.

  6. Connect the fiber optic cable to the new host adapter that you just installed.

  7. Boot the node into cluster mode.


    {0} ok boot

  8. Restart I/O activity on the affected StorEdge A3500FC controller module.

    See the RAID Manager User's Guide and your operating system documentation for instructions.

  9. Check the status of the affected StorEdge A3500FC controller module.

    Observe the front panel LEDs and interpret them by using the Sun StorEdge A3500/A3500FController Module Guide.

  10. Rebalance LUNs that are running on the affected StorEdge A3500FC controller module.

    For more information, see the Sun StorEdge RAID Manager User's Guide.

  11. Move the Sun Cluster data services back to the node in which you replaced the host adapter.

    See the Sun Cluster 3.0 12/01 Data Services Installation and Configuration Guide for instructions.