test

Sun Cluster 3.0-3.1 With Sun StorEdge A3500FC System Manual for Solaris OS

Configuring Storage Systems

This section contains the procedures about how to configure a storage system after you install Sun Cluster software. Table 1–2 lists these procedures.

To configure a storage system before you install Sun Cluster software, use the same procedures you use in a noncluster environment. For the procedures about how to configure a storage system before you install Sun Cluster software, see the Sun StorEdge RAID Manager User’s Guide.

Note –

When you upgrade firmware on a storage device or on an enclosure, redefine the stripe size of a LUN, or perform other LUN operations, a device ID might change unexpectedly. When you perform a check of the device ID configuration by running the scdidadm -c command, the following error message appears on your console if the device ID changed unexpectedly.


device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

To fix device IDs that report this error, run the scdidadm -R command for each affected device.

Table 1–2 Task Map: Configuring a Storage System

Task 

Information 

Create a logical unit number (LUN). 

How to Create a LUN

Remove a LUN. 

How to Delete a LUN

Reset the LUN configuration. 

How to Reset the LUN Configuration

The following is a list of administrative tasks that require no cluster-specific procedures. See the Sun StorEdge RAID Manager User’s Guide and the Sun StorEdge RAID Manager Release Notes for the following procedures.

ProcedureHow to Create a LUN

Use this procedure to create a logical unit number (LUN) from unassigned disk drives or remaining capacity. See theSun StorEdge RAID Manager Release Notes for the latest information about LUN administration.

This product supports the use of hardware RAID and host-based software RAID. For host-based software RAID, this product supports RAID levels 0+1 and 1+0.

Note –

You must use hardware RAID for Oracle Parallel Server (OPS) data stored on the storage array. Do not place OPS data under volume management control. You must place all non-OPS data that is stored on the storage arrays under volume management control. Use either hardware RAID, host-based software RAID, or both types of RAID to manage your non-OPS data.

Hardware RAID uses the storage array's or storage system's hardware redundancy to ensure that independent hardware failures do not impact data availability. If you mirror across separate storage arrays, host-based software RAID ensures that independent hardware failures do not impact data availability when an entire storage array is offline. Although you can use hardware RAID and host-based software RAID concurrently, you need only one RAID solution to maintain a high degree of data availability.

Note –

When you use host-based software RAID with hardware RAID, the hardware RAID levels you use affect hardware maintenance. If you use hardware RAID level 1, 3, or 5, you can perform most maintenance procedures without volume management disruptions. If you use hardware RAID level 0, some maintenance procedures require additional volume management administration because the availability of the LUNs is impacted.

Caution – Caution –

Do not configure LUNs as quorum devices. The use of LUNs as quorum devices is not supported.

Steps

  1. With all nodes booted and attached to the storage system, create the LUN on one node.

    After the LUN formatting completes, a logical name for the new LUN appears in /dev/rdsk on all nodes. These nodes are attached to the storage system.

    For the procedure about how to create a LUN, see the Sun StorEdge RAID Manager User’s Guide.

    If the following warning message displays, ignore the message. Continue with the next step.


    scsi: WARNING:
/sbus@40,0/SUNW,socal@0,0/sf@1,0/ssd@w200200a0b80740db,4 (ssd0):
corrupt label - wrong magic number
    Note –

    Use the format(1M) command to verify Solaris logical device names.

  2. Copy the /etc/raid/rdac_address file from the node on which you created the LUN to the other node. If you copy this file to the other node, you ensure consistency across both nodes.

  3. Ensure that the new logical name for the LUN that you created appears in the /dev/rdsk directory on both nodes.


    # /etc/raid/bin/hot_add

  4. On one node, update the global device namespace.


    # scgdevs

  5. Ensure that the device ID numbers for the LUNs are the same on both nodes. In the sample output that follows, the device ID numbers are different.


    # scdidadm -L
... 
33       e07a:/dev/rdsk/c1t4d2          /dev/did/rdsk/d33
33       e07c:/dev/rdsk/c0t4d2          /dev/did/rdsk/d34

  6. Are the device ID numbers that you received from running the scdidadm command in Step 5 the same for both nodes?

  7. If you want a volume manager to manage the new LUN, incorporate the new LUN into a diskset or disk group.

    For more information, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

ProcedureHow to Delete a LUN

Use this procedure to delete one or more LUNs. See the Sun StorEdge RAID Manager Release Notes for the latest information about LUN administration.

Caution – Caution –

This procedure removes all data on the LUN that you delete.

Caution – Caution –

Do not delete LUN 0.

Steps

  1. From one node that is connected to the storage system, determine the paths to the LUN that you are deleting.


    # format
AVAILABLE DISK SELECTIONS:
0. c0t5d0 <SYMBIOS-StorEdgeA3500FCr-0301 cyl3 alt2 hd64 sec64>
/pseudo/rdnexus@0/rdriver@5,0
1. c0t5d1 <SYMBIOS-StorEdgeA3500FCr-0301 cyl2025 alt2 hd64 sec64>
/pseudo/rdnexus@0/rdriver@5,1

  2. Does a volume manager manage the LUN that you are deleting?

    • If no, proceed to Step 3.

    • If yes, remove the LUN from any diskset or disk group. For more information, see your Solstice DiskSuite/Solaris Volume Manageror VERITAS Volume Manager documentation.

      LUNs that were managed by VERITAS Volume Manager must be removed from VERITAS Volume Manager control before you can delete the LUNs. To remove the LUNs, after you delete the LUN from any disk group, use the following commands. 


      # vxdisk offline cNtXdY
# vxdisk rm cNtXdY

  3. From one node, delete the LUN.

    For the procedure about how to delete a LUN, see the Sun StorEdge RAID Manager User’s Guide.

  4. Remove the paths to the LUNs you are deleting.


    # rm /dev/rdsk/cNtXdY*
# rm /dev/dsk/cNtXdY*

# rm /dev/osa/dev/dsk/cNtXdY*
# rm /dev/osa/dev/rdsk/cNtXdY*

  5. Use the lad command to determine the alternate paths to the LUNs you are deleting.

    The RAID Manager software creates two paths to the LUN in the /dev/osa/dev/rdsk directory. Substitute the cNtXdY number from the other controller module in the storage array to determine the alternate path.

    For example, with this configuration.


    # lad
c0t5d0 1T93600714 LUNS: 0 1
c1t4d0 1T93500595 LUNS: 2

    The alternate paths would be:


    /dev/osa/dev/dsk/c1t4d1*
/dev/osa/dev/rdsk/c1t4d1*

  6. Remove the alternate paths to the LUNs you are deleting.


    # rm /dev/osa/dev/dsk/cNtXdY*
# rm /dev/osa/dev/rdsk/cNtXdY*

  7. On both nodes, remove all obsolete device IDs.


    # scdidadm -C

  8. Move all resource groups and device groups off the node.


    # scswitch -S -h from-node

  9. Shut down the node.

    For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

  10. Perform a reconfiguration boot to create the new Solaris device files and links.

  11. Repeat Step 3 through Step 10 on the other node that is attached to the storage system.

ProcedureHow to Reset the LUN Configuration

Use this procedure to completely remove and reset the LUN configuration.

Caution – Caution –

If you reset a LUN configuration, a new device ID number is assigned to LUN 0. This change occurs because the software assigns a new world wide name (WWN) to the new LUN.

Steps

  1. From one node that is connected to the storage array or storage system, determine the paths to the LUNs you are resetting.


    # format

    For example:


    phys-schost-1# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t5d0 <SYMBIOS-StorEdgeA3500FCr-0301 cyl3 alt2 hd64 sec64>
/pseudo/rdnexus@0/rdriver@5,0
1. c0t5d1 <SYMBIOS-StorEdgeA3500FCr-0301 cyl2025 alt2 hd64 sec64>
/pseudo/rdnexus@0/rdriver@5,1

  2. Does a volume manager manage the LUNs on the controller module you are resetting?

    • If no, proceed to Step 3.

    • If yes, remove the LUN from any diskset or disk group. For more information, see your Solstice DiskSuite/Solaris Volume Manager or VERITAS Volume Manager documentation.

      You must completely remove LUNs that were managed by VERITAS Volume Manager from VERITAS Volume Manager control before you can delete the LUNs.


      # vxdisk offline cNtXdY
# vxdisk rm cNtXdY

  3. On one node, reset the LUN configuration.

    For the procedure about how to reset the LUN configuration, see the Sun StorEdge RAID Manager User’s Guide.

  4. Use the format command to label the new LUN 0.

  5. Remove the paths to the old LUNs you reset.


    # rm /dev/rdsk/cNtXdY*
# rm /dev/dsk/cNtXdY*

# rm /dev/osa/dev/dsk/cNtXdY*
# rm /dev/osa/dev/rdsk/cNtXdY*

  6. On both nodes, update device namespaces.


    # devfsadm -C

  7. On both nodes, remove all obsolete device IDs.


    # scdidadm -C

  8. Move all resource groups and device groups off the node.


    # scswitch -S -h from-node

  9. Shut down the node.

    For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

  10. Perform a reconfiguration boot to create the new Solaris device files and links.

    If an error message like the following appears, ignore it. Continue with the next step.

    device id for '/dev/rdsk/c0t5d0' does not match physical disk's id.

  11. After the node reboots and joins the cluster, repeat Step 5 through Step 10 on the other node. This node is attached to the storage array or storage system.

    The device ID number for the original LUN 0 is removed. A new device ID is assigned to LUN 0.

ProcedureHow to Correct Mismatched Device ID Numbers

Use this section to correct mismatched device ID numbers that might appear during the creation of A3500FC LUNs. You correct the mismatch by deleting Solaris and Sun Cluster paths to the LUNs that have DID numbers that are different. After rebooting, the paths are corrected.

Note –

Use this procedure only if you are directed to do so from How to Create a LUN.

Steps

  1. From one node that is connected to the storage system, determine the paths to the LUNs. These LUNs have different device ID numbers.


    # format

  2. Remove the paths to the LUNs that have different device ID numbers.


    # rm /dev/rdsk/cNtXdY*
# rm /dev/dsk/cNtXdY*

# rm /dev/osa/dev/dsk/cNtXdY*
# rm /dev/osa/dev/rdsk/cNtXdY*

  3. Use the lad command to determine the alternate paths to the LUNs that have different device ID numbers.

    The RAID Manager software creates two paths to the LUN in the /dev/osa/dev/rdsk directory. Substitute the cNtXdY number from the other controller module in the storage array to determine the alternate path.

    For example, with this configuration.


    # lad
c0t5d0 1T93600714 LUNS: 0 1
c1t4d0 1T93500595 LUNS: 2

    The alternate paths would be as follows.


    /dev/osa/dev/dsk/c1t4d1*
/dev/osa/dev/rdsk/c1t4d1*

  4. Remove the alternate paths to the LUNs that have different device ID numbers.


    # rm /dev/osa/dev/dsk/cNtXdY*
# rm /dev/osa/dev/rdsk/cNtXdY*

  5. On both nodes, remove all obsolete device IDs.


    # scdidadm -C

  6. Move all resource groups and device groups off the node.


    # scswitch -S -h from-node

  7. Shut down the node.

    For the procedure about how to shut down and power off a node, see your Sun Cluster system administration documentation.

  8. Perform a reconfiguration boot to create the new Solaris device files and links.

  9. Repeat Step 1 through Step 8 on the other node. This node is attached to the storage system.

  10. Return to How to Create a LUN.