Skip Navigation Links | |
Exit Print View | |
Oracle Solaris Cluster 3.3 With Fibre Channel JBOD Storage Device Manual SPARC Platform Edition |
1. Installing and Maintaining a Fibre Channel JBOD Storage Device
How to Install a Storage Array in a New Cluster
How to Add the First Storage Array to an Existing Cluster
How to Add a Subsequent Storage Array to an Existing Cluster
The maintenance procedures in FRUs That Do Not Require Oracle Solaris Cluster Maintenance Procedures are performed the same as in a noncluster environment. Table 1-2 lists the procedures that require cluster-specific steps.
Table 1-2 Task Map: Maintaining a Storage Array
|
Each storage device has a different set of FRUs that do not require cluster-specific procedures.
Choose among the following storage devices:
The following is a list of administrative tasks that require no cluster-specific procedures. See the Sun StorEdge A5000 Installation and Service Manual for the following procedures.
Before You Begin
This procedure relies on the following prerequisites and assumptions.
Your cluster is operational.
You want to retain the existing disk drives in the storage array.
If you want to replace your disk drives, see How to Replace a Disk Drive.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.
To perform this procedure, become superuser or assume a role that provides solaris.cluster.modify RBAC authorization.
Example 1-1 shows you how to apply this procedure.
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
# luxadm remove_device -F boxname
See Example 1-1 for an example of this command and its use.
For more information, see your storage documentation. For a list of storage documentation, see Related Documentation.
Repeat this step for each node that is connected to the storage array.
# luxadm insert_device
See Example 1-1 for an example of this command and its use.
If a volume management daemon such as vold is running on your node, and you have a CD-ROM drive that is connected to the node, a device busy error might be returned even if no disk is in the drive. This error is an expected behavior.
Use the following command:
# cldevice populate
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
Example 1-1 Replacing a Sun StorEdge A5x00 Storage Array When Using Oracle Solaris Cluster 3.3 Software
The following example shows how to replace a Sun StorEdge A5x00 storage array when using Oracle Solaris Cluster 3.3 software. The storage array to be replaced is venus1.
# luxadm remove_device -F venus1 WARNING!!! Please ensure that no filesystems are mounted on these device(s). All data on these devices should have been backed up. The list of devices that will be removed is: 1: Box name: venus1 Node WWN: 123456789abcdeff Device Type: SENA (SES device) SES Paths: /devices/nodes@1/sbus@1f,0/SUNW,socal@1,0/sf@0,0/ \ ses@w123456789abcdf03,0:0 /devices/nodes@1/sbus@1f,0/SUNW,socal@1,0/sf@1,0/ \ ses@w123456789abcdf00,0:0 Please verify the above list of devices and then enter 'c' or <CR> to Continue or 'q' to Quit. [Default: c]: <Return> Hit <Return> after removing the device(s). <Return> # luxadm insert_device Please hit <RETURN> when you have finished adding Fibre Channel Enclosure(s)/Device(s): <Return> # cldevice populate
Use this procedure to remove a storage array from a cluster. Example 1-2 shows you how to apply this procedure. Use the procedures in your server hardware manual to identify the storage array.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.
To perform this procedure, become superuser or assume a role that provides solaris.cluster.modify RBAC authorization.
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
# luxadm remove_device -F boxname
For more information, see your storage documentation. For a list of storage documentation, see Related Documentation.
Note - If you are using your storage arrays in a SAN-configured cluster, you must keep two FC switches configured in parallel. This configuration maintains cluster availability. See SAN Solutions in an Oracle Solaris Cluster Environment in Oracle Solaris Cluster 3.3 Hardware Administration Manual for more information.
# devfsadm -C # cldevice populate
For the procedure about how to remove host adapters, see the documentation that shipped with your nodes.
Example 1-2 Removing a Sun StorEdge A5x00 Storage Array When Using Oracle Solaris Cluster 3.3 Software
The following example shows how to remove a Sun StorEdge A5x00 storage array from a cluster running Oracle Solaris Cluster version 3.3 software. The storage array to be removed is venus1.
# luxadm remove_device -F venus1 WARNING!!! Please ensure that no file systems are mounted on these device(s). All data on these devices should have been backed up. The list of devices that will be removed is: 1: Storage Array: venus1 Node WWN: 123456789abcdeff Device Type: SENA (SES device) SES Paths: /devices/nodes@1/sbus@1f,0/SUNW,socal@1,0/sf@0,0/ \ ses@w123456789abcdf03,0:0 /devices/nodes@1/sbus@1f,0/SUNW,socal@1,0/sf@1,0/ \ ses@w123456789abcdf00,0:0 Please verify the above list of devices and then enter 'c' or <CR> to Continue or 'q' to Quit. [Default: c]: <Return> Hit <Return> after removing the device(s). <Return> # devfsadm -C # cldevice populate
For conceptual information about quorums, quorum devices, global devices, and device IDs, see your Oracle Solaris Cluster concepts documentation. For a list of Oracle Solaris Cluster documentation, see Related Documentation.
Before You Begin
This procedure assumes that your cluster is operational.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.
To perform this procedure, become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.
Install the new disk drive. Press the Return key when prompted. You can insert multiple disk drives at the same time.
# luxadm insert_device enclosure,slot
The amount of time that the devfsadm command requires to complete its processing depends on the number of devices that are connected to the node. Expect at least five minutes.
# devfsadm -C
# ls -l /dev/rdsk
You can use either the format(1M) command or copy the partitioning from another disk drive in the storage array.
If a volume management daemon such as vold is running on your node, and you have a CD-ROM drive connected to the node, a device busy error might be returned even if no disk is in the drive. This error is an expected behavior.
Run the following command:
# cldevice populate
Note - The DID that was assigned to the new disk drive might not be in sequential order in the storage array.
# cldevice list -v
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
For conceptual information about quorum, quorum devices, global devices, and device IDs, see your Oracle Solaris Cluster concepts documentation. For a list of Oracle Solaris Cluster documentation, see Related Documentation.
Example 1-3 shows you how to apply this procedure.
Before You Begin
This procedure assumes that your cluster is operational.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.
To perform this procedure, become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.
# clquorum show
For procedures about how to add and remove quorum devices, see Oracle Solaris Cluster system administration documentation.
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
If the disk error message reports the drive problem by DID, determine the Oracle Solaris device name.
# cldevice list -v
Remove the disk drive. Press the Return key when prompted.
# luxadm remove_device -F /dev/rdsk/cNtXdYsZ
# devfsadm -C # cldevice clear
Example 1-3 Removing a Disk Drive in a Sun StorEdge A5x00 Storage Array When Using Oracle Solaris Cluster 3.3 Software
The following example shows how to remove a disk drive from a Sun StorEdge A5x00 storage array in a cluster running Oracle Solaris Cluster version 3.3 software. The disk drive to be removed is d4 and is a virtual table of contents (VTOC) labelled device.
# cldevice list -v === DID Device Instances === DID Device Name: /dev/did/rdsk/d4 Full Device Path: phys0-schost1:/dev/rdsk/c1t1d0 Full Device Path: phys-schost2:/dev/rdsk/c1t1d0 Replication: none default_fencing: global # luxadm remove_device -F /dev/rdsk/c1t32d0s2 WARNING!!! Please ensure that no file systems are mounted on these device(s). All data on these devices should have been backed up. The list of devices that will be removed is: 1: Box Name "venus1" front slot 0 Please enter 'q' to Quit or <Return> to Continue: <Return> stopping: Drive in "venus1" front slot 0....Done offlining: Drive in "venus1" front slot 0....Done Hit <Return> after removing the device(s). <Return> Drive in Box Name "venus1" front slot 0 Logical Nodes being removed under /dev/dsk/ and /dev/rdsk: c1t32d0s0 c1t32d0s1 c1t32d0s2 c1t32d0s3 c1t32d0s4 c1t32d0s5 c1t32d0s6 c1t32d0s7 # devfsadm -C # cldevice clear
For conceptual information about quorum, quorum devices, global devices, and device IDs, see your Oracle Solaris Cluster concepts documentation.
Before You Begin
This procedure assumes that your cluster is operational.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.
To perform this procedure, become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.
If the disk error message reports the drive problem by device ID (DID), determine the Oracle Solaris logical device name. If the disk error message reports the drive problem by the Oracle Solaris physical device name, use your Oracle Solaris documentation to map the Oracle Solaris physical device name to the Oracle Solaris logical device name. Use this Oracle Solaris logical device name and DID throughout this procedure.
Run the following command:
# cldevice list -v
# clquorum show
For procedures about how to add and remove quorum devices, see Oracle Solaris Cluster system administration documentation.
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
Use this physical DID in Step 14 to verify that the failed disk drive has been replaced with a new disk drive. The DID and the world wide name (WWN) for the disk drive are the same.
Use the following command:
# cldevice list -v
# prtvtoc /dev/rdsk/cNtXdYs2 > filename
Note - You can also use the format utility to save the disk's partition information.
# luxadm remove_device -F /dev/rdsk/cNtXdYs2
After running the command, warning messages might display. These messages can be ignored.
# luxadm insert_device boxname,fslotnumber
or
# luxadm insert_device boxname,fslotnumber
If you are inserting a front disk drive, use the fslotnumber parameter. If you are inserting a rear disk drive, use the rslotnumber parameter.
The amount of time that the devfsadm command requires to complete depends on the number of devices that are connected to the node. Expect at least five minutes.
# devfsadm -C
# fmthard -s filename /dev/rdsk/cNtXdYs2
Note - You can also use the format utility to partition the new disk drive.
# cldevice repair
If the following physical DID is different from the physical DID in Step 5, you successfully replaced the failed disk drive with a new disk drive.
Use the following command:
# cldevice list -v
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.
For the procedure about how to add a quorum device, see Oracle Solaris Cluster system administration documentation.