Installing Sun StorEdge T3 and T3+ Arrays

This section contains the procedures about how to install storage arrays in a cluster. The following table lists these procedures.

Table 1-1 Task Map: Installing a Storage Array

Task	Information
Installing arrays in a new cluster, using a single-controller configuration	How to Install a Storage Array in a New Cluster Using a Single-Controller Configuration
Installing arrays in a new cluster, using a partner-group configuration	How to Install a Storage Array in a New Cluster, Using a Partner-Group Configuration
Adding arrays to an existing cluster, using a single-controller configuration.	How to Add a Storage Array to an Existing Cluster, Using a Single-Controller Configuration
Adding arrays to an existing cluster, using a partner-group configuration.	How to Add a Storage Array to an Existing Cluster, Using a Partner-Group Configuration
Upgrading a T3 storage array to a T3+ array.	How to Upgrade a StorEdge T3 Controller to a StorEdge T3+ Controller
Migrating a single-controller array configuration to a partner-group configuration.	How to Migrate From a Single-Controller Configuration to a Partner-Group Configuration

How to Install a Storage Array in a New Cluster Using a Single-Controller Configuration

Use this procedure to install and configure the first storage array in a new cluster, using a single-controller configuration. Perform the steps in this procedure in conjunction with the procedures in the Oracle Solaris Cluster software installation documentation and your server hardware manual.

The following procedures contain instructions for other array-installation situations:

How to Install a Storage Array in a New Cluster, Using a Partner-Group Configuration
How to Add a Storage Array to an Existing Cluster, Using a Single-Controller Configuration
How to Add a Storage Array to an Existing Cluster, Using a Partner-Group Configuration

Note - When you upgrade firmware on a storage device or on an enclosure, redefine the stripe size of a LUN, or perform other LUN operations, a device ID might change unexpectedly. When you perform a check of the device ID configuration by running the cldevice check command, the following error message appears on your console if the device ID changed unexpectedly.

device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

To fix device IDs that report this error, run the cldevice repair command for each affected device.

Install the host adapters in the nodes that are to be connected to the storage array.
To install host adapters, see the documentation that shipped with your host adapters and nodes.
Install the FC hubs/switches.
To install FC hubs/switches, see the documentation that shipped with your FC hub/switch hardware.

Note - If you are using two FC switches and Sun SAN software to create a storage area network (SAN), seeSAN Solutions in an Oracle Solaris Cluster Environment in Oracle Solaris Cluster 3.3 Hardware Administration Manual for more information.
Set up a Reverse Address Resolution Protocol (RARP) server on the network on which the new storage arrays are to reside.
This RARP server enables you to assign an IP address to the new storage array by using each storage array's unique MAC address.
To set up a RARP server, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
If you are not adding a StorEdge T3+ array, install the media interface adapters (MIAs) in the storage array that you are installing, as shown in Figure 1-1.
To install a media interface adapter (MIA), see the Sun StorEdge T3 and T3+ Array Configuration Guide.
If you are adding a StorEdge T3+ array, and if it is necessary, install gigabit interface converters (GBICs) or Small Form-Factor Pluggables (SFPs) in the FC hubs/switches, as shown in Figure 1-1.
The GBICs or SFPs let you connect the FC hubs/switches to the storage array that you are installing. To install an FC hub/switch GBIC or an SFP, see the documentation that shipped with your FC hub/switch hardware.
Install fiber-optic cables between the FC hubs/switches and the storage array, as shown in Figure 1-1.
To install a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.
Install fiber-optic cables between the FC hubs/switches and the nodes, as shown in Figure 1-1.
Install the Ethernet cables between the storage array and the Local Area Network (LAN), as shown in Figure 1-1.
Figure 1-1 Installing a Single-Controller Configuration

Note - Figure 1-1 shows how to cable two storage arrays to enable data sharing and host-based mirroring. This configuration prevents a single-point of failure.
Install power cords to each storage array that you are installing.
Power on the storage array and confirm that all components are powered on and functional.
Note - The storage array might require a few minutes to boot.

To power on a storage array, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
(Optional) Configure the storage array with logical volumes.
To configure the storage array with logical volumes, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
Access each storage array that you are adding. Install the required controller firmware for the storage array.
The Oracle Enterprise Manager Ops Center 2.5 software helps you patch and monitor your data center assets. Oracle Enterprise Manager Ops Center 2.5 helps improve operational efficiency and ensures that you have the latest software patches for your software. Contact your Oracle representative to purchase Oracle Enterprise Manager Ops Center 2.5.
Additional information for using the Oracle patch management tools is provided in Oracle Solaris Administration Guide: Basic Administration at http://docs.sun.com. Refer to the version of this manual for the Oracle Solaris OS release that you have installed.
If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Oracle Solaris Cluster System Administration Guide to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.
For required firmware, see the Sun System Handbook.
Ensure that this new storage array has a unique target address.
To verify and assign a target address, see the Sun StorEdge T3 and T3+ Array Configuration Guide.
Reset the storage array.
To reboot or reset a storage array, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Install to the nodes the Oracle Solaris operating environment. Apply any required Oracle Solaris patches forOracle Solaris Cluster software and storage array support.
To install the Oracle Solaris operating environment, see your Oracle Solaris Cluster software installation documentation. For the location of required Oracle Solaris patches and installation instructions for Oracle Solaris Cluster software support, see your Oracle Solaris Cluster release notes documentation. For a list of required Oracle Solaris patches for storage array support, see the Sun StorEdge T3 and T3+ Array Release Notes.

How to Install a Storage Array in a New Cluster, Using a Partner-Group Configuration

Use this procedure to install and configure the first storage array partner groups in a new cluster. Perform the steps in this procedure in conjunction with the procedures in the Oracle Solaris Cluster software installation documentation and your server hardware manual.

Make certain that you are using the correct procedure. This procedure contains instructions about how to install a partner group into a new cluster, before the cluster is operational. The following procedures contain instructions for other array-installation situations:

How to Add a Storage Array to an Existing Cluster, Using a Partner-Group Configuration
How to Install a Storage Array in a New Cluster Using a Single-Controller Configuration
How to Add a Storage Array to an Existing Cluster, Using a Single-Controller Configuration

device id for nodename:/dev/rdsk/cXtYdZsN does not match physical 
device's id for ddecimalnumber, device may have been replaced.

To fix device IDs that report this error, run the cldevice repair command for each affected device.

Install the host adapters in the nodes to be connected to the storage arrays.
To install host adapters, see the documentation that shipped with your host adapters and nodes.
Install the Fibre Channel (FC) switches.
To install an FC switch, see the documentation that shipped with your switch hardware.

Note - If you are using two FC switches and Sun SAN software to create a storage area network (SAN), see SAN Considerations for more information.
If you are installing Sun StorEdge T3+ arrays, install the media interface adapters (MIAs) in the Sun StorEdge T3 arrays that you are installing, as shown in Figure 1-2.
Figure 1-2 Installing a Partner-Group Configuration
To install a media interface adapter (MIA), see the Sun StorEdge T3 and T3+ Array Configuration Guide.
If necessary, install GBICs or SFPs in the FC switches, as shown in Figure 1-2.
To install a GBIC or an SFP to an FC switch, see the documentation that shipped with your FC switch hardware.
Set up a Reverse Address Resolution Protocol (RARP) server on the network on which the new storage arrays are to reside.
This RARP server enables you to assign an IP address to the new storage arrays. Assign an IP address by using the storage array's unique MAC address. For the procedure about how to set up a RARP server, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Cable the storage arrays, as shown in Figure 1-2.
1. Connect the storage arrays to the FC switches by using fiber-optic cables.
2. Connect the Ethernet cables from each storage array to the LAN.
3. Connect interconnect cables between the two storage arrays of each partner group.
4. Connect power cords to each storage array.
To install fiber-optic, Ethernet, and interconnect cables, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Power on the storage arrays. Verify that all components are powered on and functional.
To power on the storage arrays and verify the hardware configuration, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Administer the storage arrays' network settings.
Use the telnet command to access the master controller unit and administer the storage arrays. To administer the storage array network addresses and settings, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
The master controller unit is the storage array that has the interconnect cables attached to the second port of each interconnect card, when viewed from the rear of the storage arrays. For example, Figure 1-2 shows the master controller unit of the partner group as the lower storage array. In this diagram, the interconnect cables are connected to the second port of each interconnect card on the master controller unit.
Install any required storage array controller firmware.
For partner-group configurations, use the telnet command to access the master controller unit. Install the required controller firmware.
The Oracle Enterprise Manager Ops Center 2.5 software helps you patch and monitor your data center assets. Oracle Enterprise Manager Ops Center 2.5 helps improve operational efficiency and ensures that you have the latest software patches for your software. Contact your Oracle representative to purchase Oracle Enterprise Manager Ops Center 2.5.
Additional information for using the Oracle patch management tools is provided in Oracle Solaris Administration Guide: Basic Administration at http://docs.sun.com. Refer to the version of this manual for the Oracle Solaris OS release that you have installed.
If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Oracle Solaris Cluster System Administration Guide to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.
For required firmware, see the Sun System Handbook.
Ensure that each storage array has a unique target address.
To verify and assign a target address to a storage array, see the Sun StorEdge T3 and T3+ Array Configuration Guide.
Ensure that the cache and mirror settings for each storage array are set to auto.
For more information, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
Ensure that the mp_support parameter for each storage array is set to mpxio.
For more information, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
Ensure that both storage array controllers are online.
For more information about how to correct the situation if both controllers are not online, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
(Optional) Configure the storage arrays with the desired logical volumes.
To create and initialize a logical volume, see the Sun StorEdge T3 and T3+ Array Administrator's Guide. To mount a logical volume, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Reset the storage arrays.
To reboot or reset a storage array, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
On all nodes, install the Oracle Solaris operating environment. Apply the required Oracle Solaris patches for Oracle Solaris Cluster software and storage array support.
To install the Oracle Solaris operating environment, see How to Install Solaris Software in Oracle Solaris Cluster Software Installation Guide
On all nodes, update the /devices and /dev entries.
```
# devfsadm -C 
```
On all nodes, confirm that all storage arrays that you installed are visible.
```
# luxadm display 
```

How to Add a Storage Array to an Existing Cluster, Using a Single-Controller Configuration

This procedure contains instructions about how to add a new storage array to a running cluster in a single-controller configuration. The following procedures contain instructions for other array-installation situations:

How to Add a Storage Array to an Existing Cluster, Using a Partner-Group Configuration
How to Install a Storage Array in a New Cluster Using a Single-Controller Configuration
How to Install a Storage Array in a New Cluster, Using a Partner-Group Configuration

This procedure defines Node A as the node with which you begin working. Node B is another node in the cluster.

This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.

Become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify role-based access control (RBAC) authorization.
Set up a Reverse Address Resolution Protocol (RARP) server on the network on which you want the new storage arrays to be located.
Assign an IP address to the new storage arrays.
This RARP server enables you to assign an IP address to the new storage array by using the storage array's unique MAC address.
To set up a RARP server, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
If you are adding a StorEdge T3+ array, install the media interface adapter (MIA) in storage array that you are adding, as shown in Figure 1-3.
If you are not adding a StorEdge T3+ array, skip this step.
To install a media interface adapter (MIA), see the Sun StorEdge T3 and T3+ Array Configuration Guide.
If necessary, install gigabit interface converters (GBICs) or Small Form-Factor Pluggables (SFPs) in the FC hub/switch, as shown in Figure 1-3.
The GBICs or SFPs enables you to connect the FC hubs/switches to the storage array that you are adding.
To install an FC hub/switch GBIC or an SFP, see the documentation that shipped with your FC hub/switch hardware.

Note - If you are using two FC switches and Sun SAN software to create a storage area network (SAN), see SAN Solutions in an Oracle Solaris Cluster Environment in Oracle Solaris Cluster 3.3 Hardware Administration Manual for more information.
Install the Ethernet cable between the storage array and the Local Area Network (LAN), as shown in Figure 1-3.
Power on the storage array array.
Note - The storage array might require a few minutes to boot.

To power on a storage array, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Access the storage array that you are adding. If necessary, install the required controller firmware for the storage array.
The Oracle Enterprise Manager Ops Center 2.5 software helps you patch and monitor your data center assets. Oracle Enterprise Manager Ops Center 2.5 helps improve operational efficiency and ensures that you have the latest software patches for your software. Contact your Oracle representative to purchase Oracle Enterprise Manager Ops Center 2.5.
Additional information for using the Oracle patch management tools is provided in Oracle Solaris Administration Guide: Basic Administration at http://docs.sun.com. Refer to the version of this manual for the Oracle Solaris OS release that you have installed.
If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Oracle Solaris Cluster System Administration Guide to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.
For required firmware, see the Sun System Handbook.
If this new storage array does not yet have a unique target address, change the target address for this new storage array.
If the target address for this array is already unique, skip this step.
To verify and assign a target address, see the Sun StorEdge T3 and T3+ Array Configuration Guide.
Install a fiber-optic cable between the FC hub/switch and the storage array, as shown in Figure 1-3.
To install a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

Figure 1-3 Adding a Single-Controller Configuration: Part I

Note - Figure 1-3 shows how to cable two storage arrays to enable data sharing and host-based mirroring. This configuration prevents a single-point of failure.
Configure the new storage array.
To create a logical volume, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
Determine the resource groups and device groups that are running on Node A and Node B.
Record this information because you use this information in Step 40 and Step 41 of this procedure to return resource groups and device groups to these nodes.
Use the following command:
```
# clresourcegroup status + 
# cldevicegroup status +
```
Move all resource groups and device groups off Node A.
```
# clnode evacuate
```

If you need to install a host adapter in Node A, and if it is the first FC host adapter on Node A, determine whether the Fibre Channel support packages are already installed on these nodes.

This product requires the following packages.

# pkginfo | egrep Wlux
system    SUNWluxd   Sun Enterprise Network Array sf Device Driver
system    SUNWluxdx  Sun Enterprise Network Array sf Device Driver
                                    (64-bit)
system    SUNWluxl   Sun Enterprise Network Array socal Device Driver
system    SUNWluxlx  Sun Enterprise Network Array socal Device Driver
                                    (64-bit)
system    SUNWluxop  Sun Enterprise Network Array firmware and utilities

If this is not the first FC host adapter on Node A, skip to Step 16. If you do not need to install a host adapter in Node A, skip to Step 35.

If the Fibre Channel support packages are not installed, install the required support packages that are missing.
The storage array packages are located in the Product directory of the Oracle Solaris DVD. Add any necessary packages.
Shut down Node A.
To shut down and power off a node, see your Oracle Solaris Cluster system administration documentation.
Power off Node A.
Install the host adapter in Node A.
To install a host adapter, see the documentation that shipped with your host adapter and node.
If necessary, power on and boot Node A into noncluster mode by adding -x to your boot instruction.
To boot a node in noncluster mode, see Chapter 3, Shutting Down and Booting a Cluster, in Oracle Solaris Cluster System Administration Guide.
If necessary, upgrade the host adapter firmware on Node A.
The Oracle Enterprise Manager Ops Center 2.5 software helps you patch and monitor your data center assets. Oracle Enterprise Manager Ops Center 2.5 helps improve operational efficiency and ensures that you have the latest software patches for your software. Contact your Oracle representative to purchase Oracle Enterprise Manager Ops Center 2.5.
Additional information for using the Oracle patch management tools is provided in Oracle Solaris Administration Guide: Basic Administration at http://docs.sun.com. Refer to the version of this manual for the Oracle Solaris OS release that you have installed.
If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Oracle Solaris Cluster System Administration Guide to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.
For required firmware, see the Sun System Handbook.
Connect a fiber-optic cable between the FC hub/switch and Node A, as shown in Figure 1-4.
To install an FC host adapter GBIC or an SFP, see your host adapter documentation. To install a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

Figure 1-4 Adding a Single-Controller Configuration: Part II
If necessary, install the required Oracle Solaris patches for array support on Node A.
For a list of required Oracle Solaris patches for storage array support, see the Sun StorEdge T3 and T3+ Array Release Notes.
Shut down Node A.
To shut down and power off a node, see your Oracle Solaris Cluster system administration documentation.
To create the new Oracle Solaris device files and links on Node A, perform a reconfiguration boot.
```
# boot -r
```
Label the new logical volume.
To label a logical volume, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
(Optional) On Node A, verify that the device IDs (DIDs) are assigned to the new LUNs.
```
# cldevice list -n NodeA -v
```

If you need to install a host adapter in Node B, and if the host adapter that you are installing the first FC host adapter on Node B, determine whether the Fibre Channel support packages are already installed on these nodes.

This product requires the following packages.

# pkginfo | egrep Wlux
system    SUNWluxd       Sun Enterprise Network Array sf Device Driver
system    SUNWluxdx       Sun Enterprise Network Array sf Device Driver 
                                    (64-bit)
system    SUNWluxl       Sun Enterprise Network Array socal Device Driver
system    SUNWluxlx       Sun Enterprise Network Array socal Device Driver 
                                    (64-bit)
system    SUNWluxop       Sun Enterprise Network Array firmware and utilities

If this is not the first FC host adapter on Node B, skip to Step 29. If you do not need to install a host adapter, skip to Step 34.

If the Fibre Channel support packages are not installed, install the required support packages that are missing.
The storage array packages are located in the Product directory of the Oracle Solaris DVD. Add any necessary packages.
Shut down Node B.
To shut down and power off a node, see your Oracle Solaris Cluster system administration documentation.
Power off Node B.
For more information, see your Oracle Solaris Cluster system administration documentation.
Install the host adapter in Node B.
To install a host adapter, see the documentation that shipped with your host adapter and node.
If necessary, power on and boot Node B into noncluster mode by adding -x to your boot instruction.
To boot a node in noncluster mode, see Chapter 3, Shutting Down and Booting a Cluster, in Oracle Solaris Cluster System Administration Guide.
If necessary, upgrade the host adapter firmware on Node B.
The Oracle Enterprise Manager Ops Center 2.5 software helps you patch and monitor your data center assets. Oracle Enterprise Manager Ops Center 2.5 helps improve operational efficiency and ensures that you have the latest software patches for your software. Contact your Oracle representative to purchase Oracle Enterprise Manager Ops Center 2.5.
Additional information for using the Oracle patch management tools is provided in Oracle Solaris Administration Guide: Basic Administration at http://docs.sun.com. Refer to the version of this manual for the Oracle Solaris OS release that you have installed.
If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Oracle Solaris Cluster System Administration Guide to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.
For required firmware, see the Sun System Handbook.
If necessary, install a GBIC or an SFP, as shown in Figure 1-5.
To install an FC hub/switch GBIC or an SFP, see the documentation that shipped with your FC hub/switch hardware.

Note - If you are using two FC switches and Sun SAN software to create a storage area network (SAN), see SAN Solutions in an Oracle Solaris Cluster Environment in Oracle Solaris Cluster 3.3 Hardware Administration Manual for more information.
If necessary, connect a fiber-optic cable between the FC hub/switch and Node B, as shown in Figure 1-5.
To install an FC host adapter GBIC or an SFP, see your host adapter documentation. To install a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Configuration Guide.

Figure 1-5 Adding a Single-Controller Configuration: Part III
If necessary, install the required Oracle Solaris patches for storage array support on Node B.
For a list of required Oracle Solaris patches for storage array support, see the Sun StorEdge T3 and T3+ Array Release Notes.
Shut down Node B.
To shut down and power off a node, see your Oracle Solaris Cluster system administration documentation.
Perform a reconfiguration boot to create the new Oracle Solaris device files and links on Node B.
```
# boot -r
```
(Optional) On Node B, verify that the device IDs (DIDs) are assigned to the new LUNs.
```
# cldevice list -n NodeB -v
```
(Optional) Restore the device groups to the original node.
Do the following for each device group that you want to return to the original node.
```
# cldevicegroup switch -n nodename devicegroup1[ devicegroup2 …]
```
-n nodename

The node to which you are restoring device groups.

devicegroup1[ devicegroup2 …]

The device group or groups that you are restoring to the node.
(Optional) Restore the resource groups to the original node.
Do the following for each resource group that you want to return to the original node.
```
# clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]
```
nodename

For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

resourcegroup1[ resourcegroup2 …]

The resource group or groups that you are returning to the node or nodes.
Perform volume management administration to incorporate the new logical volumes into the cluster.
For more information, see your Solaris Volume Manager or Veritas Volume Manager documentation.

How to Add a Storage Array to an Existing Cluster, Using a Partner-Group Configuration

This procedure contains instructions for adding new storage array partner groups to a running cluster. The following procedures contain instructions for other array-installation situations:

How to Install a Storage Array in a New Cluster, Using a Partner-Group Configuration
How to Install a Storage Array in a New Cluster Using a Single-Controller Configuration
How to Add a Storage Array to an Existing Cluster, Using a Single-Controller Configuration

This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical.

Become superuser or assume a role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization.
Set up a Reverse Address Resolution Protocol (RARP) server on the network you want the new storage arrays to reside on. Afterward, assign an IP address to the new storage arrays.
Note - Assign an IP address to the master controller unit only. The master controller unit is the storage array that has the interconnect cables attached to the second port of each interconnect card, as shown in Figure 1-6.

This RARP server enables you to assign an IP address to the new storage arrays. Assign an IP address by using the storage array's unique MAC address. To set up a RARP server, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Install the Ethernet cable between the storage arrays and the local area network (LAN), as shown in Figure 1-6.
If they are not already installed, install interconnect cables between the two storage arrays of each partner group, as shown in Figure 1-6.
To install interconnect cables, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

Figure 1-6 Adding a Partner-Group Configuration: Part I
Power on the storage arrays.
Note - The storage arrays might require several minutes to boot.

To power on storage arrays, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Administer the storage arrays' network addresses and settings.
Use the telnet command to access the master controller unit and to administer the storage arrays.
To administer the network address and the settings of a storage array, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Install any required storage array controller firmware upgrades.
For partner-group configurations, use the telnet command to the master controller unit. If necessary, install the required controller firmware for the storage array.
For the required revision number of the storage array controller firmware, see the Sun StorEdge T3 Disk Tray Release Notes.
Ensure that each storage array has a unique target address.
To verify and assign a target address to a storage array, see the Sun StorEdge T3 and T3+ Array Configuration Guide.
Ensure that the cache and mirror settings for each storage array are set to auto.
For more information, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
Ensure that the mp_support parameter for each storage array is set to mpxio.
For more information, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
Configure the new storage arrays with the desired logical volumes.
To create and initialize a logical volume, see the Sun StorEdge T3 and T3+ Array Administrator's Guide. To mount a logical volume, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
Reset the storage arrays.
To reboot or reset a storage array, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.
If you are adding Sun StorEdge T3+ arrays, install the media interface adapter (MIA) in the Sun StorEdge T3 arrays that you are adding, as shown in Figure 1-6.
To install an MIA, see the Sun StorEdge T3 and T3+ Array Configuration Guide.
If necessary, install GBICs or SFPs in the FC switches, as shown in Figure 1-6.
To install a GBIC or an SFP to an FC switch, see the documentation that shipped with your FC switch hardware.
Install a fiber-optic cable between each FC switch and both new storage arrays of the partner group, as shown in Figure 1-6.
To install a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

Note - If you are using two FC switches and Sun SAN software to create a storage area network (SAN), see SAN Considerations for more information.
Determine the resource groups and device groups that are running on all nodes.
Record this information because you use it in Step 30 of this procedure to return resource groups and device groups to these nodes.
Use the following command:
```
# clresourcegroup status +
# cldevicegroup status +
```
Move all resource groups and device groups off each node in the cluster.
```
# clnode evacuate
```

If you need to install host adapters in the node, and if the host adapter you are installing is the first adapter on the node, determine whether the required support packages are already installed on this node.

The following packages are required.

# pkginfo | egrep Wlux
system     SUNWluxd     Sun Enterprise Network Array sf Device Driver
system     SUNWluxdx    Sun Enterprise Network Array sf Device Driver
                                (64-bit)
system     SUNWluxl     Sun Enterprise Network Array socal Device Driver
system     SUNWluxlx    Sun Enterprise Network Array socal Device Driver
                                (64-bit)
system     SUNWluxop    Sun Enterprise Network Array firmware and utilities
system     SUNWluxox    Sun Enterprise Network Array libraries (64-bit)

If this is not the first host adapter on the node, skip to Step 20.

If the required support packages are not present, install them.
The support packages are located in the Product directory of the Oracle Solaris DVD. Add any missing packages.
If you need to install host adapters in the node, shut down and power off the node, and then install them in the node.
To shut down and power off a node, see your Oracle Solaris Cluster system administration documentation.
To install host adapters, see the documentation that shipped with your host adapters and nodes.
If you installed host adapters in the node, power on and boot the node into noncluster mode.
To boot nodes, see Chapter 3, Shutting Down and Booting a Cluster, in Oracle Solaris Cluster System Administration Guide.
If you installed host adapters in the node, and if necessary, upgrade the host adapter firmware on the node.
If necessary, install the required Oracle Solaris patches for storage array support on the node.
The Oracle Enterprise Manager Ops Center 2.5 software helps you patch and monitor your data center assets. Oracle Enterprise Manager Ops Center 2.5 helps improve operational efficiency and ensures that you have the latest software patches for your software. Contact your Oracle representative to purchase Oracle Enterprise Manager Ops Center 2.5.
Additional information for using the Oracle patch management tools is provided in Oracle Solaris Administration Guide: Basic Administration at http://docs.sun.com. Refer to the version of this manual for the Oracle Solaris OS release that you have installed.
If you must apply a patch when a node is in noncluster mode, you can apply it in a rolling fashion, one node at a time, unless instructions for a patch require that you shut down the entire cluster. Follow the procedures in How to Apply a Rebooting Patch (Node) in Oracle Solaris Cluster System Administration Guide to prepare the node and to boot it in noncluster mode. For ease of installation, consider applying all patches at the same time. That is, apply all patches to the node that you place in noncluster mode.
For required firmware, see the Sun System Handbook.
If you installed host adapters in the node, reboot the node in cluster mode.
Connect fiber-optic cables between the node and the FC switches, as shown in Figure 1-7.
To install a fiber-optic cable, see the Sun StorEdge T3 and T3+ Array Installation, Operation, and Service Manual.

Note - If you are using two FC switches and Sun SAN software to create a storage area network (SAN), see SAN Considerations for more information.

Figure 1-7 Adding a Partner-Group Configuration: Part II
On the current node, update the /devices and /dev entries.
```
# devfsadm
```
From any node in the cluster, update the global device namespace.
```
# cldevice populate
```
Label the new storage array logical volume.
To label a logical volume, see the Sun StorEdge T3 and T3+ Array Administrator's Guide.
(Optional) On the current node, verify that the device IDs (DIDs) are assigned to the new LUNs.
```
# cldevice list -n CurrentNode -v
```
(Optional) Restore the resource groups to the original node.
Do the following for each resource group that you want to return to the original node.
```
# clresourcegroup switch -n nodename resourcegroup1[ resourcegroup2 …]
```
nodename

For failover resource groups, the node to which the groups are returned. For scalable resource groups, the node list to which the groups are returned.

resourcegroup1[ resourcegroup2 …]

The resource group or groups that you are returning to the node or nodes.
For each of the other nodes in the cluster, repeat Step 17 through Step 30.
Perform volume management administration to incorporate the new logical volumes into the cluster.
For more information, see your Veritas Volume Manager documentation.

Skip Navigation Links
Exit Print View
	Oracle Solaris Cluster 3.3 With Sun StorEdge T3 or T3+ Array Manual SPARC Platform Edition