C H A P T E R  4

Configuring Multipathing Software

This chapter explains how to configure multipathing feature for FC devices provided in the Solaris 10 OS. It also provides the considerations while enabling or disabling multipathing feature on SPARC based systems, x86 based systems, per-port basis and third-party storage devices.

This chapter contains the following sections:


Configuring Multipathing

The multipathing feature for FC devices can be configured to control all Sun supported FC HBAs. Multipathing is disabled by default for FC devices on SPARC based systems, but is enabled by default on x86 based systems. Configuration of the multipathing software depends on how you intend to use your system.



Note - The multipathing feature is not available for parallel SCSI devices but is available for FC disk devices. Multipathing is not supported on tape drives or libraries, or on IP over FC.




Enabling or Disabling Multipathing

Before you change multipathing configuration consider the following and then follow the instructions for your machine architecture (SPARC or x86) described in the subsequent sections.

Some devices need to be properly configured to work with the multipathing software. Refer to your storage array documentation for details on the device specific configuration for your device.

In the /dev and /devices trees, multipath-enabled devices receive new names that indicate that they are under multipath control. This means a device will have a different name from its original name when it is under multipath control.

Example device name with multipath disabled:


/dev/dsk/c1t1d0s0

Example device name with multipath enabled:


/dev/dsk/c3t2000002037CD9F72d0s0

Because of this, if you have applications that directly consume device names, they must be configured to use the new names whenever you change multipath configuration from disabled to enabled or vice-versa.

The system's /etc/vfstab file and the dump configuration also contains references to device names. On SPARC based systems, the stmsboot(1M) command described in the following sections automatically updates the /etc/vfstab and the dump configuration with the new device names. On x86 based systems this must be performed manually. You need to do this only if the device names listed in /etc/vfstab or your dump device are affected by the device name changes. For updating /etc/vfstab, simply edit the file with any text editor and substitute new device names in place of old device names. Use dumpadm(1M) command to update the dump configuration.


Enabling or Disabling Multipathing on SPARC Based Systems

On SPARC based systems, you can enable or disable multipathing on FC devices using the stmsboot(1M) command. The command also updates /etc/vfstab and dump configuration to reflect devices name changes during the next reboot.

The following consideration apply (for stmsboot -e, -d, and -u options):



Note - The behavior of stmsboot command in the Solaris 10 differs from the stmsboot command provided by Sun StorEdge SAN Foundation software in the Solaris 8 and 9 OS. In the Solaris 8 and 9 OS, stmsboot command is used to enable or disable multipathing on the boot device only. In the Solaris 10, the command is used to enable or disable multipathing on all devices. The stmsboot(1M) is available only on SPARC based systems.




procedure icon  To Enable Multipathing on SPARC Based Systems

Perform the following steps to enable multipathing on all FC devices on SPARC based systems. If you want to enable multipathing only on specific FC ports refer to the section Enabling or Disabling Multipathing on a Per-Port Basis.

1. The multipathing software automatically recognizes Sun supported devices. If you want to enable multipathing on third-party devices, add entries to /kernel/drv/scsi_vhci.conf file as mentioned in the section Device Name Change Considerations.

2. Run the stmsboot -e command:


# stmsboot -e
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y

3. You are prompted to reboot. During the reboot, /etc/vfstab and the dump configuration will be updated to reflect the device name changes.

After the reboot, if necessary, configure your applications to use new device names as described in Device Name Change Considerations.


procedure icon  To Disable Multipathing on SPARC Based Systems

Perform the following steps to disable multipathing on all FC devices on SPARC based systems, if you have enabled it before. If you want to disable multipathing only on specific FC ports refer to the section Enabling or Disabling Multipathing on a Per-Port Basis.

1. Run the stmsboot -d command:


# stmsboot -d
WARNING: This operation will require a reboot.
Do you want to continue? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now? [y/n] (default: y) y

2. You are prompted to reboot. During the reboot, /etc/vfstab and the dump configuration will be updated to reflect the device name changes.

After the reboot, if necessary, configure your applications to use new device names as described in Device Name Change Considerations.


Enabling or Disabling Multipathing on x86 Based Systems

Multipathing is enabled by default on x86 based systems. Use the following steps to disable multipathing or to enable multipathing if you have disabled it previously.



Caution - If the critical file systems needed for booting, such as the root (the `/'), /usr, or /var file systems, are on FC disks, you should not disable multipathing on them otherwise your system will not boot. You may disable multipathing on all FC devices if the critical file systems are not on FC disks. If the critical file systems are on FC disks, you may disable multipathing only on those FC HBA controller ports which aren't connected to the disks containing the critical file systems using the instructions in the section Enabling or Disabling Multipathing on a Per-Port Basis.



1. The multipathing software automatically recognizes Sun supported devices. If you want to enable multipathing on third-party devices, add entries to /kernel/drv/scsi_vhci.conf file as mentioned in the section To Configure Third-Party Devices.

2. Open the /kernel/drv/fp.conf file in a text editor

3. If you want to enable multipathing:

Change the value of mpxio-disable to "no". If the entry doesn't exist add a new entry.


mpxio-disable="no";

If you want to disable multipathing:

Change the value of mpxio-disable to "yes". If the entry doesn't exist add a new entry.


mpxio-disable="yes";



Note - This will not affect your per-port multipathing configuration. For information on per-port configuration refer to the section Enabling or Disabling Multipathing on a Per-Port Basis.



4. Save and exit the fp.conf file.

5. Perform a reconfiguration reboot.


# touch /reconfigure
# shutdown -g0 -y -i6

6. If necessary, perform device name updates as described in Device Name Change Considerations.


Enabling or Disabling Multipathing on a Per-Port Basis

Multipathing can be enabled or disabled on specific FC Host Bus Adapter (HBA) controller ports for finer control. If you enable multipathing on a specific HBA controller port, all supported devices connected to the controller port will have multipathing enabled on them.

The procedure described below applies for both SPARC and x86 based machines.

Considerations for Per-Port Configuration

Before you start configuring the software by port, consider the following:


procedure icon  To Configure Multipathing by Port

1. Log in as superuser.

Determine the HBA controller ports that you want the multipathing software to control. For example, to select the desired device, perform an ls -l command on /dev/fc. The following is an example of the ls -l command output.


lrwxrwxrwx   1 root   root            49 Apr 17 18:14 fp0 ->
../../devices/pci@6,2000/SUNW,qlc@2/fp@0,0:devctl
lrwxrwxrwx   1 root   root            49 Apr 17 18:14 fp1 ->
../../devices/pci@7,2000/SUNW,qlc@2/fp@0,0:devctl
lrwxrwxrwx   1 root   root            49 Apr 17 18:14 fp2 ->
../../devices/pci@a,2000/SUNW,qlc@2/fp@0,0:devctl
lrwxrwxrwx   1 root   root            49 Apr 17 18:14 fp3 ->
../../devices/pci@b,2000/SUNW,qlc@2/fp@0,0:devctl
lrwxrwxrwx   1 root   root            50 Apr 17 18:14 fp4 ->
../../devices/pci@12,2000/SUNW,qlc@2/fp@0,0:devctl
lrwxrwxrwx   1 root   root            56 Apr 17 18:14 fp5 ->
../../devices/pci@13,2000/pci@2/SUNW,qlc@4/fp@0,0:devctl
lrwxrwxrwx   1 root   root            56 Apr 17 18:14 fp6 ->
../../devices/pci@13,2000/pci@2/SUNW,qlc@5/fp@0,0:devctl
lrwxrwxrwx   1 root   root            56 Apr 17 18:14 fp7 ->
../../devices/sbus@7,0/SUNW,qlc@0,30400/fp@0,0:devctl



Note - The fp7 is a SBus HBA. The fp5 and fp6 include two /pci elements. This indicates a dual PCI HBA. The rest of the entries do not have additional PCI bridges and are single PCI HBAs.



2. Open the /kernel/drv/fp.conf file and explicitly enable or disable multipathing on an HBA controller port. This file allows you to enable or disable both the global multipath setting, as well as multipath settings for specific ports.

Change the value of global mpxio-disable property. If the entry doesn't exist add a new entry. The global setting applies to all ports except the ports specified by the per-port entries.

a. To enable multipathing globally, change to


mpxio-disable="no";

b. To disable multipathing globally, change to


mpxio-disable="yes";

Add the per-port mpxio-disable entries - one entry for every HBA controller port you want to configure. Per-port settings override the global setting for the specified ports.

a. To enable multipathing on a HBA port, add


name="fp" parent="parent name" port=port-number mpxio-disable="no";

b. To disable multipathing on a HBA port, add


name="fp" parent="parent name" port=port-number mpxio-disable="yes";

The following example disables multipathing on all HBA controller ports except the two specified ports:


mpxio-disable="yes";
name="fp" parent="/pci@6,2000/SUNW,qlc@2" port=0 mpxio-disable="no";
name="fp" parent="/pci@13,2000/pci@2/SUNW,qlc@5" port=0 mpxio-disable="no";

3. If running on a SPARC based system, perform the following:

Run the stmsboot -u command:


# stmsboot -u
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y

You are prompted to reboot. During the reboot, /etc/vfstab and the dump configuration will be updated to reflect the device name changes.

If running on a x86 based system, perform a reconfiguration reboot.


# touch /reconfigure
# shutdown -g0 -y -i6

4. If necessary, perform device name updates as described in Device Name Change Considerations.


procedure icon  To Configure an HBA Controller Port on a SPARC System

The following steps show the configuration of a sample single PCI HBA. fp0 ->../../devices/pci@6,2000/SUNW,qlc@2/fp@0,0:devctl This sample entry from the ls -l command output shown in To Configure Multipathing by Port indicates a single PCI HBA.

1. Open the /kernel/drv/fp.conf file in a text editor

2. To explicitly enable multipathing on this HBA port, but disable on the rest of the HBA ports, add the following:


mpxio-disable="yes";
name="fp" parent="/pci@6,2000/SUNW,qlc@2" port=0 mpxio-disable="no";

To explicitly disable multipathing on this HBA port, but enable on the rest of the HBA ports, add the following:


mpxio-disable="no";
name="fp" parent="/pci@6,2000/SUNW,qlc@2" port=0 mpxio-disable="yes";

3. Save and exit the file.

4. Run the stmsboot -u command:


# stmsboot -u
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y

You are prompted to reboot. During the reboot, /etc/vfstab and the dump configuration will be updated to reflect the device name changes.

5. After the reboot, if necessary, configure your applications to use new device names as described in Device Name Change Considerations.


procedure icon  To Configure an HBA Controller Port on an x86 Based System

The following steps show the configuration of a sample dual PCI HBA. fp6 -> ../../devices/pci@13,2000/pci@2/SUNW,qlc@5/fp@0,0:devctl This sample entry from the ls -l command output shown in To Configure Multipathing by Port indicates a dual PCI HBA.

1. Open the /kernel/drv/fp.conf file in a text editor

2. To explicitly enable multipathing on this HBA port, but disable on the rest of the HBA ports, add the following:


mpxio-disable="yes";
name="fp" parent="/pci@13,2000/pci@2/SUNW,qlc@5" port=0 mpxio-disable="no";

To explicitly disable multipathing on this HBA port, but enable on the rest of the HBA ports, add the following:


mpxio-disable="no";
name="fp" parent="/pci@13,2000/pci@2/SUNW,qlc@5" port=0 mpxio-disable="yes";

3. Save and exit the file.

4. Perform a reconfiguration reboot.


# touch /reconfigure
# shutdown -g0 -y -i6

5. If necessary, perform device name updates as described in Device Name Change Considerations.


Disabling Multipathing on Third-Party Devices

Multipathing can be disabled for all devices of a certain Vendor ID/Product ID combination (from Inquiry data). This exclusion is specified in scsi_vhci.conf.

1. Open the /kernel/drv/scsi_vhci.conf file in a text editor

2. Add the vendor ID and product ID entries.

The vendor ID and product ID are the vendor and product indentification strings that the device returns in SCSI INQUIRY data. The vendor ID must be eight characters long. You must specify all eight characters (even if the trailing characters are spaces). The product ID can be up to 16 characters long.


device-type-scsi-options-list =
"VendorID1ProductID1", "disable-option",
"VendorID2ProductID2", "disable-option",
...
"VendorIDnProductIDn", "disable-option";
disable-option = 0x7000000;

The entries in the preceding example are separated by the ',' character and the last vendor/product entry is terminated by the ';' character. For example, to add a device from a "ACME" vendor with a product ID of "MSU", you would add:


device-type-scsi-options-list =
"ACME	MSU", "disable-option";
disable-option = 0x7000000;

In addition to "ACME," to add a device from the "XYZ" vendor with a product ID of "ABC", you would add:


device-type-scsi-options-list =
"ACME	MSU", "disable-option",
"XYZ	ABC", "disable-option";
disable-option = 0x7000000;

3. Save and exit the scsi_vhci.conf file.

4. Perform one of the following:

You are prompted to reboot. During the reboot, /etc/vfstab and the dump configuration are updated to reflect the device name changes.

5. If necessary, perform the device name updates as described under the bullet titled Device Name Change Considerations.


Configuring Automatic Failback

Some storage devices have controllers configured as PRIMARY and SECONDARY as part of the array configuration. The SECONDARY paths may operate at a lower performance level than the PRIMARY paths. The multipathing software uses the PRIMARY path to talk to the storage device and keeps the SECONDARY path on standby. In the event of a PRIMARY path failure, the multipathing software automatically directs all I/O traffic over the SECONDARY path, with the PRIMARY path taken offline. This is called a "failover" operation. When the failure associated with the PRIMARY path has been repaired, the multipathing software automatically directs all I/O traffic over the PRIMARY path and keeps the SECONDARY path standby as before. This is called a "failback" operation.

If you want, you may disable the automatic failback operation, in which case, the multipathing software will not automatically failback to the PRIMARY path. Later, after the failure associated with the PRIMARY path has been repaired, you may do a manual failback operation using the luxadm(1M) command.

1. Open the /kernel/drv/scsi_vhci.conf file in a text editor.

2. To disable automatic failback capability, change the auto-failback entry to:


auto-failback="disable";

To enable automatic failback capability (if you disabled it before), change the auto-failback entry to:


auto-failback="enable";

3. Save and exit the file.

4. Reboot the system.


# shutdown -g0 -y -i6


Configuring Third-Party Storage Devices

Multipathing can be configured on third-party symmetric storage devices. A symmetric storage device is defined as a storage device in which all paths to the storage device are active and I/O commands can be issued through any path.



Note - Before configuring any third-party device, ensure that they are supported. Refer to your third-party user documentation, or third-party vendor for information on proper vendor and product IDs, modes and various settings required for the device to work with multipathing software.



Considerations for Third-Party Device Configuration

Before you configure third-party devices for multipathing, be aware of the following:


procedure icon  To Configure Third-Party Devices

Perform the following steps to configure third-party devices if your system already has multipathing enabled. If your system has multipathing disabled, you can configure third-party devices while enabling multipathing as described earlier in this chapter.

1. Open the /kernel/drv/scsi_vhci.conf file in a text editor.

2. Add the vendor ID and product ID entries.

The vendor ID and product ID are the vendor and product identification strings that the device returns in SCSI INQUIRY data. The vendor ID must be eight characters long. You must specify all eight characters (even if the trailing characters are spaces).

The product Id can be up to 16 characters long.


device-type-scsi-options-list =
"VendorID1ProductID1", "symmetric-option",
"VendorID2ProductID2", "symmetric-option",
...
"VendorIDnProductIDn", "symmetric-option";
symmetric-option = 0x1000000;

Note that the entries are separated by the ',' character and the last vendor/product entry is terminated by the ';' character.

For example, to add a device from a "ACME" vendor with a product ID of "MSU", you would add:


device-type-scsi-options-list =
"ACME    MSU", "symmetric-option";
symmetric-option = 0x1000000;

In addition to "ACME", to add a device from a "XYZ" vendor with a product ID of "ABC", you would add:


device-type-scsi-options-list =
"ACME    MSU", "symmetric-option",
"XYZ     ABC", "symmetric-option";
symmetric-option=0x1000000;

3. Save and exit the scsi_vhci.conf file.

4. If running on a SPARC based system, perform the following:

Run the stmsboot -u command:


# stmsboot -u
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y

You are prompted to reboot. During the reboot, /etc/vfstab and the dump configuration will be updated to reflect the device name changes.

If running on a x86 based system, perform a reconfiguration reboot.


# touch /reconfigure
# shutdown -g0 -y -i6

5. If necessary, perform device name updates as described in Device Name Change Considerations.


Displaying Device Name Changes

The stmsboot(1M) command, available only on SPARC based systems, can be used to display the mapping between non-multipath enabled and multipath enabled devices names, after the changes made to the multipath configuration by stmsboot -e, -d or -u options take effect. Both the non-multipath enabled and multipath enabled device names must exist in order to show the mapping.

The -L option displays the mapping of devices on all controllers.


# stmsboot -L
non-STMS device name                    STMS device name
----------------------------------------------------------------
/dev/rdsk/c2t8d0	/dev/rdsk/c10t500000E01046DEE0d0
/dev/rdsk/c2t0d0	/dev/rdsk/c10t500000E01046B070d0
/dev/rdsk/c2t3d0	/dev/rdsk/c10t20000020372A40AFd0
/dev/rdsk/c2t12d0	/dev/rdsk/c10t500000E01046DEF0d0
/dev/rdsk/c2t11d0	/dev/rdsk/c10t500000E01046E390d0
/dev/rdsk/c3t8d0	/dev/rdsk/c10t500000E01046DEE0d0
/dev/rdsk/c3t0d0	/dev/rdsk/c10t500000E01046B070d0
/dev/rdsk/c3t3d0	/dev/rdsk/c10t20000020372A40AFd0
/dev/rdsk/c3t12d0	/dev/rdsk/c10t500000E01046DEF0d0
/dev/rdsk/c3t11d0	/dev/rdsk/c10t500000E01046E390d0

The -l option displays the mapping of devices on only the specified controller.The following example displays the mapping of controller 3.


# stmsboot -l3
non-STMS device name                    STMS device name
----------------------------------------------------------------
/dev/rdsk/c3t8d0	/dev/rdsk/c10t500000E01046DEE0d0
/dev/rdsk/c3t0d0	/dev/rdsk/c10t500000E01046B070d0
/dev/rdsk/c3t3d0	/dev/rdsk/c10t20000020372A40AFd0
/dev/rdsk/c3t12d0	/dev/rdsk/c10t500000E01046DEF0d0
/dev/rdsk/c3t11d0	/dev/rdsk/c10t500000E01046E390d0