test

Solaris Express SAN Configuration and Multipathing Guide

Chapter 4 Configuring Multipathing Software

This chapter explains how to configure the Sun StorageTek Traffic Manager software that is provided in the Solaris Express Developer Edition OS for Fibre Channel (FC) devices . It also provides the considerations while enabling or disabling the multipathing feature on SPARC based systems, x86 based systems, per-port basis and third-party storage devices.

The following topics are discussed:

Configuring Multipathing

The multipathing feature for FC devices can be configured to control all Sun supported FC HBAs. Multipathing is disabled by default for FC devices on SPARC based systems, but is enabled by default on x86 based systems. Configuration of the multipathing software depends on how you intend to use your system.

Note –

The multipathing feature is not available for parallel SCSI devices but is available for FC disk devices. Multipathing is not supported on tape drives or libraries, or on IP over FC.

Before you configure third-party devices for multipathing, be aware of the following:

Multipathing Considerations

Before you change multipathing configuration note the following considerations. Then follow the instructions for your machine architecture (SPARC or x86) described in the subsequent sections. Some devices need to be properly configured to work with the multipathing software. Refer to your storage array documentation for details on the device specific configuration for your device.

In the /dev and /devices trees, multipath-enabled devices receive new names that indicate that they are under multipath control. A device therefore will have a different name from its original name when it is under multipath control.

Device name with multipath disabled:


/dev/dsk/c1t1d0s0

Device name with multipath enabled:


/dev/dsk/c0t60003BA27D5170003E5D2A7A0007F3D2d0s0

Therefore, applications that use device names directly must be configured to use the new names whenever you change multipath configuration from disabled to enabled or vice-versa.

The system’s /etc/vfstab file and the dump configuration also contain references to device names. On both SPARC and x86/x64-based systems the stmsboot(1m) command described in the following sections automatically updates the /etc/vfstab file dump configuration with the new device names. If you have application-dependent file systems which are not listed in the file /etc/vfstab, you can use the stmsboot command to determine the mapping between the old and new device paths.

Caution – Caution –

if you have run devfsadm -C or performed a reconfiguration reboot then the old device paths will not exist and the stmsboot -L command will fail to provide this information.

Enabling and Disabling Multipathing

With the release of the Solaris Express Developer Edition OS, the stmsboot(1m) command is now available for use on SPARC, x86, and x64 based systems to enable or disable multipathing for Fibre Channel (FC) and SAS devices. The stmsboot command updates the /etc/vfstab file and dump configuration to reflect device name changes during the next reboot. You do not need to manually edit the /kernel/drv/fp.conf or /kernel/drv/mpt.conf files.

The following considerations apply to the stmsboot -e, -d, and -u options:

Note –

The behavior of the stmsboot command in the Solaris Express Developer Edition OS differs from the stmsboot command provided by Sun StorEdge SAN Foundation software in the Solaris 8 and Solaris 9 OS. In the Solaris 8 and Solaris 9 OS, the stmsboot command is used to enable or disable multipathing on the boot device only for SPARC based hosts only. In the Solaris Express Developer Edition OS, the command is used to enable or disable multipathing on all capable devices.

The stmsboot command is available on both SPARC and x86/x64-based systems from the Solaris Express Developer Edition OS onwards. In the Solaris Express Developer Edition OS the stmsboot command is used to enable, disable or update multipathing for Serial-Attached SCSI devices as well as FC devices.

ProcedureTo Enable Multipathing

Perform the following steps to enable multipathing on all multipath-capable devices on SPARC or x86/x64 based systems. If you want to enable multipathing only on specific FC or SAS HBA ports, refer to Enabling or Disabling Multipathing on a Per-Port Basis.

The multipathing software automatically recognizes Sun supported devices. If you want to enable multipathing on third-party devices, add entries to the /kernel/drv/scsi_vhci.conf file as described by Device Name Change Considerations in Multipathing Considerations.

  1. Run the stmsboot -e command:


    # stmsboot -e
WARNING: stmsboot operates on each supported multipath-capable controller
         	detected in a host. In your system, these controllers are

	/devices/pci@780/pci@0/pci@8/SUNW,qlc@0/fp@0,0
	/devices/pci@780/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2
	/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0
	/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0

	If you do NOT wish to operate on these controllers, please quit stmsboot
	and re-invoke with -D { fp | mpt } to specify which controllers you wish
	to modify your multipathing configuration for.

	Do you wish to continue? [y/n] (default: y) y
	Checking mpxio status for driver fp
	Checking mpxio status for driver mpt
	WARNING: This operation will require a reboot.
	Do you want to continue ? [y/n] (default: y) y
	The changes will come into effect after rebooting the system.
	Reboot the system now ? [y/n] (default: y) y
    Note –

    During the reboot, /etc/vfstab and the dump configuration are updated to reflect the device name changes.

  2. (Optional) After the reboot, if necessary, configure your applications to use new device names as described by Device Name Change Considerations in Multipathing Considerations.

ProcedureTo Disable Multipathing

Perform the following steps to disable multipathing on all multipath-capable devices on SPARC or x86/x64 based systems. If you want to disable multipathing only on specific FC or SAS HBA ports, refer to Enabling or Disabling Multipathing on a Per-Port Basis.

  1. Run the stmsboot -d command.


    # stmsboot -d
WARNING: stmsboot operates on each supported multipath-capable controller
         	detected in a host. In your system, these controllers are

	/devices/pci@780/pci@0/pci@8/SUNW,qlc@0/fp@0,0
	/devices/pci@780/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2
	/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0
	/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0

	If you do NOT wish to operate on these controllers, please quit stmsboot
	and re-invoke with -D { fp | mpt } to specify which controllers you wish
	to modify your multipathing configuration for.

	Do you wish to continue? [y/n] (default: y) y
	Checking mpxio status for driver fp
	Checking mpxio status for driver mpt
	WARNING: This operation will require a reboot.
	Do you want to continue ? [y/n] (default: y) y
	The changes will come into effect after rebooting the system.
	Reboot the system now ? [y/n] (default: y) y
    Note –

    During the reboot, /etc/vfstab and the dump configuration are updated to reflect the device name changes.

  2. (Optional) After the reboot, if necessary, configure your applications to use new device names as described by Device Name Change Considerations in Multipathing Considerations.

Enabling or Disabling Multipathing on a Per-Port Basis

Multipathing can be enabled or disabled on specific Fibre Channel Host Bus Adapter (HBA) controller ports. If you enable multipathing on a specific HBA port controller port, all supported devices connected to that controller port will be enabled for multipath operation.

The following procedure applies to both SPARC and x86 based machines.

Port Configuration Considerations

Before you start configuring the software by port, consider the following:

ProcedureTo Configure Multipathing by Port

Depending on how many ports you want the multipathing software to control, you can enable or disable multipathing globally or for specified ports.

  1. Log in as root (su - root).

  2. Determine the HBA controller ports that you want the multipathing software to control.

    For example, to select the desired device, perform an ls -l command on /dev/cfg directory. The following example shows the ls -l command output.


    lrwxrwxrwx  1 root  root  50 Jan 29 21:33 c0 -> 
     ../../devices/pci@7c0/pci@0/pci@1/pci@0/ide@8:scsi
lrwxrwxrwx  1 root  root  61 Jan 29 21:33 c1 ->
     ../../devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1:scsi
lrwxrwxrwx  1 root  root  61 Jan 29 21:33 c2 -> 
     ../../devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2:scsi
lrwxrwxrwx  1 root  root  53 Jan 29 21:33 c3 -> 
     ../../devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0:scsi
lrwxrwxrwx  1 root  root  54 Apr 16 20:28 c5 ->
     ../../devices/pci@780/pci@0/pci@8/SUNW,qlc@0/fp@0,0:fc
lrwxrwxrwx  1 root  root  56 Apr 16 20:28 c6 -> 
     ../../devices/pci@780/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0:fc
    Note –

    Controllers c5 and c6 are ports A and B on a dual-port FC HBA. Controllers c1 and c3 are single port SAS HBA ports. Controller c2 is the internal SAS controller in a Sun FireTM T2000 server.

    Once you have determined the port, or ports, you want to explicitly enable or disable multipathing for, go to the next step.

  3. Enable or disable specific ports as follows:

    • FC HBA ports

      1. Edit file /kernel/drv/fp.conf

      2. Add the following line for each FC HBA port that is to be enabled:

        name="fp" parent="parent name" port=port-number mpxio-disable="no";

        where parent name is the port device name, and port-number is the FC HBA port number.

        For example, the following entries disable multipathing on all FC HBA controller ports except for the two specified ports:

        mpxio-disable="yes";
name="fp" parent="/pci@6,2000/SUNW,qlc@2" port=0 mpxio-disable="no";
name="fp" parent="/pci@13,2000/pci@2/SUNW,qlc@5" port=0 mpxio-disable="no";

      3. Add the following line for each FC HBA port that is to be disabled:

        name="fp" parent="parent name" port=port-number mpxio-disable="yes";

        For example:

        name="fp" parent="/pci@6,2000/SUNW,qlc@2" port=0 mpxio-disable="yes";

  4. Type stmsboot -u to start the reboot and configuration process.

    You are prompted to reboot. During the reboot, the /etc/vfstab file and your dump device configuration are updated to reflect any device name changes.

  5. (Optional) After the reboot, if necessary, configure your applications to use new device names as described in Multipathing Considerations.

Configuring Third-Party Storage Devices

Note –

Before configuring any third-party device, ensure that they are supported. Refer to your third-party user documentation, or third-party vendor for information on proper vendor and product IDs, modes and various settings required for the device to work with multipathing software.

Third-Party Device Configuration Considerations

Before you configure third-party devices for multipathing, be aware of the following:

When multipathing is enabled, the multipath access still depends on a device specific scsi_vhci failover implementation accepting the device. The default way this occurs, automatically, is for the scsi_vhci code to call a probe function in each failover implementation, looking for the first probe result that indicates the device is supported.

A probe implementation determines support based on some combination of scsi_inquiry(9S) data. A device with INQUIRY data indicating T10 Target-Port-Group-Support (TPGS) compliance will use the standards-based TPGS failover implementation. For noncompliant devices, a failover implementation's probe will typically determine support based on VID/PID match against a private compiled-in table.

To override the probe process, the scsi_vhci.conf file supports a scsi-vhci-failover-override property. The value of scsi-vhci-failover-override can be used to establish support for a device not currently accepted by probe, override probe support, or disable multipath support for a device.

Configuring Third-Party Storage Devices: New Devices

Multipathing can be configured on third-party symmetric storage devices. A symmetric storage device is defined as a storage device in which all paths to the storage device are active and I/O commands can be issued through any path.

ProcedureTo Configure New Third-Party Devices

Perform the following steps to configure third-party devices if your system already has multipathing enabled. If your system has multipathing disabled, you can configure third-party devices while enabling multipathing as described earlier in this chapter.

  1. Add the vendor ID and product ID entries to the /kernel/drv/scsi_vhci.conf file.

    The vendor ID and product ID are the vendor and product identification strings that the device returns in SCSI INQUIRY data. The vendor ID must be eight characters long. You must specify all eight characters even if the trailing characters are spaces.

    The product ID can be up to 16 characters long.

    scsi-vhci-failover-override =
"VendorID1ProductID1", "f_sym",
"VendorID2ProductID2", "f_sym",
...
"VendorIDnProductIDn", "f_sym";

    Note that the entries are separated by the ’,’ character and the last vendor/product entry is terminated by the ’;’ character.

    For example, to add a device from a “ACME” vendor with a product ID of “MSU”, and to add a device from vendor “XYZ” vendor with a product ID of “ABC”, you would add the following lines to file /kernel/drv/scsi_vhci.conf:

    To add a device from a “XYZ” vendor with a product ID of “ABC”, you would add:

    scsi-vhci-failover-override =
"ACME    MSU", "f_sym",
"XYZ     ABC", "f_sym";

  2. Save and exit the scsi_vhci.conf file.

  3. Type stmsboot -u to start the reboot and configuration process.

    You are prompted to reboot. During the reboot, the /etc/vfstab file and the dump configuration are updated to reflect the device name changes.

  4. If necessary, perform device name updates as described in Enabling and Disabling Multipathing.

Configuring Third-Party Storage Devices: Disabling Devices

Multipathing can be disabled for all devices of a certain Vendor ID/Product ID combination from Inquiry data. This exclusion is specified in the scsi_vhci.conf file.

ProcedureTo Disable Third-Party Devices

  1. Add the vendor ID and product ID entries to the /kernel/drv/scsi_vhci.conf file.

    The vendor ID and product ID are the vendor and product identification strings that the device returns in SCSI INQUIRY data. The vendor ID must be eight characters long. You must specify all eight characters even if the trailing characters are spaces. The product ID can be up to 16 characters long.

    scsi-vhci-failover-override =
"VendorID1ProductID1", "NONE",
"VendorID2ProductID2", "NONE",
...
"VendorIDnProductIDn", "NONE";

    The entries in the preceding example are separated by the ’,’ character and the last vendor/product entry is terminated by the ’;’ character. For example, to add a device from vendor “ACME” with a product ID of “MSU,” and a vendor device from vendor “XYZ” with product ID “ABC,” you would add the following lines to the file /kernel/drv/scsi_vhci.conf:

    scsi-vhci-failover-override =
"ACME    MSU", "NONE",
"XYZ     ABC", "NONE";

  2. Save and exit the scsi_vhci.conf file.

  3. Type stmsboot -u to start the reboot and configuration process.

    You are prompted to reboot. During the reboot, the /etc/vfstab file and the dump configuration are updated to reflect the device name changes.

  4. If necessary, perform the device name updates as described in Enabling and Disabling Multipathing.

Displaying Device Name Changes

The stmsboot(1M) command, available on Solaris Express Developer's Edition OS SPARC, x86 and x64 platforms, can be used to display the mapping between non-multipathed and multipathed device names, after the changes made to the multipath configuration by invoking stmsboot with the -e, -d, or -u option. Both the non-multipathed and the multipathed device names must exist in order to show the mapping.

The -L option displays the mapping of devices on all controllers.


# stmsboot -L
non-STMS device name                    STMS device name
----------------------------------------------------------------
/dev/rdsk/c2t8d0     /dev/rdsk/c10t500000E01046DEE0d0
/dev/rdsk/c2t0d0     /dev/rdsk/c10t500000E01046B070d0
/dev/rdsk/c2t3d0     /dev/rdsk/c10t20000020372A40AFd0
/dev/rdsk/c2t12d0    /dev/rdsk/c10t500000E01046DEF0d0
/dev/rdsk/c2t11d0    /dev/rdsk/c10t500000E01046E390d0
/dev/rdsk/c3t8d0     /dev/rdsk/c10t500000E01046DEE0d0
/dev/rdsk/c3t0d0     /dev/rdsk/c10t500000E01046B070d0
/dev/rdsk/c3t3d0     /dev/rdsk/c10t20000020372A40AFd0
/dev/rdsk/c3t12d0    /dev/rdsk/c10t500000E01046DEF0d0
/dev/rdsk/c3t11d0    /dev/rdsk/c10t500000E01046E390d0

The -l option displays the mapping of devices on only the specified controller. The following example displays the mapping of controller 3.


# stmsboot -l3
non-STMS device name                    STMS device name
----------------------------------------------------------------
/dev/rdsk/c3t8d0    /dev/rdsk/c10t500000E01046DEE0d0
/dev/rdsk/c3t0d0    /dev/rdsk/c10t500000E01046B070d0
/dev/rdsk/c3t3d0    /dev/rdsk/c10t20000020372A40AFd0
/dev/rdsk/c3t12d0    /dev/rdsk/c10t500000E01046DEF0d0
/dev/rdsk/c3t11d0    /dev/rdsk/c10t500000E01046E390d0

Configuring Automatic Failback

Some storage devices have controllers configured as PRIMARY and SECONDARY as part of the array configuration. The secondary paths may operate at a lower performance level than the primary paths. The multipathing software uses the primary path to talk to the storage device and keeps the secondary path on standby. In the event of a primary path failure, the multipathing software automatically directs all I/O traffic over the secondary path, with the primary path taken offline. This process is called a “failover” operation. When the failure associated with the primary path has been repaired, the multipathing software automatically directs all I/O traffic over the primary path and keeps the secondary path standby as before. This process is called a “failback” operation.

You can disable the automatic failback operation so the multipathing software does not automatically failback to the primary path. Later, after the failure associated with the primary path has been repaired, you can do a manual failback operation using the luxadm(1M) command.

ProcedureTo Configure Automatic Failback

  1. Open the /kernel/drv/scsi_vhci.conf file in a text editor.

  2. To disable automatic failback capability, change the auto-failback entry to:


    auto-failback="disable";

    To enable automatic failback capability (if you disabled it before), change the auto-failback entry to:


    auto-failback="enable";

  3. Save and exit the file.

  4. Reboot the system.


    # shutdown -g0 -y -i6