C H A P T E R  5

SAS Multipathing

You can use J4200/J4400 arrays in a serial-attached SCSI (SAS) multipathing configuration. Multipathing provides higher availability by allowing data to use multiple paths to the arrays from Host Bus Adaptors (HBAs) on the same server or on different servers.

Multipathing capability resides in the operating systems running on the servers, not on the J4200/J4400 arrays. You enable, disable, and configure multipathing through the operating software.

This chapter includes the following information:


System Requirements

To use multipathing with the J4200/J4400 arrays, your configuration must meet the requirements specified in this section.

Required Operating Systems

TABLE 5-1 lists the operating systems that support multipathing with the J4200/J4400 arrays


TABLE 5-1 Operating Systems That Support J4200/J4400 Multipathing

Operating System

Versions That Support Multipathing With J4200/J4400 Arrays

Microsoft Windows

 

Windows 2008 (32/64 bit)

Solaristrademark Operating System

Solaris 10, update 6

 

Linux Operating System

Linux RHEL 4, version 4.5 and higher

Linux RHEL 5, version 5.2 and higher

Linux SUSE 9, SP 3 and higher

Linux SUSE 10, SP 2 and higher


Required Patch for Solaris 10 U6 With SATA Drives

To use the Solaris 10, update 6, operating system with multipathing and SATA drives, you must use a patch for the operating system. The patch ID is 138888-03 5.10 (SPARC) and 138889-03 5.10_x86 (x86).



Note - These patches are required only for multipathing with SATA drives. Multipathing with SAS drives and Solaris 10, U6 requires no patches.


Required Firmware and Drivers

To use multipathing with the J4200/J4400 arrays, the firmware in your configuration must meet these requirements:

For each operating system, TABLE 5-2 lists the required drivers.

 


TABLE 5-2 Required Drivers By Operating System

Operating System

Required Driver(s)

Windows 2008

Included with the software

Solaris 10, update 6

Included with software

Linux RHEL 4, version 4.5 and higher

MPT 3.16.00.00

Linux RHEL 5, version 5.2 and higher

MPT 4.16.00.00

Linux SUSE 9, SP 3 and higher

MPT 3.16.00.00

Linux SUSE 10, SP 2 and higher

MPT 4.16.00.00


Where to Get Drivers

For the Linux MPT drivers, go to the following location:

http://www.lsi.com/support/sun/sg_xpci8sas_e_sRoHS.html

Required HBA

To use multipathing with the J4200/J4400 arrays, your servers must be using the following HBA:

Required CAM Versions for Multipathing

For the latest information about which versions of the Sun StorageTek Common Array Manager (CAM) you must use in a multipathing configuration, refer to the Sun Storage J4200/J4400 Array Release Notes, part number 820-3222-xx, which is available at the following location:

http://docs.sun.com/app/docs/prod/j4200.array


Configuration Examples

This section includes examples of how to cable J4200/J4400 arrays to each other and to host HBAs in a multipathing environment.

Multipathing With One Array and One Host

FIGURE 5-1 shows an example of SAS cables connecting the ports on the host’s HBA to the SIM boards on the back of an array.

FIGURE 5-1 Cabling Example for Multipathing With One Array and One Host


Shows a cable connecting each port on an HBA on the backof a host to one of the Host or SIM Link In ports on an array.


Figure Legend

1

Host or SIM Link In

2

Host Out

3

SIM Link Out


 

Multipathing With Two Arrays and Two Hosts

FIGURE 5-2 shows an example of cabling for multipath in a configuration with two hosts, with two HBAs on each host, and two arrays cabled together.

FIGURE 5-2 Cabling Example for Multipathing With Two Arrays and Two Hosts


Shows two hosts, with two HBAs each, cabled to two different arrays.

 

Figure Legend

1

Host or SIM Link In

2

Host Out

3

SIM Link Out


Multipathing With Two Arrays and One Host

FIGURE 5-3 shows an example of cabling for multipath in a configuration with one host, with two HBAs on that host, and two arrays cabled together.

FIGURE 5-3 Cabling Example for Multipathing With Two Arrays and One Host


Shows one host, with two HBAs each, cabled to two different arrays.

 

Figure Legend

1

Host or SIM Link In

2

Host Out

3

SIM Link Out


 


Enabling and Disabling Multipathing in the Solaris Operating System

This sections describes how to enable and disable multipathing in the Solaris 10, update 6, operating system.

Use the stmsboot command to enable multipathing in the Solaris 10, update 6, operating system. stmsboot manages enumeration of multipath-capable devices with multipathing; it has these characteristics:

This section describes the stmsboot(1M) administration program for multipathing.

Device Renaming

In the /dev and /devices trees, multipathing-enabled devices receive new names that indicate that they are under multipathing control.

stmsboot Options

TABLE 5-3 describes the options supported by stmsboot(1M).


TABLE 5-3 stmsboot Options

-e [-D fp | mpt]

Enables multipathing on all supported multipath-capable controller ports.

  • Multipath-capable ports include SAS (mpt[7D]) controller ports.
  • Prompts you to reboot.
  • During reboot, vfstab and the dump configuration update to reflect the device-name changes.
  • Specifying -D mpt limits the enabling operation to ports attached using the specified driver.

-d [-D fp | mpt]

Disables multipathing on all supported multipath-capable controller ports.

  • Multipath-capable ports include SAS (mpt[7D]) controller ports.
  • Prompts you to reboot.
  • During reboot, vfstab and the dump configuration update to reflect the device name changes.
  • Specifying -D mpt limits the disabling operation to ports attached using the specified driver.

-u

Updates vfstab and the dump configuration after manually enabling or disabling multipathing on specific multipath- capable controller ports.

  • Prompts you to reboot.
  • During reboot, vfstab and the dump configuration update to reflect the device name changes.

-L

Displays the device name changes from non-multipathing-device names to multipathing device names on multipath-enabled controller ports.

If multipathing is not enabled, no mappings display.

-l controller_number

Displays the device-name changes from non-multipathing device names to multipathing device names for the specified controller.

If multipathing is not enabled, no mappings display.


stmsboot Conditions

stmsboot enables and disables multipathing on the host, with these conditions:



Note - The system administrator is responsible for modifying application configuration (for example, backup software, DBMS, and so forth) to reflect updated device names.



procedure icon  To Enable Multipathing on all Multipath-Capable Controllers



Note - Multipathing is not supported on all controllers. After enabling multipathing, only supported controllers are placed under multipathing control. Non-supported controllers remain unchanged.


single-step bullet  Type:

# stmsboot -e



# stmsboot -e
Warning: stmsboot operates on each supported multipath-capable controller detected in a host. In your system, these controllers are
	/devices/pci@780/pci@0/pci@8/SUNW,qlc@0/fp@0,0
	/devices/pci@780/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
	/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2
	/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0
	/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0
 
If you do NOT wish to operate on these controllers, please quit stmsboot and re-invoke with -D { fp | mpt } to specify which controllers you wish to modify your multipathing configuration for.
 
Do you wish to continue? [y/n] (default: y) y
Checking mpxio status for driver fp
Checking mpxio status for driver mpt
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y
 



Note - stmsboot updates the /etc/vfstab file and dump configuration to reflect device-name changes during reboot.


The following conditions apply to the stmsboot -e, -d, and -u options:


procedure icon  To Disable Multipathing on All Multipath-Capable Controllers

single-step bullet  Type:

# stmsboot -d



# stmsboot -d 
WARNING: stmsboot operates on each supported multipath-capable controller detected in a host. In your system, these controllers are
/devices/pci@780/pci@0/pci@8/SUNW,qlc@0/fp@0,0
/devices/pci@780/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0
/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@1
/devices/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2
/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0
/devices/pci@7c0/pci@0/pci@9/LSILogic,sas@0
 
If you do NOT wish to operate on these controllers, please quit stmsboot and re-invoke with -D { fp | mpt } to specify which controllers you wish to modify your multipathing configuration for.
 
Do you wish to continue? [y/n] (default: y) y
Checking mpxio status for driver fp
Checking mpxio status for driver mpt
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y
 



Note - During reboot, /etc/vfstab and the dump configuration update to reflect the device-name changes.



procedure icon  To Enable Multipathing on Multipath-Capable Controller Ports

single-step bullet  Type:

# stmsboot -D mpt -e


procedure icon  To Disable Multipathing on Multipath-Capable Controller Ports

single-step bullet  Type:

# stmsboot -D mpt -d

Configuring Multipathing on Selected Ports

To enable multipathing on specific ports and disable multipathing on others, edit the /kernel/drv/mpt.conf file. To update vfstab(4) and dumpadm(1M) configurations to reflect the changed device names, type:

# stmsboot -u

Dynamic Discovery of SAS Devices

SAS devices, added and removed dynamically with the mpt(7d) driver, enable your system to detect specific targets and LUNs attached, with these conditions:



Note - To clean up the CTD# for your device tables, run the following command: devfsadm -C.



Enabling and Disabling Multipathing in the Windows Operating System

This section describes how to enable and disable multipathing in the
Windows 2008 operating system.


procedure icon  To Enable Multipathing in Windows 2008

1. From the Start menu, select Server Manager, as shown in FIGURE 5-4.

The Server Manager opens.

FIGURE 5-4 Start Menu With Server Manager Selection


Screen showing Start menu opened and Server Manager selected.

2. Select Features from the list of folders on the left to open the Features panel, as shown in FIGURE 5-5.

FIGURE 5-5 Server Manager Feature Display


Screen showing Features Summary for Server Manager.

In the Features panel, you can click the Features Summary arrow to open or close the Features Summary and click the Features arrow to view the status of installed features.

3. Click Add Features to enable the Add Features Wizard.

The Add Features Wizard Select Features window opens with the default Features selected and a list of optional features available for installation in your system, as shown in FIGURE 5-6.

FIGURE 5-6 Add Features Wizard Select Features Window


Screen showing wizard for selecting features.

4. Select the Multipath I/O option from the Message Queuing sublist.

5. Click Next.

The Add Features Wizard Confirm Installation Selections window opens, as shown in FIGURE 5-7.

FIGURE 5-7 Confirm Installation Selections Window


Screen showing confirmation screen for installation selections.

6. Click Install.

The Add Features Wizard Installation Results window opens as shown in FIGURE 5-8.

FIGURE 5-8 Add Features Wizard Installation Results Window


Screen showing results of installation.

7. Click Close.

8. Close the Server Manager main window.


procedure icon  To Discover Device Multipaths

1. From the Windows Start menu (FIGURE 5-4), select Administrative Tools.

The MPIO Properties window, as shown in FIGURE 5-9..

FIGURE 5-9 MPIO Properties Window


Screen showing MPIO Properties display, with list of devices.

2. Select the Discover Multi-Paths tab, as shown in FIGURE 5-10.

FIGURE 5-10 MPIO Properties Window Device Multi-Paths Tab


Screen showing Device Multipaths tab.

3. Highlight the Device Hardware for which you want to discover multipaths, as shown in FIGURE 5-11.

FIGURE 5-11 Highlighted Device Hardware for Discovering Multipaths


Screen showing devices highlighted for discovering multipaths.

4. Click OK.

The Reboot Required window opens, as shown in FIGURE 5-12.

FIGURE 5-12 Reboot Required Window


Screen asking if system should be rebooted.

5. Click Yes.



Note - For best results, after reboot, log in and change the default settings of the MPIO policies. See To Select a Load Balancing Policy.


 


procedure icon  To Select a Load Balancing Policy

1. From the Windows Start menu, select Server Manager, as shown in FIGURE 5-4.

2. Select Diagnostics > Device Manager from the list of folders on the left to open the Device Manager window, as shown in FIGURE 5-13.

FIGURE 5-13 Device Manager Window


Screen showing Device Manager window with list of disk drivess opened.

3. Click the Device name.

4. Click Disk drives.

The list of disk drives opens, as shown in FIGURE 5-13.

5. Move your cursor over the device drive name for which you want information.

6. Right-click to open a menu and select Properties, as shown in FIGURE 5-14.

FIGURE 5-14 Menu to Modify Device Configuration


Screen showing menu for modifying device configuration.

The Disk Drive Properties window opens, as shown in FIGURE 5-15.

FIGURE 5-15 Disk Drive Properties Window


Screen showing Device Properties window.

7. Use the drop-down menu to select a load-balancing policy, as shown in FIGURE 5-16.

The choices for load-balancing policies are listed in TABLE 5-4. By default, the Fail Over Policy is configured. For best results with the J4200/J4400 array, use the Round Robin policy.


TABLE 5-4 Load Balance Policies

Load Balance Policy

Description

Fail Over Only

The failover policy employs one active/optimized path and all other paths are active/unoptimized, standby, or unavailable. The non-active/optimized paths will be tried round-robin upon failure of the active/optimized path until an available path is found.

Round Robin

The round robin policy attempts to evenly distribute incoming requests to all processing paths.

Round Robin with Subset

The round robin with subset policy executes the round robin policy only on paths designated as active/optimized. The non-active/optimized paths will be tried on a round-robin approach upon failure of all active/optimized paths.

Least Queue Depth

The least queue depth policy compensates for uneven loads by distributing proportionately more I/O requests to lightly loaded processing paths.

Weight Paths

The weighted paths policy allows the user to specify the relative processing load of each path. A large number means that the path priority is low


FIGURE 5-16 Round Robin Load Balance Policy Selected


Screen showing Round Robin load balance policy selected.

8. Click OK.



Note - Load balance policie are set on a per-device basis. Repeat this procedure for each disk drive.



procedure icon  To Disable Multipathing in Windows 2008

1. From the Windows Start menu, select Administrative Tools > MPIO, as shown in FIGURE 5-17.

The MPIO Properties window opens, as shown in FIGURE 5-18.

FIGURE 5-17 Selecting Administrative Tools > MPIO From the Start Menu


Screen showing selection of MPIO from Administrative Tools on the Start Menu

 

FIGURE 5-18 MPIO Properties Window


Screen showing MPIO Properties window, listing MPIO-ed devices.

2. Highlight a device in the list, as shown in FIGURE 5-19.

 

FIGURE 5-19 Highlighting Device For Which to Disable Multipathing


Screen showing one device in the hardare Id list highlighted.

3. Click Remove.

The Reboot Required window opens, as shown in FIGURE 5-20.

FIGURE 5-20 Reboot Required Window


Screen showing system asking if you want to reboot now.

4. Click Yes.

The system reboots.

5. After reboot, select Administrative Tools > MPIO from the Windows Start menu.

The MPIO Properties window opens, showing the device removed from the list, as shown in FIGURE 5-21.

 

 

FIGURE 5-21 MPIO Properties Window With Device Removed From List


Screen showing list of hardware Ids, with selected device removed.

6. Repeat Step 2 through Step 5 until all devices are removed from the Device Hardware Id list.

7. From the Windows Start menu, select Server Manager to open the Server Manager window.

8. Select Features from the list of folders on the left to open the Features panel, as shown in FIGURE 5-22.

 

 

 

FIGURE 5-22 Server Manager Window With Features Panel Open


Screen showing Server Manager window.

9. Select Remove Features.

The Remove Features Wizard window opens, as shown in FIGURE 5-23.

 

FIGURE 5-23 Remove Features Wizard Window


Screen showing wizard that lets you select feaures to disable.

10. Highlight Multipath I/O from the Message Queuing sublist.

The Confirm Removal Selections window opens, as shown in FIGURE 5-24.

FIGURE 5-24 Confirm Removal Selections Window


Screen showing system asking you to confirm removal of multipath I/O.

11. Click Remove.

The Removal Results window opens, as shown in FIGURE 5-25.

FIGURE 5-25 Removal Results Window


Screen showing that multipath I/O was removed.

12. Click Close.

The system asks if you want to restart the server now, as shown in FIGURE 5-26.

 

FIGURE 5-26 Restart Now Window


Screen showing system asking if you want to restart now.

13. Click Yes.

When the sever has restarted, the Removal Results window opens, as shown in FIGURE 5-27.

FIGURE 5-27 Removal Results Window


Screen showing that multipath I/O was successfully removed.

 


Enabling and Disabling Multipathing in the Linux Operating System

This section describes how to enable and disable multipathing in supported versions of the Linux operating system.


procedure icon  To Enable Multipathing in Linux

1. Attach a J4200 or a J4400 to a server with a supported version of Linux installed.

2. On the server, edit or create the /etc/multipath.conf file.

3. Reboot the server.

4. After the reboot, make sure that the OS discovers all the disks in the J4200/J4400 arrays either by using the Linux commands, fdisk or lsscsi.

5. Partition any disks you want to the desired sizes.

6. Use the Linux command modprobe to add the loadable kernel modules dm-multipath and dm-round-robin.

# modprobe dm-multipath

# modprobe dm-round-robin

7. Start the multipathd daemon.

For Linux SUSE 9, use the following command:

# multipathd -v0

For other supported Linux versions, use the following command:

# service multipathd start

8. Start the multipathing device mapper target autoconfig.

# multipath -v2

9. List the multipath devices that have been created.

# multipath -ll

The output should list the same number of devices as there are disks in the J4200/J4400 arrays; it will look similar to the following:


35000c5000357625bdm-2 SEAGATE,ST340008SSUN0.4

[size=373G][features=0][hwhandler=0]

\_ round-robin 0 [prio=2][active]

\_ 0:0:0:0 sda 8:0 [active][ready]

\_ 1:0:0:0 sdm 8:192 [active][ready]



procedure icon  To Disable Multipathing in Linux

1. If a RAID volume, LVM volume, or volume mount have been placed over the device node of the multipathed disk, quiesce the volume.

2. Use the multipath -f command to disable multipathing to a specific device.

multipath -f mpath1

3. Use the multipath -F command to disable multipathing on all multipathed devices.

multipath -F



Note - If the message map in use appears for a device when you attempt to disable multipathing, the device is still in use. You must unmount or otherwise quiesce the device before you can disable multipathing. If you cannot quiesce the device, edit the /etc/multipath.conf file to exclude the device and then reboot the server.