17.7 About Device Multipathing

17.7.1 Configuring Multipathing

Multiple paths to storage devices can provide connection redundancy, failover capability, load balancing, and improved performance. Device-Mapper Multipath (DM-Multipath) is a multipathing tool that allows you to represent multiple I/O paths between a server and a storage device as a single path.

You would be most likely to configure multipathing with a system that can access storage on a Fibre Channel-based storage area network (SAN). You can also use multipathing on an iSCSI initiator if redundant network connections exist between the initiator and the target. However, Oracle VM does not support multipathing over iSCSI.

Figure 17.2 shows a simple DM-Multipath configuration where two I/O paths are configured between a server and a disk on a SAN-attached storage array:

Figure 17.2 DM-Multipath Mapping of Two Paths to a Disk over a SAN

The diagram shows a simple DM-Multipath configuration where two I/O paths are configured between a server and a disk on a SAN-attached storage array. DM-Multipath creates a single multipath device, /dev/mapper/mpathN, that subsumes the underlying devices, /dev/sdc and /dev/sdf.


Without DM-Multipath, the system treats each path as being separate even though it connects the server to the same storage device. DM-Multipath creates a single multipath device, /dev/mapper/mpathN, that subsumes the underlying devices, /dev/sdc and /dev/sdf.

You can configure the multipathing service (multipathd) to handle I/O from and to a multipathed device in one of the following ways:

Active/Active

I/O is distributed across all available paths, either by round-robin assignment or dynamic load-balancing.

Active/Passive (standby failover)

I/O uses only one path. If the active path fails, DM-Multipath switches I/O to a standby path. This is the default configuration.

Note

DM-Multipath can provide failover in the case of path failure, such as in a SAN fabric. Disk media failure must be handled by using either a software or hardware RAID solution.

17.7.1 Configuring Multipathing

The procedure in this section demonstrates how to set up a simple multipath configuration.

To configure multipathing on a server with access to SAN-attached storage:

  1. Install the device-mapper-multipath package:

    # yum install device-mapper-multipath
  2. You can now choose one of two configuration paths:

    • To set up a basic standby failover configuration without editing the /etc/multipath.conf configuration file, enter the following command:

      # mpathconf --enable --with_multipathd y --with_chkconfig y

      This command also starts the multipathd service and configures the service to start after system reboots.

      Skip the remaining steps of this procedure.

    • To edit /etc/multipath.conf and set up a more complex configuration such as active/active, follow the remaining steps in this procedure.

  3. Initialize the /etc/multipath.conf file:

    # mpathconf --enable
  4. Edit /etc/multipath.conf and define defaults, blacklist, blacklist_exceptions, multipaths, and devices sections as required, for example:

    defaults {
        udev_dir              /dev
        polling_interval      10
        path_selector         "round-robin 0"
        path_grouping_policy  multibus
        getuid_callout        "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
        prio                  alua
        path_checker          readsector0
        rr_min_io             100
        max_fds               8192
        rr_weight             priorities
        failback              immediate
        no_path_retry         fail
        user_friendly_names   yes
    }
    
    blacklist {
        # Blacklist by WWID
        wwid "*"
    
        # Blacklist by device name 
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
    
        # Blacklist by device type
        device {
          vendor    "COMPAQ  "
          product   "HSV110 (C)COMPAQ"
        }
    }
    
    blacklist_exceptions {
        wwid "3600508b4000156d700012000000b0000"
        wwid "360000970000292602744533032443941"
    }
    
    multipaths {
        multipath {
            wwid                  3600508b4000156d700012000000b0000
            alias                 blue
            path_grouping_policy  multibus
            path_checker          readsector0
            path_selector         "round-robin 0"
            failback              manual
            rr_weight             priorities
            no_path_retry         5
        }
        multipath {
            wwid                  360000970000292602744533032443941
            alias                 green
        }
    }
    
    devices {
        device {
            vendor                "SUN"
            product               "(StorEdge 3510|T4"
            path_grouping_policy  multibus
            getuid_callout        "/sbin/scsi_id --whitelisted --device=/dev/%n"
            path_selector         "round-robin 0"
            features              "0"
            hardware_handler      "0"
            path_checker          directio
            prio                  const
            rr_weight             uniform
            rr_min_io             1000
        }
    }

    The sections have the following purposes:

    defaults

    Defines default multipath settings, which can be overridden by settings in the devices section, and which in turn can be overridden by settings in the multipaths section.

    blacklist

    Defines devices that are excluded from multipath topology discovery. Blacklisted devices cannot subsumed by a multipath device.

    The example shows the three ways that you can use to exclude devices: by WWID (wwid), by device name (devnode), and by device type (device).

    blacklist_exceptions

    Defines devices that are included in multipath topology discovery, even if the devices are implicitly or explicitly listed in the blacklist section.

    multipaths

    Defines settings for a multipath device that is identified by its WWID.

    The alias attribute specifies the name of the multipath device as it will appear in /dev/mapper instead of a name based on either the WWID or the multipath group number.

    To obtain the WWID of a SCSI device, use the scsi_id command:

    # scsi_id --whitelisted --replace-whitespace --device=device_name
    devices

    Defines settings for individual types of storage controller. Each controller type is identified by the vendor, product, and optional revision settings, which must match the information in sysfs for the device.

    You can find details of the storage arrays that DM-Multipath supports and their default configuration values in /usr/share/doc/device-mapper-multipath-version/multipath.conf.defaults, which you can use as the basis for entries in /etc/multipath.conf.

    To add a storage device that DM-Multipath does not list as being supported, obtain the vendor, product, and revision information from the vendor, model, and rev files under /sys/block/device_name/device.

    The following entries in /etc/multipath.conf would be appropriate for setting up active/passive multipathing to an iSCSI LUN with the specified WWID.

    defaults {
        user_friendly_names    yes
        getuid_callout         "/bin/scsi_id --whitelisted --replace-whitespace --device=/dev/%n”
    }
    
    multipaths {
        multipath {
            wwid 360000970000292602744533030303730
        }
    }

    In this standby failover configuration, I/O continues through a remaining active network interface if a network interfaces fails on the iSCSI initiator.

    For more information about configuring entries in /etc/multipath.conf, refer to the multipath.conf(5) manual page.

  5. Start the multipathd service and configure the service to start after system reboots:

    # service multipathd start
    # chkconfig multipathd on

Multipath devices are identified in /dev/mapper by their World Wide Identifier (WWID), which is globally unique. Alternatively, if you set the value of user_friendly_names to yes in the defaults section of /etc/multipath.conf or by specifying the --user_friendly_names n option to mpathconf, the device is named mpathN where N is the multipath group number. An alias attribute in the multipaths section of /etc/multipath.conf specifies the name of the multipath device instead of a name based on either the WWID or the multipath group number.

You can use the multipath device in /dev/mapper to reference the storage in the same way as you would any other physical storage device. For example, you can configure it as an LVM physical volume, file system, swap partition, Automatic Storage Management (ASM) disk, or raw device.

To display the status of DM-Multipath, use the mpathconf command, for example:

# mpathconf
multipath is enabled
find_multipaths is enabled
user_friendly_names is enabled
dm_multipath modules is loaded
multipathd is chkconfiged on

To display the current multipath configuration, specify the -ll option to the multipath command, for example:

# multipath -ll
mpath1(360000970000292602744533030303730) dm-0 SUN,(StorEdge 3510|T4
size=20G features=‘0’ hwhandler=‘0’ wp=rw
|-+- policy=‘round-robin 0’ prio=1 status=active
| ‘- 5:0:0:2 sdb 8:16    active ready running
‘-+- policy=‘round-robin 0’ prio=1 status=active
  ‘- 5:0:0:3 sdc 8:32    active ready running

In this example, /dev/mapper/mpath1 subsumes two paths (/dev/sdb and /dev/sdc) to 20 GB of storage in an active/active configuration using round-robin I/O path selection. The WWID that identifies the storage is 360000970000292602744533030303730 and the name of the multipath device under sysfs is dm-0.

If you edit /etc/multipath.conf, restart the multipathd service to make it re-read the file:

# service multipathd restart

For more information, see the mpathconf(8), multipath(8), multipathd(8), multipath.conf(5), and scsi_id(8) manual pages.