test

IP Network Multipathing Administration Guide

Chapter 1 Overview of IP Network Multipathing

IP Network Multipathing provides both load spreading and failover when you have multiple network interface cards connected to the same IP link (for example, Ethernet).

Introduction

IP Network Multipathing provides your system with the following capabilities:

If a failure occurs in the network adapter, and if you have an alternate adapter connected to the same IP link, the system switches all the network accesses automatically from the failed adapter to the alternate adapter. This process ensures uninterrupted access to the network. Also, when you have multiple network adapters connected to the same IP link, you achieve increased traffic throughput by spreading the traffic across multiple network adapters.

Note –

Other IP related documents, such as RFC 2460, use the term link instead of IP link. This document uses the term IP link to avoid confusion with IEEE 802. In IEEE 802, link refers to a single wire from an Ethernet NIC to an Ethernet switch.)

See IP link definition in the Glossary or refer to Table 1–1.

IP Network Multipathing Features

The Solaris implementation of IP Network Multipathing provides the following features:

Communication Failures

Communication failures can occur in the following ways:

  1. Transmit/receive path of the NIC can stop transmitting packets.

  2. Attachment of the NIC to the link is down.

  3. Port on the switch does not transmit/receive packets.

  4. Physical interface in a group not present at system boot.

  5. Host on the other end is not responding or the router that is forwarding the packets is not responding.

The Solaris implementation of IP Network Multipathing addresses the first four types of communication failures.

IP Network Multipathing Components

The following table identifies and describes the components that make up IP Network Multipathing.

Table 1–1 IP Network Multipathing Components

Component 

Description 

IP Link 

A communication facility or medium over which nodes can communicate at the link layer. The link layer is the layer immediately below IPv4/IPv6. Examples include Ethernets (simple or bridged) or ATM networks. One or more IPv4 subnet numbers/prefixes are assigned to an IP link. A subnet number/prefix can not be assigned to more than one IP link. In ATM LANE, an IP link is a single emulated LAN. When using ARP, the scope of the ARP protocol is a single IP link. 

Network Interface Card (NIC) 

Network adapter which is either internal or a separate card that serves as an interface to a link. 

Physical interface 

A node's attachment to a link. This attachment is often implemented as a device driver plus a network adapter. Some network adapters can have multiple points of attachment, for example, qfe. The usage of Network adapter in this document refers to a "Single Point of Attachment."

Physical interface group 

The set of physical interfaces on a system that are connected to the same link. They are identified by assigning the same (non-null) character string name to all the physical interfaces in the group. 

Physical interface group name 

A name assigned to a physical interface that identifies the group. The name is local to a system. Multiple physical interfaces, sharing the same group name, form a physical interface group. 

Failure detection 

The process of detecting when a NIC or the path from the NIC to some layer 3 device no longer works. 

Repair detection 

The process of detecting when a NIC or the path from the NIC to some layer 3 device starts operating correctly after a failure. 

Failover 

The process of switching network access from a failed interface to a good physical interface. Network access includes IPv4 unicast, multicast, and broadcast traffic, as well as IPv6 unicast and multicast traffic. 

Failback 

The process of switching back network access to an interface detected as having been repaired. 

Standby Interface 

A physical interface that is not used to carry data traffic unless some other physical interface in the group has failed.  

Solaris Network Multipathing

The following components implement Solaris network multipathing:

The in.mpathd daemon detects failures and implements various policies for failover and failback. After in.mpathd detects a failure or repair, in.mpathd sends an ioctl to do the failover or failback. IP, which implements the ioctl, does the network access failover transparently and automatically.

Caution – Caution –

You should not use Alternate Pathing while using IP Network Multipathing on the same set of NICs. Likewise, you should not use IP Network Multipathing while using Alternate Pathing. You can use Alternate Pathing and IP Network Multipathing at the same time on different sets of NICs.

Detecting Physical Interface Failures

The in.mpathd daemon sends ICMP echo probes to the targets connected to the link on all the interfaces that belong to a group to detect failures and repair. After you add an interface to a multipathing group and assign a test address, the daemon sends probes to detect failures on all the interfaces of the multipathing group. How to Configure a Multipathing Interface Group with Two Interfaces describes the steps you perform to configure test address and groups.

Because in.mpathd determines what targets to probe dynamically, you cannot configure the targets. Routers connected to the link are chosen as targets for probing. If no routers exist on the link, arbitrary hosts on the link are chosen. A multicast packet sent to the “all hosts” multicast address (224.0.0.1 in IPv4 and ff02::1 in IPv6) determines the arbitrary hosts. The first few hosts that respond to the echo packets are chosen as targets for probing. If in.mpathd cannot find routers or hosts that responded to ICMP echo packets, in.mpathd cannot detect failures.

To ensure that each NIC in the group functions properly, in.mpathd probes all the targets separately through all the interfaces in the multipathing group. If there are no replies to five consecutive probes, in.mpathd considers the interface as having failed. The probing rate depends on the failure detection time (FDT). The default value for failure detection time is 10 seconds. The in.mpathd(1M) man page describes how to change the failure detection time. For a failure detection time of 10 seconds, the probing rate is approximately one probe every two seconds.

If five consecutive probes fail, in.mpathd considers the interface as having failed. After a failure is detected, failover of all network access takes place from the failed interface to another functional interface in the group. If you have configured a standby interface, in.mpathd chooses the standby interface for failover of IP addresses, broadcasts, and multicast memberships. If you have not configured a standby interface, in.mpathd chooses the interface with the least number of IP addresses.

Physical interfaces in the same group that are not present at system boot represent a special case of failure detection. The startup script /etc/init.d/network detects these types of failure. This type of failure displays error messages similar to the following:


moving addresses from failed IPv4 interfaces: hme0 (moved to hme1)
moving addresses from failed IPv6 interfaces: hme0 (moved to hme1)

Currently, this type of failure cannot be automatically repaired by a failback. See How to Recover a Physical Interface That Was Not Present at System Boot.

Detecting Physical Interface Repairs

To detect if a physical interface has been repaired, in.mpathd sends probes through the failed NIC. When an interface fails, all addresses are moved to another functional interface in the group. Because in.mpathd needs an address for probing so that it can detect repairs, you must configure a test IP address that will not move during the failover. Moreover, you should not allow a normal application to use this test address, because the failover of network access will not take place for these addresses. How to Configure a Multipathing Interface Group with Two Interfaces describes the steps that you perform. in.mpathd considers the interface repaired if it receives responses to 10 consecutive probe packets. Then, if in.mpathd detects a failure, failback of all network access takes place to the repaired interface.

As noted in Detecting Physical Interface Failures, automatic failback is not supported for physical interfaces that are not present at system boot. See How to Recover a Physical Interface That Was Not Present at System Boot.

Link Failures

If all the NICs appear to fail at the same time, it is considered a link failure and in.mpathd does not do any failovers. This is also true when all the targets fail at the same time. In this case, in.mpathd flushes all of its current targets and discovers new targets (see Detecting Physical Interface Failures).

Administering Multipathing Groups with Multiple Physical Interfaces

This section describes how you enable IP Network Multipathing. To use the IP Network Multipathing feature, you should have more than one physical interface connected to the same IP link. For example, the same Ethernet switch or the same IP subnet, configured under the same multipathing group, works. If you have just one physical interface, refer to Administering Multipathing Groups With a Single Physical Interface.

Multipathing groups are identified by non-null names. For example, math-link, bio-link, and chem-link, make good names. The names typically represent where these groups are connected. When failure is detected in one of the network adapters in the multipathing group, all network access is failed over from the failed adapter to the good adapter in the group. The failover of network access includes IPv4 unicast, broadcast, and multicast traffic, as well as IPv6 unicast and multicast traffic. For IP network multipathing to function properly, the following conditions must exist for the network adapters that are part of the same multipathing group:

  1. You must push and configure the same set of STREAMS modules on all network adapters in the multipathing group.

  2. If you have plumbed IPv4 on one network adapter, then you must plumb IPv4 on all network adapters in the multipathing group.

  3. If you have plumbed IPv6 on one network adapter, then you must plumb IPv6 on all network adapters in the multipathing group.

  4. All Ethernet network adapters in the system should have unique MAC address in the case of ethernets. This is achieved by setting local-mac-address? to true in the openboot PROM for SPARC platforms. Nothing needs to be done for IA (x86) platforms.

  5. All network adapters of the multipathing group must be connected to the same IP link.

  6. The multipathing group should not contain dissimilar interfaces. The interfaces that are grouped together should be of the same interface type that is defined in /usr/include/net/if_types.h. For example, you cannot combine Ethernet with Token ring, and you cannot combine a Token bus with ATM (asynchronous transfer mode).

  7. When you use IP network multipathing with ATMs, you must configure the ATM for LAN emulation (multipathing over classical IP instances is not currently supported).

Note –

The fourth condition concerns all interfaces in the system, not just those belonging to the multipathing group.

For the adapters which do not come with factory set unique mac addresses, you can manually configure a mac address for each adapter as a work around. For example, use the ifconfig ether command in a start script file.

Note –

The mac addresses configured manually will not be maintained across system reboot. You are responsible for choosing unique mac addresses. IP Network Multipathing might behave unpredictably if the mac address of adapters are not unique.

Grouping Physical Interfaces

You use the ifconfig command to configure groups. This command uses a new group parameter that requires a group name and places both the IPv4 and IPv6 instance of the interface in that group. The group parameter has the following syntax:


ifconfig interface-name group group-name
Note –

Avoid using spaces in group names. The ifconfig status display does not show spaces. Consequently, if you have created two similar group names, but one of them contains a space, these group names look alike in the status display. However, they are different group names. This might be confusing.

Placing the IPv4 instance under a particular group automatically places the IPv6 instance under the same group. Also, you can place a second interface, connected to the same subnet, in the same group by using the same command (see How to Configure a Multipathing Interface Group with Two Interfaces).

You can remove an interface from a multipathing group by using a null string with the group sub-command (see How to Remove an Interface From a Group).

To place an interface in a new group when it is already part of some multipathing group, you do not need to remove it from any existing group. Placing the interface in a new group automatically removes it from any existing group (see How to Move an Interface From an Existing Group to a Different Group).

You can have any number of network adapters that you can configure in the same multipathing group. You cannot use the group parameter with logical interfaces. For example, you can use it with hme0, but not with hme0:1.

You must connect all the interfaces in the multipathing group to the same IP link, because when an interface fails, the failover operation moves all the IP addresses from the failed interface to an interface in the group that is functional. For routers to continue routing packets to the addresses that have been switched to the functional interface, the functional interface must be connected to the same IP link.

Configuring Test Addresses

You must configure all physical interfaces of a multipathing group with a test address. You need test addresses to detect failures and repairs. If a test address is not configured, it is not chosen for failover. Only in.mpathd uses test addresses. Normal applications should not use this address. This address will not failover when the interface fails. In IPv4, you should configure the test address in such a way that normal applications do not use the test address (See How to Configure a Multipathing Interface Group with Two Interfaces).

This section describes test address configuration concepts for the following Internet protocols:

IPv4 Test Addresses

The in.mpathd multipathing daemon requires a test IP address for detecting failures and repairs. You must use a routeable address for this IP address. That is, the subnet prefix of the address must be known to any routers present on the link. You use the ifconfig command's new -failover option to configure a test address. Use the following syntax to configure a test address:


# ifconfig interface-name addif ip-address <other-parameters> -failover up

For <other-parameters>, use the parameters required by your configuration. See the ifconfig(1M) man page for descriptions. How to Configure a Multipathing Interface Group with Two Interfaces shows the steps you perform for an IPv4 test address.

For example, to add a new logical interface with an address of 19.16.85.21, the netmask and broadcast address set to the default value, and also configure the interface with a test address, type the following:


# ifconfig hme0 addif 19.16.85.21 netmask + broadcast + -failover up
Note –

You must mark an IPv4 test address as deprecated to prevent applications from using the test address (see How to Configure a Multipathing Interface Group with Two Interfaces).

Use failover without the dash to turn on the failover attribute of the address.

Note –

All test IP addresses in a multipathing group must use the same network prefix. That is, the test IP addresses must belong to a single IP subnet.

IPv6 Test Addresses

To configure an IPv6 test address, you use the link-local address itself, because link-local addresses are tied to the physical interface. Thus, you do not need a separate IP address in the IPv6 case. For IPv6, the -failover option has the following syntax:


# ifconfig interface-name inet6 -failover

How to Configure a Multipathing Interface Group with Two Interfaces shows the steps you perform for an IPv6 test address.

When a multipathing group has both IPv4 and IPv6 plumbed on all the group's interfaces, you might not need a separate IPv4 test address. The in.mpathd daemon can probe the interfaces using an IPv6 link-local address. IPv6 link-local addresses are created when IPv6 is plumbed.

Use failover without the dash to turn on the failover attribute of the address.

Note –

The only valid IPv6 test address is the link-local address.

Preventing Applications From Using Test Addresses

After you have configured a test address, you need to ensure that this address is not used by normal applications. If you let applications use the test address, applications will fail, because test addresses do not failover during the failover operation. To ensure that IP does not pick the test address for normal applications, you mark the test address deprecated by using the ifconfig command. This parameter has the following syntax:


ifconfig interface-name deprecated

After you mark the address as deprecated, IP does not pick this address as a source address for any communication, unless the applications explicitly bind to the address. Only in.mpathd explicitly binds to such an address (see How to Configure a Multipathing Interface Group with Two Interfaces).

Because link-local addresses are not present in the name service (DNS, NIS, and NIS+), applications do not use link-local addresses for communication. Consequently, you do not need to mark IPv6 test addresses as deprecated.

Note –

You must not mark IPv6 link-local addresses as deprecated.

Use the -deprecated option to turn off the deprecated attribute of the address.

Note –

IPv4 test addresses should not be placed in the name service tables (DNS/NIS/NIS+). In IPv6, link-local addresses are used as test addresses and are not normally placed in the name service tables.

Using the hostname File to Configure Groups and Test Addresses

You can use the /etc/hostname.interface files to configure multipathing groups and test address. To configure a multipathing group using the /etc/hostname.interface file, you can add a line to the file using the following syntax:


interface-address <parameters>  group group-name up \
addif logical-interface-address <parameters> up

For example, to create the group test with the following configuration:

You add the following line to the /etc/hostname.hme0 file:


19.16.85.19 netmask + broadcast + group test up \
addif 19.16.85.21 deprecated -failover netmask + broadcast + up

How to Configure a Multipathing Interface Group with Two Interfaces shows the steps you perform to configure the IPv4 hostname file.

For IPv6 setup, add a line to the /etc/hostname6.interface file using the following syntax:


<parameter> group group-name up

For example, to create a test group for hme0 with an IPv6 test address, add the following line to the /etc/hostname6.hme0 file:


-failover group test up

How to Configure a Multipathing Interface Group with Two Interfaces shows the steps you perform to configure the IPv6 hostname6 file.

Configuring Standby Interfaces

You can configure multipathing groups with standby interfaces. As the name implies, the interface is considered as standby and is not used unless some other interface in the group fails.

The standby interface is not used to send normal data packets. Consequently, limited traffic flows on a standby interface. You must configure standby interfaces with a test address to insure that probes are sent to determine if the interface is functional. If you do not configure standby interfaces with a test address, the interface is not chosen for failovers when another interface in the group fails. A standby interface might carry traffic under the following conditions:

Thus, the system does not normally select a standby interface (except for probes), unless it is explicitly chosen by an application. If some interface in the group fails, all network access is failed over to the standby interface. To configure a standby interface, you use the ifconfig command's new standby parameter using the following syntax:


# ifconfig interface-name standby group group-name

How to Configure a Multipathing Group Where One of the Interfaces is a Standby Interface shows the steps you perform.

The in.mpathd daemon sends probes on the standby interface once a test address is configured on the standby interface. You should configure only test addresses on a standby interface. If any other address is added on the standby, the addition of these addresses will fail. If standby is marked on an interface which already has addresses other than test addresses, automatic failover of these addresses will take place to a different interface in the group, leaving behind only the test address, if there is one. It is advisable not to configure non-test address on a standby interface.

You need to mark the address as a test address by using the ifconfig command's deprecated and -failover option before setting standby or setting up.

To configure a test address on a standby interface, use the following syntax:


# ifconfig interface-name plumb ip-address <other-parameters> deprecated -failover standby up

For <other-parameters>, use the parameters required by your configuration. See the ifconfig(1M) man page for descriptions.

Note –

Standby interfaces are not used for failover if there is no test address configured on that interface.

For example, to create a test address with the following configuration:

You type the following command line:


# ifconfig hme2 plumb 19.16.85.22 netmask + broadcast + deprecated -failover standby up
Note –

The interface is marked as a standby interface only after the address is marked as a NOFAILOVER address.

How to Configure a Multipathing Group Where One of the Interfaces is a Standby Interface shows the steps you perform.

You can clear a standby interface using the following syntax:


# ifconfig interface-name -standby

Administering Multipathing Groups With a Single Physical Interface

When you have only one network adapter in the multipathing group, you can configure the network adapter to detect failures on that NIC alone.

Because failovers cannot occur with only one NIC in the group, you do not need a separate test address on each of the physical interface in the group. You can configure the normal address as an IFF_NOFAILOVER address, which is sufficient for the daemon to send out probes on that interface. Unlike the multiple physical interface case, you do not have to mark it deprecated. Because there is only one interface in the group, addresses can not failover to a different interface. Hence, using this address does not cause any application failures.

Use the following syntax to configure the interface's IPv4 address as a NOFAILOVER:


# ifconfig interface-name -failover group group-name

For IPv6, use the following syntax:


# ifconfig interface-name inet6 -failover group group-name

When the daemon detects failures, the interface is marked and logged appropriately on the console.

Note –

You cannot verify whether the target being probed has failed or the NIC has failed, because there is only one physical interface through which the target can be probed. If there is only one default router on the subnet, you should turn off multipathing when there is a single physical interface in the group. If a separate IPv4 and IPv6 default router exists (or multiple default routers exist), there is at least more than one target to probe. Hence, it is safe to turn on multipathing.

Removing Network Adapters From Multipathing Groups

When you execute the ifconfig command's group parameter with a null string, the interface is removed from the existing group (see How to Remove an Interface From a Group). Be careful when removing interfaces from a group. If some other interface in the multipathing group failed, a failover could have happened earlier. For example, if hme0 failed previously, all addresses are failed over to hme1 (assuming hme1 is part of the same group). Removing hme1 from the group causes in.mpathd to move all the failover addresses back to some other interface in the group. If no other interfaces are functioning in the group, failover might not restore all the network accesses.

Similarly, when an interface is part of the group and it needs to be unplumbed, you should remove it from the group first, and then ensure that it has all the original IP addresses configured on it. The in.mpathd daemon tries to restore the original configuration of an interface that is removed from the group. You need to ensure that the configuration is restored before unplumbing the interface. Refer to Multipathing Daemon to see how interfaces look before and after a failover.

Detached Network Adapters

Dynamic Reconfiguration (DR) uses IP Network Multipathing to decommission a specific network device without impacting existing IP users. Before a NIC is DR-detached (off-lined), all failover IP addresses hosted on that NIC are automatically failed over to another NIC in the same IP Network Multipathing group. The test addresses are brought down and the NIC unplumbed.

Also, with the IP Network Multipathing reboot-safe feature, the static IP addresses in the /etc/hostname.* file that are associated with the missing card are hosted automatically on an alternate interface within the same IP Network Multipathing group. However, these addresses will not be moved back to the original interface automatically if the original interface is inserted back into the system at a later time.

Note –

NIC attaches and re-attaches require manual intervention.

Multipathing Daemon

The in.mpathd multipathing daemon detects failures and repairs by sending out probes on all the interfaces that are part of a group. When an interface is part of a group and has a test address, the daemon starts sending out probes for determining failures on that interface. If there are no replies to five consecutive probes, the interface is considered as having failed. The probing rate depends on the failure detection time. By default, failure detection time is 10 seconds. Thus, the probing rate is one probe every two seconds. To avoid synchronization in the network, probing is not periodic. If five consecutive probes fail, in.mpathd considers the interface as failed and performs a failover of the network access from the failed interface to another interface in the group that is functioning properly. If a standby interface is configured, it is chosen for failover of the IP addresses, and broadcasts and multicast memberships. If no standby interface exists, the interface with the least number of IP addresses is chosen. Refer to the man page in.mpathd(1M) for more information.

The following two examples show a typical configuration and how the configuration automatically changes when an interface fails. When the hme0 interface fails, notice that all addresses move from hme0 to hme1.

Example 1–1 Interface Configuration Before an Interface Failure


hme0: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
     inet 19.16.85.19 netmask ffffff00 broadcast 19.16.85.255
     groupname test
hme0:1: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 
     index 2 inet 19.16.85.21 netmask ffffff00 broadcast 129.146.85.255
hme1: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
     inet 19.16.85.20 netmask ffffff00 broadcast 19.16.85.255
     groupname test
hme1:1: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 
     index 2 inet 19.16.85.22 netmask ffffff00 broadcast 129.146.85.255
hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6,NOFAILOVER> mtu 1500 index 2
     inet6 fe80::a00:20ff:feb9:19fa/10
     groupname test
hme1: flags=a000841<UP,RUNNING,MULTICAST,IPv6,NOFAILOVER> mtu 1500 index 2
     inet6 fe80::a00:20ff:feb9:1bfc/10
     groupname test
Example 1–2 Interface Configuration After an Interface Failure


hme0: flags=19000842<BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER,FAILED> mtu 0 index 2
        inet 0.0.0.0 netmask 0 
        groupname test
hme0:1: flags=19040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FAILED> 
        mtu 1500 index 2 inet 19.16.85.21 netmask ffffff00 broadcast 129.146.85.255
hme1: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 19.16.85.20 netmask ffffff00 broadcast 19.16.85.255
        groupname test
hme1:1: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 
        index 2 inet 19.16.85.22 netmask ffffff00 broadcast 129.146.85.255
hme1:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
        inet 19.16.85.19 netmask ffffff00 broadcast 19.16.18.255
hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6,NOFAILOVER,FAILED> mtu 1500 index 2
        inet6 fe80::a00:20ff:feb9:19fa/10 
        groupname test
hme1: flags=a000841<UP,RUNNING,MULTICAST,IPv6,NOFAILOVER> mtu 1500 index 2
        inet6 fe80::a00:20ff:feb9:1bfc/10 
        groupname test

You can see that the FAILED flag is set on hme0 to indicate that hme0 has failed. You can also see that hme1:2 is now created. hme1:2 was originally hme0. The address 19.16.85.19 then becomes accessible through hme1. Multicast memberships associated with 19.16.85.19 can still receive packets, but now through hme1. When the failover of address 19.16.85.19 from hme0 to hme1 took place, a dummy address 0.0.0.0 was created on hme0. The dummy address is removed when a subsequent failback takes place. The dummy address is created so that hme0 can still be accessed. hme0:1 cannot exist without hme0.

Similarly, failover of the IPv6 address from hme0 to hme1 took place. In IPv6, multicast memberships are associated with interface indexes. They also failover from hme0 to hme1. All the addresses that in.ndpd configures also move (this is not shown in the examples).

The in.mpathd daemon continues to probe through the failed NIC, hme0. After it receives 10 consecutive replies (for a default failure detection time of 10 seconds), it considers the interface repaired and invokes the failback. After failback, the original configuration is re-established.

See in.mpathd(1M) man page for a description of all error messages logged on the console during failures and repairs.

Multipathing Configuration File

The in.mpathd daemon uses the settings in the /etc/default/mpathd configuration file to invoke multipathing. Changes to this file are read by in.mpathd at startup and on SIGHUP. This file contains the following default settings and information:


#
# Time taken by mpathd to detect a NIC failure in ms. The minimum time
# that can be specified is 100 ms.
# 
FAILURE_DETECTION_TIME=10000
#

# Failback is enabled by default. To disable failback turn off this option
#
FAILBACK=yes
#

# By default only interfaces configured as part of multipathing groups 
# are tracked. Turn off this option to track all network interfaces 
# on the system
#
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes

How to Configure the Multipathing Configuration File shows the steps you perform to configure the /etc/default/mpathd configuration file.

Failure Detection Time

You can set a lower value of failure detection time. Sometimes these values might not be achieved if the load on the network is too high. Then in.mpathd prints a message on the console, indicating that the time cannot be met. It also prints the time that it can meet currently. If the response comes back correctly, in.mpathd meets the failure detection time provided in this file.

Failback

After a failover, failbacks take place when the failed interface is repaired. However, in.mpathd does not failback the interface if FAILBACK is set to no.

As noted in Detecting Physical Interface Failures, automatic failback is not supported for physical interfaces that are not present at system boot. See How to Recover a Physical Interface That Was Not Present at System Boot.

Track Interfaces Only With Groups Option

By turning off this option, in.mpathd tracks all interfaces in the system. When a failure is detected, an appropriate message is logged on the console. For this option to function properly, Ethernet addresses on all the interfaces must be unique.