Solaris Network Multipathing

The following components implement Solaris network multipathing:

• Multipathing daemon – in.mpathd

• IP

The in.mpathd daemon detects failures and implements various policies for failover and failback. After in.mpathd detects a failure or repair, in.mpathd sends an ioctl to do the failover or failback. IP, which implements the ioctl, does the network access failover transparently and automatically.

You should not use Alternate Pathing while using IP Network Multipathing on the same set of NICs. Likewise, you should not use IP Network Multipathing while using Alternate Pathing. You can use Alternate Pathing and IP Network Multipathing at the same time on different sets of NICs.

Detecting Physical Interface Failures

The in.mpathd daemon sends ICMP echo probes to the targets connected to the link on all the interfaces that belong to a group to detect failures and repair. After you add an interface to a multipathing group and assign a test address, the daemon sends probes to detect failures on all the interfaces of the multipathing group. How to Configure a Multipathing Interface Group with Two Interfaces describes the steps you perform to configure test address and groups.

Because in.mpathd determines what targets to probe dynamically, you cannot configure the targets. Routers connected to the link are chosen as targets for probing. If no routers exist on the link, arbitrary hosts on the link are chosen. A multicast packet sent to the “all hosts” multicast address (224.0.0.1 in IPv4 and ff02::1 in IPv6) determines the arbitrary hosts. The first few hosts that respond to the echo packets are chosen as targets for probing. If in.mpathd cannot find routers or hosts that responded to ICMP echo packets, in.mpathd cannot detect failures.

To ensure that each NIC in the group functions properly, in.mpathd probes all the targets separately through all the interfaces in the multipathing group. If there are no replies to five consecutive probes, in.mpathd considers the interface as having failed. The probing rate depends on the failure detection time (FDT). The default value for failure detection time is 10 seconds. The in.mpathd(1M) man page describes how to change the failure detection time. For a failure detection time of 10 seconds, the probing rate is approximately one probe every two seconds.

If five consecutive probes fail, in.mpathd considers the interface as having failed. After a failure is detected, failover of all network access takes place from the failed interface to another functional interface in the group. If you have configured a standby interface, in.mpathd chooses the standby interface for failover of IP addresses, broadcasts, and multicast memberships. If you have not configured a standby interface, in.mpathd chooses the interface with the least number of IP addresses.

Physical interfaces in the same group that are not present at system boot represent a special case of failure detection. The startup script /etc/init.d/network detects these types of failure. This type of failure displays error messages similar to the following:

moving addresses from failed IPv4 interfaces: hme0 (moved to hme1)
moving addresses from failed IPv6 interfaces: hme0 (moved to hme1)

Currently, this type of failure cannot be automatically repaired by a failback. See How to Recover a Physical Interface That Was Not Present at System Boot.

Detecting Physical Interface Repairs

To detect if a physical interface has been repaired, in.mpathd sends probes through the failed NIC. When an interface fails, all addresses are moved to another functional interface in the group. Because in.mpathd needs an address for probing so that it can detect repairs, you must configure a test IP address that will not move during the failover. Moreover, you should not allow a normal application to use this test address, because the failover of network access will not take place for these addresses. How to Configure a Multipathing Interface Group with Two Interfaces describes the steps that you perform. in.mpathd considers the interface repaired if it receives responses to 10 consecutive probe packets. Then, if in.mpathd detects a failure, failback of all network access takes place to the repaired interface.

As noted in Detecting Physical Interface Failures, automatic failback is not supported for physical interfaces that are not present at system boot. See How to Recover a Physical Interface That Was Not Present at System Boot.

Link Failures

If all the NICs appear to fail at the same time, it is considered a link failure and in.mpathd does not do any failovers. This is also true when all the targets fail at the same time. In this case, in.mpathd flushes all of its current targets and discovers new targets (see Detecting Physical Interface Failures).