JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle Solaris Administration: Network Interfaces and Network Virtualization     Oracle Solaris 11 Information Library
search filter icon
search icon

Document Information


1.  Overview of the Networking Stack

Network Configuration in This Oracle Solaris Release

The Network Stack in Oracle Solaris

Network Devices and Datalink Names

Administration of Other Link Types

Part I Network Auto-Magic

2.  Introduction to NWAM

3.  NWAM Configuration and Administration (Overview)

4.  NWAM Profile Configuration (Tasks)

5.  NWAM Profile Administration (Tasks)

6.  About the NWAM Graphical User Interface

Part II Datalink and Interface Configuration

7.  Using Datalink and Interface Configuration Commands on Profiles

8.  Datalink Configuration and Administration

9.  Configuring an IP Interface

10.  Configuring Wireless Interface Communications on Oracle Solaris

11.  Administering Bridges

12.  Administering Link Aggregations

13.  Administering VLANs

14.  Introducing IPMP

What's New With IPMP

Deploying IPMP

Why You Should Use IPMP

When You Must Use IPMP

Comparing IPMP and Link Aggregation

Using Flexible Link Names on IPMP Configuration

How IPMP Works

IPMP Components in Oracle Solaris

Types of IPMP Interface Configurations

IPMP Addressing

IPv4 Test Addresses

IPv6 Test Addresses

Failure and Repair Detection in IPMP

Types of Failure Detection in IPMP

Probe-Based Failure Detection

Link-Based Failure Detection

Failure Detection and the Anonymous Group Feature

Detecting Physical Interface Repairs

The FAILBACK=no Mode

IPMP and Dynamic Reconfiguration

Attaching New NICs

Detaching NICs

Replacing NICs

IPMP Terminology and Concepts

15.  Administering IPMP

16.  Exchanging Network Connectivity Information With LLDP

Part III Network Virtualization and Resource Management

17.  Introducing Network Virtualization and Resource Control (Overview)

18.  Planning for Network Virtualization and Resource Control

19.  Configuring Virtual Networks (Tasks)

20.  Using Link Protection in Virtualized Environments

21.  Managing Network Resources

22.  Monitoring Network Traffic and Resource Usage



Failure and Repair Detection in IPMP

To ensure continuous availability of the network to send or receive traffic, IPMP performs failure detection on the IPMP group's underlying IP interfaces. Failed interfaces remain unusable until these are repaired. Remaining active interfaces continue to function while any existing standby interfaces are deployed as needed.

Types of Failure Detection in IPMP

The in.mpathd daemon handles the following types of failure detection:

Probe-Based Failure Detection

Probe-based failure detection consists of using ICMP probes to check whether an interface has failed. The implementation of this failure detection method depends on whether test addresses are used or not.

Probe-Based Failure Detection Without Using Test Addresses

With no test addresses, this method is implemented by using two types of probes:

Note - You must enable transitive probing to use this failure detection method that does not require test addresses.

Probe-Based Failure Detection Using Test Addresses

This failure detection method involves sending and receiving ICMP probe messages that use test addresses. These messages, also called probe traffic or test traffic, go out over the interface to one or more target systems on the same local network. The daemon probes all the targets separately through all the interfaces that have been configured for probe-based failure detection. If no replies are made in response to five consecutive probes on a given interface, in.mpathd considers the interface to have failed. The probing rate depends on the failure detection time (FDT). The default value for failure detection time is 10 seconds. However, you can tune the failure detection time in the IPMP configuration file. For instructions, go to How to Configure the Behavior of the IPMP Daemon. To optimize probe-based failure detection, you must set multiple target systems to receive the probes from the multipathing daemon. By having multiple target systems, you can better determine the nature of a reported failure. For example, the absence of a response from the only defined target system can indicate a failure either in the target system or in one of the IPMP group's interfaces. By contrast, if only one system among several target systems does not respond to a probe, then the failure is likely in the target system rather than in the IPMP group itself.

The in.mpathd daemon determines which target systems to probe dynamically. First the daemon searches the routing table for target systems on the same subnet as the test addresses that are associated with the IPMP group's interfaces. If such targets are found, then the daemon uses them as targets for probing. If no target systems are found on the same subnet, then in.mpathd sends multicast packets to probe neighbor hosts on the link. The multicast packet is sent to the all hosts multicast address, in IPv4 and ff02::1 in IPv6, to determine which hosts to use as target systems. The first five hosts that respond to the echo packets are chosen as targets for probing. If in.mpathd cannot find routers or hosts that responded to the multicast probes, then ICMP echo packets, in.mpathd cannot detect probe-based failures. In this case, the ipmpstat -i utility will report the probe state as unknown.

You can use host routes to explicitly configure a list of target systems to be used by in.mpathd. For instructions, refer to Configuring for Probe-Based Failure Detection.

Group Failure

A group failure occurs when all interfaces in an IPMP group appear to fail at the same time. In this case, no underlying interface is usable. Also, when all the target systems fail at the same time and probe-based failure detection is enabled, the in.mpathd daemon flushes all of its current target systems and probes for new target systems.

In an IPMP group that has no test addresses, a single interface that can probe the active interface will be designated as a prober. This designated interface will have both the FAILED flag and PROBER flag set. The data address is bound to this interface which allows the interface to continue probing the target to detect recovery.

Link-Based Failure Detection

Link-based failure detection is always enabled, provided that the interface supports this type of failure detection.

To determine whether a third-party interface supports link-based failure detection, use the ipmpstat -i command. If the output for a given interface includes an unknown status for its LINK column, then that interface does not support link-based failure detection. Refer to the manufacturer's documentation for more specific information about the device.

These network drivers that support link-based failure detection monitor the interface's link state and notify the networking subsystem when that link state changes. When notified of a change, the networking subsystem either sets or clears the RUNNING flag for that interface, as appropriate. If the in.mpathd daemon detects that the interface's RUNNING flag has been cleared, the daemon immediately fails the interface.

Failure Detection and the Anonymous Group Feature

IPMP supports failure detection in an anonymous group. By default, IPMP monitors the status only of interfaces that belong to IPMP groups. However, the IPMP daemon can be configured to also track the status of interfaces that do not belong to any IPMP group. Thus, these interfaces are considered to be part of an “anonymous group.” When you issue the command ipmpstat -g, the anonymous group will be displayed as double-dashes (--). In anonymous groups, the interfaces would have their data addresses function also as test addresses. Because these interfaces do not belong to a named IPMP group, then these addresses are visible to applications. To enable tracking of interfaces that are not part of an IPMP group, see How to Configure the Behavior of the IPMP Daemon.

Detecting Physical Interface Repairs

Repair detection time is twice the failure detection time. The default time for failure detection is 10 seconds. Accordingly, the default time for repair detection is 20 seconds. After a failed interface has been marked with the RUNNING flag again and the failure detection method has detected as repaired, in.mpathd clears the interface's FAILED flag. The repaired interface is redeployed depending on the number of active interfaces that the administrator has originally set.

When an underlying interface fails and probe-based failure detection is used, the in.mpathd daemon continues probing, either by means of the designated prober when no test addresses are configured, or by using the interface's test address. During an interface repair, the restoration proceeds depending on the original configuration of the failed interface:

To see a graphical presentation of how IPMP behaves during interface failure and repair, see How IPMP Works.

The FAILBACK=no Mode

By default, active interfaces that have failed and then repaired automatically return to become active interfaces in the group. This behavior is controlled by the setting of the FAILBACK parameter in the daemon's configuration file. However, even the insignificant disruption that occurs as data addresses are remapped to repaired interfaces might not be acceptable to some administrators. The administrators might prefer to allow an activated standby interface to continue as an active interface. IPMP allows administrators to override the default behavior to prevent an interface to automatically become active upon repair. These interfaces must be configured in the FAILBACK=no mode. For related procedures, see How to Configure the Behavior of the IPMP Daemon.

When an active interface in FAILBACK=no mode fails and is subsequently repaired, the IPMP daemon restores the IPMP configuration as follows:

Note - The FAILBACK=NO mode is set for the whole IPMP group. It is not a per-interface tunable parameter.