System Administration Guide: IP Services

Chapter 31 Administering IPMP (Tasks)

This chapter provides tasks for administering interface groups with IP network multipathing (IPMP). The following major topics are discussed:

For an overview of IPMP concepts, refer to Chapter 30, Introducing IPMP (Overview).

Configuring IPMP (Task Maps)

This section contains links to the tasks that are described in this chapter.

Configuring and Administering IPMP Groups (Task Map)

Task	Description	For Instructions
Plan for an IPMP group.	Lists all ancillary information and required tasks before you can configure an IPMP group.	How to Plan for an IPMP Group
Configure an IPMP interface group with multiple interfaces.	Configures multiple interfaces as members of an IPMP group.	How to Configure an IPMP Group With Multiple Interfaces
Configure an IPMP group where one of the interfaces is a standby interface.	Configures one of the interfaces in a multiple interface IPMP group as a standby interface.	How to Configure a Standby Interface for an IPMP Group
Configure an IPMP group that consists of a single interface.	Creates a single interface IPMP group.	How to Configure a Single Interface IPMP Group
Display the IPMP group to which a physical interface belongs.	Explains how to obtain the name of an interface's IPMP group from the output of the `ifconfig` command.	How to Display the IPMP Group Membership of an Interface
Add an interface to an IPMP group.	Configures a new interface as a member of an existing IPMP group.	How to Add an Interface to an IPMP Group
Remove an interface from an IPMP group.	Explains how to remove an interface from an IPMP group.	How to Remove an Interface From an IPMP Group
Move an interface from an existing IPMP group to a different group.	Moves interfaces among IPMP groups.	How to Move an Interface From One IPMP Group to Another Group
Change three default settings for the `in.mpathd` daemon.	Customizes failure detection time and other parameters of the `in.mpathd` daemon.	How to Configure the `/etc/default/mpathd` File

Administering IPMP on Interfaces That Support Dynamic Reconfiguration (Task Map)

Task	Description	For Instructions
Remove an interface that has failed.	Removes a failed interface on a system.	How to Remove a Physical Interface That Has Failed (DR-Detach)
Replace an interface that has failed.	Replaces a failed interface.	How to Replace a Physical Interface That Has Failed (DR-Attach)
Recover an interface that was not configured at boot time.	Recovers a failed interface.	How to Recover a Physical Interface That Was Not Present at System Boot

Configuring IPMP Groups

This section provides procedures for configuring IPMP groups. It also describes how to configure an interface as a standby.

Planning for an IPMP Group

Before you configure interfaces on a system as part of an IPMP group, you need to do some preconfiguration planning.

How to Plan for an IPMP Group

The following procedure includes the planning tasks and information to be gathered prior to configuring the IPMP group. The tasks do not have to be performed in sequence.

Decide which interfaces on the system are to be part of the IPMP group.

An IPMP group usually consists of at least two physical interfaces that are connected to the same IP link. However, you can configure a single interface IPMP group, if required. For an introduction to IPMP groups, refer to IPMP Interface Configurations. For example, you can configure the same Ethernet switch or the same IP subnet under the same IPMP group. You can configure any number of interfaces into the same IPMP group.

You cannot use the group parameter of the ifconfig command with logical interfaces. For example, you can use the group parameter with hme0, but not with hme0:1.

Verify that each interface in the group has a unique MAC address.

For instructions, refer to SPARC: How to Ensure That the MAC Address of an Interface Is Unique.

Choose a name for the IPMP group.

Any non-null name is appropriate for the group. You might want to use a name that identifies the IP link to which the interfaces are attached.

Ensure that the same set of STREAMS modules is pushed and configured on all interfaces in the IPMP group.

All interfaces in the same group must have the same STREAMS modules configured in the same order.
1. Check the order of STREAMS modules on all interfaces in the prospective IPMP group.
  
  You can print out a list of STREAMS modules by using the ifconfig interface modlist command. For example, here is the ifconfig output for an hme0 interface:
  # ifconfig hme0 modlist 0 arp 1 ip 2 hme
  Interfaces normally exist as network drivers directly below the IP module, as shown in the output from ifconfig hme0 modlist. They should not require additional configuration.
  
  However, certain technologies, such as NCA or IP Filter, insert themselves as STREAMS modules between the IP module and the network driver. Problems can result in the way interfaces of the same IPMP group behave.
  
  If a STREAMS module is stateful, then unexpected behavior can occur on failover, even if you push the same module onto all of the interfaces in a group. However, you can use stateless STREAMS modules, provided that you push them in the same order on all interfaces in the IPMP group.
2. Push the modules of an interface in the standard order for the IPMP group.
  ifconfig interface modinsert module-name
  ifconfig hme0 modinsert ip

Use the same IP addressing format on all interfaces of the IPMP group.

If one interface is configured for IPv4, then all interfaces of the group must be configured for IPv4. Suppose you have an IPMP group that is composed of interfaces from several NICs. If you add IPv6 addressing to the interfaces of one NIC, then all interfaces in the IPMP group must be configured for IPv6 support.

Check that all interfaces in the IPMP group are connected to the same IP link.

Verify that the IPMP group does not contain interfaces with different network media types.

The interfaces that are grouped together should be of the same interface type, as defined in /usr/include/net/if_types.h. For example, you cannot combine Ethernet and Token ring interfaces in an IPMP group. As another example, you cannot combine a Token bus interface with asynchronous transfer mode (ATM) interfaces in the same IPMP group.

For IPMP with ATM interfaces, configure the ATM interfaces in LAN emulation mode.

IPMP is not supported for interfaces using Classical IP over ATM.

Configuring IPMP Groups

This section contains configuration tasks for a typical IPMP group with at least two physical interfaces.

For an introduction to multiple interface IPMP groups, refer to IPMP Group.
For planning tasks, refer to Planning for an IPMP Group.
To configure an IPMP group with only one physical interface, refer to Configuring IPMP Groups With a Single Physical Interface.

How to Configure an IPMP Group With Multiple Interfaces

The following steps for configuring an IPMP group also apply when configuring VLANs into an IPMP group.

Before You Begin

You need to have already configured the IPv4 addresses, and, if appropriate, the IPv6 addresses of all interfaces in the prospective IPMP group.

Caution –

You must configure only one IPMP group for each subnet or L2 broadcast domain. For more information, see Basic Requirements of IPMP

On the system with the interfaces to be configured, assume the Primary Administrator role, or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Place each physical interface into an IPMP group.
# ifconfig interface group group-name
For example, to place hme0 and hme1 under group testgroup1, you would type the following commands:
# ifconfig hme0 group testgroup1 # ifconfig hme1 group testgroup1
Avoid using spaces in group names. The ifconfig status display does not show spaces. Consequently, do not create two similar group names where the only difference is that one name also contains a space. If one of the group names contains a space, these group names look the same in the status display.

In a dual-stack environment, placing the IPv4 instance of an interface under a particular group automatically places the IPv6 instance under the same group.

(Optional) Configure an IPv4 test address on one or more physical interfaces.

You need to configure a test address only if you want to use probe-based failure detection on a particular interface. Test addresses are configured as logical interfaces of the physical interface that you specify to the ifconfig command.

If one interface in the group is to become the standby interface, do not configure a test address for that interface at this time. You configure a test address for the standby interface as part of the task How to Configure a Standby Interface for an IPMP Group.

Use the following syntax of the ifconfig command for configuring a test address:
# ifconfig interface addif ip-address parameters -failover deprecated up
For example, you would create the following test address for the primary network interface hme0:
# ifconfig hme0 addif 192.168.85.21 netmask + broadcast + -failover deprecated up
This command sets the following parameters for the primary network interface hme0:
- Address set to 192.168.85.21
- Netmask and broadcast address set to the default value
- -failover and deprecated options set
  
  Note –
  You must mark an IPv4 test address as deprecated to prevent applications from using the test address.

Check the IPv4 configuration for a specific interface.

You can always view the current status of an interface by typing ifconfig interface. For more information on viewing an interface's status, refer to How to Get Information About a Specific Interface.

You can get information about test address configuration for a physical interface by specifying the logical interface that is assigned to the test address.
# ifconfig hme0:1 hme0:1: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2 inet 192.168.85.21 netmask ffffff00 broadcast 192.168.85.255

(Optional) If applicable, configure an IPv6 test address.
# ifconfig interface inet6 -failover
Physical interfaces with IPv6 addresses are placed into the same IPMP group as the interfaces' IPv4 addresses. This happens when you configure the physical interface with IPv4 addresses into an IPMP group. If you first place physical interfaces with IPv6 addresses into an IPMP group, physical interfaces with IPv4 addresses are also implicitly placed in the same IPMP group.

For example, to configure hme0 with an IPv6 test address, you would type the following:
# ifconfig hme0 inet6 -failover
You do not need to mark an IPv6 test address as deprecated to prevent applications from using the test address.

Check the IPv6 configuration.

# ifconfig hme0 inet6
	hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6,NOFAILOVER> mtu 1500 index 2
        	inet6 fe80::a00:20ff:feb9:17fa/10 
        	groupname test

The IPv6 test address is the link-local address of the interface.

(Optional) Preserve the IPMP group configuration across reboots.
- For IPv4, add the following line to the /etc/hostname.interface file:
  interface-address <parameters> group group-name up \ addif logical-interface -failover deprecated <parameters> up
  In this instance, the test IPv4 address is configured only on the next reboot. If you want the configuration to be invoked in the current session, do steps 1, 2, and, optionally 3.
- For IPv6, add the following line to the /etc/hostname6.interface file:
  -failover group group-name up
  This test IPv6 address is configured only on the next reboot. If you want the configuration to be invoked in the current session, do steps 1, 2, and, optionally, 5.

(Optional) Add more interfaces to the IPMP group by repeating steps 1 through 6.

You can add new interfaces to an existing group on a live system. However, changes are lost across reboots.

Example 31–1 Configuring an IPMP Group With Two Interfaces

Suppose you want to do the following:

Have the netmask and broadcast address set to the default value.
Configure the interface with a test address 192.168.85.21.

You would type the following command:

# ifconfig hme0 addif 192.168.85.21 netmask + broadcast + -failover deprecated up

You must mark an IPv4 test address as deprecated to prevent applications from using the test address. See How to Configure an IPMP Group With Multiple Interfaces.

To turn on the failover attribute of the address, you would use the failover option without the dash

All test IP addresses in an IPMP group must use the same network prefix. The test IP addresses must belong to a single IP subnet.

Example 31–2 Preserving an IPv4 IPMP Group Configuration Across Reboots

Suppose you want to create an IPMP group called testgroup1 with the following configuration:

Physical interface hme0 with the data address 192.168.85.19
A logical interface with the test address 192.168.85.21

Note –
In this example, physical interface and data address are paired together. Likewise for logical interface and test address. However, no inherent relationships exist between an interface “type” and the address type.
deprecated and -failover options set
Netmask and broadcast address set to the default value

You would add the following line to the /etc/hostname.hme0 file:

192.168.85.19 netmask + broadcast + group testgroup1 up \
	addif 192.168.85.21 deprecated -failover netmask + broadcast + up

Similarly, to place the second interface hme1 under the same group testgroup1 and to configure a test address, you would add the following line:

192.168.85.20 netmask + broadcast + group testgroup1 up \
	addif 192.168.85.22 deprecated -failover netmask + broadcast + up

Example 31–3 Preserving an IPv6 IPMP Group Configuration Across Reboots

To create a test group for interface hme0 with an IPv6 address, you would add the following line to the /etc/hostname6.hme0 file:

-failover group testgroup1 up

Similarly, to place the second interface hme1 in group testgroup1 and to configure a test address, you would add the following line to the /etc/hostname6.hme1 file:

-failover group testgroup1 up

Troubleshooting

During IPMP group configuration, in.mpathd outputs a number of messages to the system console or to the syslog file. These messages are informational in nature and indicate that the IPMP configuration functions correctly.

This message indicates that interface hme0 was added to IPMP group testgroup1. However, hme0 does not have a test address configured. To enable probe-based failure detection, you need to assign a test address to the interface.
May 24 14:09:57 host1 in.mpathd[101180]: No test address configured on interface hme0; disabling probe-based failure detection on it. testgroup1

This message appears for all interfaces with only IPv4 addresses that are added to an IPMP group.

May 24 14:10:42 host4 in.mpathd[101180]: 
NIC qfe0 of group testgroup1 is not 
plumbed for IPv6 and may affect failover capability

This message should appear when you have configured a test address for an interface.

Created new logical interface hme0:1
May 24 14:16:53 host1 in.mpathd[101180]: 
Test address now configured on interface hme0;
 enabling probe-based failure detection on it

Configuring Target Systems

Probe-based failure detection involves the use of target systems, as explained in Probe-Based Failure Detection. For some IPMP groups, the default targets used by in.mpathd is sufficient. However, for some IPMP groups, you might want to configure specific targets for probe-based failure detection. You accomplish probe-based failure detection by setting up host routes in the routing table as probe targets. Any host routes that are configured in the routing table are listed before the default router. Therefore, IPMP uses the explicitly defined host routes for target selection. You can use either of two methods for directly specifying targets: manually setting host routes or creating a shell script that can become a startup script.

Consider the following criteria when evaluating which hosts on your network might make good targets.

Make sure that the prospective targets are available and running. Make a list of their IP addresses.
Ensure that the target interfaces are on the same network as the IPMP group that you are configuring.
The netmask and broadcast address of the target systems must be the same as the addresses in the IPMP group.
The target host must be able to answer ICMP requests from the interface that is using probe-based failure detection.

How to Manually Specify Target Systems for Probe-Based Failure Detection

Add a route to a particular host to be used as a target in probe-based failure detection.
$ route add -host destination-IP gateway-IP -static
Replace the values of destination-IP and gateway-IP with the IPv4 address of the host to be used as a target. For example, you would type the following to specify the target system 192.168.85.137, which is on the same subnet as the interfaces in IPMP group testgroup1.
$ route add -host 192.168.85.137 192.168.85.137 -static

Add routes to additional hosts on the network to be used as target systems.

How to Specify Target Systems in a Shell Script

On the system where you have configured an IPMP group, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Create a shell script that sets up static routes to your proposed targets.

For example, you could create a shell script called ipmp.targets with the following contents:

TARGETS="192.168.85.117 192.168.85.127 192.168.85.137"

case "$1" in
        'start')
            /usr/bin/echo "Adding static routes for use as IPMP targets"
		for target in $TARGETS; do
	  /usr/sbin/route add -host $target $target
		done
                  ;;
        'stop')
              /usr/bin/echo "Removing static routes for use as IPMP targets"
		 for target in $TARGETS; do
		/usr/sbin/route delete -host $target $target
		 done
                  ;;
  esac

Copy the shell script to the startup script directory.
# cp ipmp.targets /etc/init.d

Change the permissions on the new startup script.
# chmod 744 /etc/init.d/ipmp.targets

Change ownership of the new startup script.

# chown root:sys /etc/init.d/ipmp.targets

Create a link for the startup script in the /etc/init.d directory.
# ln /etc/init.d/ipmp.targets /etc/rc2.d/S70ipmp.targets
The S70 prefix in the file name S70ipmp.targets orders the new script properly with respect to other startup scripts.

Configuring Standby Interfaces

Use this procedure if you want the IPMP group to have an active-standby configuration. For more information on this type of configuration, refer to IPMP Interface Configurations.

How to Configure a Standby Interface for an IPMP Group

Before You Begin

You must have configured all interfaces as members of the IPMP group.
You should not have configured a test address on the interface to become the standby interface.

For information on configuring an IPMP group and assigning test addresses, refer to How to Configure an IPMP Group With Multiple Interfaces.

On the system with the standby interfaces to be configured, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Configure an interface as a standby and assign the test address.
# ifconfig interface plumb \ ip-address other-parameters deprecated -failover standby up
A standby interface can have only one IP address, the test address. You must set the -failover option before you set the standby up option. For <other-parameters>, use the parameters that are required by your configuration, as described in the ifconfig(1M) man page.
- For example, to create an IPv4 test address, you would type the following command:
  # ifconfig hme1 plumb 192.168.85.22 netmask + broadcast + deprecated -failover standby up
  hme1
  
  Defines hme1 as the physical interface to be configured as the standby interface.
  
  192.168.85.22
  
  Assigns this test address to the standby interface.
  
  deprecated
  
  Indicates that the test address is not used for outbound packets.
  
  -failover
  
  Indicates that the test address does not fail over if the interface fails.
  
  standby
  
  Marks the interface as a standby interface.
- For example, to create an IPv6 test address, you would type the following command:
  # ifconfig hme1 plumb -failover standby up

Check the results of the standby interface configuration.

# ifconfig hme1
hme1: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,
      STANDBY,INACTIVE mtu 1500 
         index 4 inet 192.168.85.22 netmask ffffff00 broadcast 19.16.85.255
         groupname test

The INACTIVE flag indicates that this interface is not used for any outbound packets. When a failover occurs on this standby interface, the INACTIVE flag is cleared.

Note –

You can always view the current status of an interface by typing the ifconfig interface command. For more information on viewing interface status, refer to How to Get Information About a Specific Interface.

(Optional) Preserve the IPv4 standby interface across reboots.

Assign the standby interface to the same IPMP group, and configure a test address for the standby interface.

For example, to configure hme1 as the standby interface, you would add the following line to the /etc/hostname.hme1 file:
192.168.85.22 netmask + broadcast + deprecated group test -failover standby up

(Optional) Preserve the IPv6 standby interface across reboots.

Assign the standby interface to the same IPMP group, and configure a test address for the standby interface.

For example, to configure hme1 as the standby interface, add the following line to the /etc/hostname6.hme1 file:
-failover group test standby up

Example 31–4 Configuring a Standby Interface for an IPMP Group

Suppose you want to create a test address with the following configuration:

Physical interface hme2 as a standby interface
Test address of 192.168.85.22
deprecated and -failover options set
Netmask and broadcast address set to the default value

You would type the following:

# ifconfig hme2 plumb 192.168.85.22 netmask + broadcast + \
deprecated -failover standby up

The interface is marked as a standby interface only after the address is marked as a NOFAILOVER address.

You would remove the standby status of an interface by typing the following:

# ifconfig interface -standby

Configuring IPMP Groups With a Single Physical Interface

When you have only one interface in an IPMP group, failover is not possible. However, you can enable failure detection on that interface by assigning the interface to an IPMP group. You do not have to configure a dedicated test IP address to establish failure detection for a single interface IPMP group. You can use a single IP address for sending data and detecting failure.

How to Configure a Single Interface IPMP Group

On the system with the prospective single interface IPMP group, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

For IPv4, create the single interface IPMP group.

Use the following syntax to assign the single interface to an IPMP group.
# ifconfig interface group group-name
The following example assigns the interface hme0 into the IPMP group v4test:
# ifconfig hme0 group v4test
After this step is performed, IPMP enables link-based failure detection on the interface.

In addition, you can also use the -failover subcommand of the ifconfig command to enable probe-based failure detection. The following example enables probe-based failure detection on hme0 by using the IP address currently assigned to hme0:
# ifconfig hme0 -failover
Note that unlike multiple-interface groups, the same IP address can act as both a data address and a test address. To enable applications to use the test address as a data address, test addresses must never be marked deprecated on single-interface IPMP groups.

For IPv6, create the single interface IPMP group.

Use the following syntax to assign a single interface to an IPMP group:
# ifconfig interface inet6 group group-name
For example, to add the single interface hme0 into the IPMP group v6test, type the following:
# ifconfig hme0 inet6 group v6test
After this step is performed, IPMP enables link-based failure detection on the interface.

In addition, you can also use the -failover subcommand of the ifconfig command to enable probe-based failure detection. The following example enables probe-based failure detection on hme0 by using the IP address currently assigned to hme0:
# ifconfig hme0 inet6 -failover
Note that unlike multiple-interface groups, the same IP address can act as both a data address and a test address. To enable applications to use the test address as a data address, test addresses must never be marked deprecated on single-interface IPMP groups.

In a single physical interface configuration, you cannot verify whether the target system that is being probed has failed or whether the interface has failed. The target system can be probed through only one physical interface. If only one default router is on the subnet, turn off IPMP if a single physical interface is in the group. If a separate IPv4 and IPv6 default router exists, or multiple default routers exist, more than one target system needs to be probed. Hence, you can safely turn on IPMP.

Maintaining IPMP Groups

This section contains tasks for maintaining existing IPMP groups and the interfaces that compose those groups. The tasks presume that you have already configured an IPMP group, as explained in Configuring IPMP Groups.

How to Display the IPMP Group Membership of an Interface

On the system with the IPMP group configuration, become superuser or assume an equivalent role.

Roles contain authorizations and privileged commands. For more information about roles, see Configuring RBAC (Task Map) in System Administration Guide: Security Services.

Display information about the interface, including the group to which the interface belongs.
# ifconfig interface

If applicable, display IPv6 information for the interface.
# ifconfig interface inet6

Example 31–5 Displaying Physical Interface Groups

To display the group name for hme0, you would type the following:

# ifconfig hme0
	hme0: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 
      index 2 inet 192.168.85.19 netmask ffffff00 broadcast 192.168.85.255
      groupname testgroup1

To display the group name for only the IPv6 information, you would type the following:

# ifconfig hme0 inet6
	hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2
        	inet6 fe80::a00:20ff:feb9:19fa/10 
        	groupname testgroup1

How to Add an Interface to an IPMP Group

On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Add the interface to the IPMP group.
# ifconfig interface group group-name
The interface specified in interface becomes a member of IPMP group group-name.

Example 31–6 Adding an Interface to an IPMP Group

To add hme0 to the IPMP group testgroup2, you would type the following command:

# ifconfig hme0 group testgroup2
  hme0: flags=9000843<UP ,BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER> mtu 1500 index 2
  inet 192.168.85.19 netmask ff000000 broadcast 10.255.255.255
  groupname testgroup2
  ether 8:0:20:c1:8b:c3

How to Remove an Interface From an IPMP Group

When you execute the ifconfig command's group parameter with a null string, the interface is removed from its current IPMP group. Be careful when removing interfaces from a group. If some other interface in the IPMP group has failed, a failover could have happened earlier. For example, if hme0 failed previously, all addresses are failed over to hme1, if hme1 is part of the same group. The removal of hme1 from the group causes the in.mpathd daemon to return all the failover addresses to some other interface in the group. If no other interfaces are functioning in the group, failover might not restore all the network accesses.

Similarly, when an interface in a group needs to be unplumbed, you should first remove the interface from the group. Then, ensure that the interface has all the original IP addresses configured. The in.mpathd daemon tries to restore the original configuration of an interface that is removed from the group. You need to ensure that the configuration is restored before unplumbing the interface. Refer to What Happens During Interface Failover to see how interfaces look before and after a failover.

On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Remove the interface from the IPMP group.
# ifconfig interface group ""
The quotation marks indicate a null string.

Example 31–7 Removing an Interface From a Group

To remove hme0 from the IPMP group test, you would type the following command:

# ifconfig hme0 group ""
	# ifconfig hme0
	hme0: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500
    index 2 inet 192.168.85.19 netmask ffffff00 broadcast 192.168.85.255
	# ifconfig hme0 inet6
	hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2
    inet6 fe80::a00:20ff:feb9:19fa/10

How to Move an Interface From One IPMP Group to Another Group

You can place an interface in a new IPMP group when the interface belongs to an existing IPMP group. You do not need to remove the interface from the current IPMP group. When you place the interface in a new group, the interface is automatically removed from any existing IPMP group.

On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Move the interface to a new IPMP group.
# ifconfig interface group group-name
Placing the interface in a new group automatically removes the interface from any existing group.

Example 31–8 Moving an Interface to a Different IPMP Group

To change the IPMP group of interface hme0, you would type the following:

# ifconfig hme0 group cs-link

This command removes the hme0 interface from IPMP group test and then puts the interface in the group cs-link.

Replacing a Failed Physical Interface on Systems That Support Dynamic Reconfiguration

This section contains procedures that relate to administering systems that support dynamic reconfiguration (DR).

Note –

The tasks pertain only to IP layers that are configured by using the ifconfig command. Layers before or after the IP layer, such as ATM or other services, require specific manual steps if the layers are not automated. The steps in the next procedures are used to unconfigure interfaces during predetachment and configure interface after postattachment.

How to Remove a Physical Interface That Has Failed (DR-Detach)

This procedure shows how to remove a physical interface on a system that supports DR. The procedure assumes that the following conditions already exist:

Physical interfaces hme0 and hme1 are the example interfaces.
Both interfaces are in the same IPMP group.
hme0 has failed.
Logical interface hme0:1 has the test address.
You are replacing the failed interface with the same physical interface name, for example, hme0 with hme0.

Note –

You can skip Step 2 if the test address is plumbed by using the /etc/hostname.hme0 file.

On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Display the test address configuration.

# ifconfig hme0:1

hme0:1:
flags=9040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3
inet 192.168.233.250 netmask ffffff00 broadcast 192.168.233.255

You need this information to replumb the test address when replacing the physical interface.

Remove the physical interface.

Refer to the following sources for a complete description of how to remove the physical interface:
- cfgadm(1M) man page
- Sun Enterprise 6x00, 5x00, 4x00, and 3x00 Systems Dynamic Reconfiguration User's Guide
- Sun Enterprise 10000 DR Configuration Guide

How to Replace a Physical Interface That Has Failed (DR-Attach)

This procedure shows how to replace a physical interface on a system that supports DR.

On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Replace the physical interface.

Refer to the instructions in the following sources:
- cfgadm(1M) man page
- Sun Enterprise 6x00, 5x00, 4x00, and 3x00 Systems Dynamic Reconfiguration User's Guide
- Sun Enterprise 10000 DR Configuration Guide, or Sun Fire 880 Dynamic Reconfiguration User's Guide

Recovering a Physical Interface That Was Not Present at System Boot

Note –

The following procedure pertains only to IP layers that are configured by using the ifconfig command. Layers before or after the IP layer, such as ATM or other services, require specific manual steps if the layers are not automated. The specific steps in the next procedure are used to unconfigure interfaces during predetachment and to configure interfaces after postattachment.

Recovery after dynamic reconfiguration is automatic for an interface that is part of the I/O board on a Sun Fire™ platform. If the NIC is a Sun Crypto Accelerator I - cPCI board, the recovery is also automatic. Consequently, the following steps are not required for an interface that is coming back as part of a DR operation. For more information on the Sun Fire x800 and Sun Fire 15000 systems, see the cfgadm_sbd(1M) man page. The physical interface fails back to the configuration that is specified in the /etc/hostname.interface file. See Configuring IPMP Groups for details on how to configure interfaces to preserve the configuration across reboots.

Note –

On Sun Fire legacy (Exx00) systems, DR detachments are still subject to manual procedures. However, DR attachments are automated.

How to Recover a Physical Interface That Was Not Present at System Boot

You must complete the following procedure before you recover a physical interface that was not present at system boot. The example in this procedure has the following configuration:

Physical interfaces hme0 and hme1 are the interfaces.
Both interfaces are in the same IPMP group.
hme0 was not installed at system boot.

Note –

The failback of IP addresses during the recovery of a failed physical interface takes up to three minutes. This time might vary, depending on network traffic. The time also depends on the stability of the incoming interface to fail back the failed-over interfaces by the in.mpathd daemon.

On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Retrieve the failed network information from the failure error message of the console log.

See the syslog(3C)man page. The error message might be similar to the following:
moving addresses from failed IPv4 interfaces: hme1 (moved to hme0)
This message indicates that the IPv4 addresses on the failed interface hme1 have failed over to the hme0 interface.

Alternatively, you might receive the following similar message:
moving addresses from failed IPv4 interfaces: hme1 (couldn't move, no alternative interface)
This message indicates that no active interface could be found in the same group as failed interface hme1. Therefore, the IPv4 addresses on hme1 could not fail over.

Attach the physical interface to the system.

Refer to the following for instructions on how to replace the physical interface:
- cfgadm(1M) man page
- Sun Enterprise 10000 DR Configuration Guide
- Sun Enterprise 6x00, 5x00, 4x00, and 3x00 Systems Dynamic Reconfiguration User's Guide

Refer to the message content from Step 2. If the addresses could not be moved, go to Step 6. If the addresses were moved, continue to Step 5.

Unplumb the logical interfaces that were configured as part of the failover process.
1. Review the contents of the /etc/hostname.moved-from-interface file to determine what logical interfaces were configured as part of the failover process.
2. Unplumb each failover IP address.
  # ifconfig moved-to-interface removeif moved-ip-address
  Note –
  Failover addresses are marked with the failover parameter, or are not marked with the -failover parameter. You do not need to unplumb IP addresses that are marked -failover.
  
  For example, assume that the contents of the /etc/hostname.hme0 file contains the following lines:
  inet 10.0.0.4 -failover up group one addif 10.0.0.5 failover up addif 10.0.0.6 failover up
  To unplumb each failover IP address, you would type the following commands:
  # ifconfig hme0 removeif 10.0.0.5 # ifconfig hme0 removeif 10.0.0.6

Reconfigure the IPv4 information for the replaced physical interface by typing the following command for each interface that was removed:

# ifconfig removed-from-NIC <parameters>

For example, you would type the following commands:

# ifconfig hme1 inet plumb
# ifconfig hme1 inet 10.0.0.4 -failover up group one
# ifconfig hme1 addif 10.0.0.5 failover up
# ifconfig hme1 addif 10.0.0.6 failover up

Modifying IPMP Configurations

Use the IPMP configuration file /etc/default/mpathd to configure the following system-wide parameters for IPMP groups.

FAILURE_DETECTION_TIME
TRACK_INTERFACES_ONLY_WITH_GROUPS
FAILBACK

How to Configure the `/etc/default/mpathd` File

On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Edit the /etc/default/mpathd file.

Change the default value of one or more of the three parameters.
1. Type the new value for the FAILURE_DETECTION_TIME parameter.
  FAILURE_DETECTION_TIME=n
  where n is the amount of time in seconds for ICMP probes to detect whether an interface failure has occurred. The default is 10 seconds.
2. Type the new value for the FAILBACK parameter.
  FAILBACK=[yes | no]
  - yes- The yes value is the default failback behavior of IPMP. When the repair of a failed interface is detected, network access fails back to the repaired interface, as described in IPMP Failure Detection and Recovery Features.
  - no - The no indicates that data traffic does not move back to a repaired interface. When a failed interfaces is detected as repaired, the INACTIVE flag is set for that interface. This flag indicates that the interface is currently not to be used for data traffic. The interface can still be used for probe traffic.
    
    For example, suppose an IPMP group consists of two interfaces, ce0 and ce1. Then assume that the value FAILBACK=no is set in /etc/default/mpathd. If ce0 fails, its traffic fails over to ce1, as is the expected behavior of IPMP. However, when IPMP detects that ce0 is repaired, traffic does not fail back from ce1, due to the FAILBACK=no parameter in /etc/default/mpathd. The ce0 interface retains its INACTIVE status and is not used for traffic unless the ce1 interface fails. If the ce1 interface fails, the addresses on ce1 are migrated back to ce0, whose INACTIVE flag is then cleared. This migration occurs provided that ce0 is the only INACTIVE interface in the group. If other INACTIVE interfaces exist in the group, the addresses might be migrated to an INACTIVE interface other than ce0.
3. Type the new value for the TRACK_INTERFACES_ONLY_WITH_GROUPS parameter.
  TRACK_INTERFACES_ONLY_WITH_GROUPS=[yes | no]
  - yes- The yes value is the default behavior of IPMP. This parameter causes IPMP to ignore network interfaces that are not configured into an IPMP group.
  - no - The no value sets failure and repair detection for all network interfaces, regardless of whether they are configured into an IPMP group. However, when a failure or repair is detected on an interface that is not configured into an IPMP group, no failover or failback occurs. Therefore, theno value is only useful for reporting failures and does not directly improve network availability.

Restart the in.mpathd daemon.
# pkill -HUP in.mpathd

Chapter 31 Administering IPMP (Tasks)

Configuring IPMP (Task Maps)

Configuring and Administering IPMP Groups (Task Map)

Administering IPMP on Interfaces That Support Dynamic Reconfiguration (Task Map)

Configuring IPMP Groups

Planning for an IPMP Group

How to Plan for an IPMP Group

Configuring IPMP Groups

How to Configure an IPMP Group With Multiple Interfaces

Before You Begin

Example 31–1 Configuring an IPMP Group With Two Interfaces

Example 31–2 Preserving an IPv4 IPMP Group Configuration Across Reboots

Example 31–3 Preserving an IPv6 IPMP Group Configuration Across Reboots

Troubleshooting

See Also

Configuring Target Systems

How to Manually Specify Target Systems for Probe-Based Failure Detection

How to Specify Target Systems in a Shell Script

Configuring Standby Interfaces

How to Configure a Standby Interface for an IPMP Group

Before You Begin

Example 31–4 Configuring a Standby Interface for an IPMP Group

Configuring IPMP Groups With a Single Physical Interface

How to Configure a Single Interface IPMP Group

Maintaining IPMP Groups

How to Display the IPMP Group Membership of an Interface

Example 31–5 Displaying Physical Interface Groups

How to Add an Interface to an IPMP Group

Example 31–6 Adding an Interface to an IPMP Group

How to Remove an Interface From an IPMP Group

Example 31–7 Removing an Interface From a Group

How to Move an Interface From One IPMP Group to Another Group

Example 31–8 Moving an Interface to a Different IPMP Group

Replacing a Failed Physical Interface on Systems That Support Dynamic Reconfiguration

How to Remove a Physical Interface That Has Failed (DR-Detach)

How to Replace a Physical Interface That Has Failed (DR-Attach)

Recovering a Physical Interface That Was Not Present at System Boot

How to Recover a Physical Interface That Was Not Present at System Boot

Modifying IPMP Configurations

How to Configure the /etc/default/mpathd File

How to Configure the `/etc/default/mpathd` File