System Administration Guide: Network Interfaces and Network Virtualization

Configuring IPMP Groups

This section provides procedures that are used to plan and configure IPMP groups.

How to Plan an IPMP Group

The following procedure includes the required planning tasks and information to be gathered prior to configuring an IPMP group. The tasks do not have to be performed in sequence.

Determine the general IPMP configuration that would suit your needs.

Your IPMP configuration depends on what your network needs to handle the type of traffic that is hosted on your system. IPMP spreads outbound network packets across the IPMP group's interfaces, and thus improves network throughput. However, for a given TCP connection, inbound traffic normally follows only one physical path to minimize the risk of processing out-of-order packets.

Thus, if your network handles a huge volume of outbound traffic, configuring multiple interfaces into an IPMP group can improve network performance. If instead, the system hosts heavy inbound traffic, then the number of interfaces in the group does not necessarily improve performance by load spreading traffic. However, having multiple interfaces helps to guarantee network availability during interfaces failure.

For SPARC based systems, verify that each interface in the group has a unique MAC address.

To configure a unique MAC address for each interface in the system, see SPARC: How to Ensure That the MAC Address of an Interface Is Unique.

Ensure that the same set of STREAMS modules is pushed and configured on all interfaces in the IPMP group.

All interfaces in the same group must have the same STREAMS modules configured in the same order.
1. Check the order of STREAMS modules on all interfaces in the prospective IPMP group.
  
  You can print a list of STREAMS modules by using the ifconfig interface modlist command. For example, here is the ifconfig output for an hme0 interface:
  # ifconfig hme0 modlist 0 arp 1 ip 2 hme
  Interfaces normally exist as network drivers directly below the IP module, as shown in the output from ifconfig hme0 modlist. They should not require additional configuration.
  
  However, certain technologies insert themselves as a STREAMS module between the IP module and the network driver. If a STREAMS module is stateful, then unexpected behavior can occur on failover, even if you push the same module onto all of the interfaces in a group. However, you can use stateless STREAMS modules, provided that you push them in the same order on all interfaces in the IPMP group.
2. Push the modules of an interface in the standard order for the IPMP group.
  ifconfig interface modinsert module-name@position
  ifconfig hme0 modinsert vpnmod@3

Use the same IP addressing format on all interfaces of the IPMP group.

If one interface is configured for IPv4, then all interfaces of the group must be configured for IPv4. For example, if you add IPv6 addressing to one interface, then all interfaces in the IPMP group must be configured for IPv6 support.

Determine the type of failure detection that you want to implement.

For example, if you want to implement probe-based failure detection, then you must configure test addresses on the underlying interfaces. For related information, seeTypes of Failure Detection in IPMP.

Ensure that all interfaces in the IPMP group are connected to the same local network.

For example, you can configure Ethernet switches on the same IP subnet into an IPMP group. You can configure any number of interfaces into an IPMP group.

Note –
You can also configure a single interface IPMP group, for example, if your system has only one physical interface. For related information, see Types of IPMP Interface Configurations.

Ensure that the IPMP group does not contain interfaces with different network media types.

The interfaces that are grouped together should be of the same interface type, as defined in /usr/include/net/if_types.h. For example, you cannot combine Ethernet and Token ring interfaces in an IPMP group. As another example, you cannot combine a Token bus interface with asynchronous transfer mode (ATM) interfaces in the same IPMP group.

For IPMP with ATM interfaces, configure the ATM interfaces in LAN emulation mode.

IPMP is not supported for interfaces using Classical IP over ATM.

How to Configure an IPMP Group by Using DHCP

In the current IPMP implementation, IPMP groups can be configured with Dynamic Host Configuration Protocol (DHCP) support.

A multiple-interfaced IPMP group can be configured with active-active interfaces or active-standby interfaces. For related information, see Types of IPMP Interface Configurations. The following procedure describes steps to configure an active-standby IPMP group by using DHCP.

Before You Begin

Make sure that IP interfaces that will be in the prospective IPMP group have been correctly configured over the system's network data links. For procedures to configure links and IP interfaces, see Data Link and IP Interface Configuration (Tasks). For information about configuring IPv6 interfaces, see Configuring an IPv6 Interface in System Administration Guide: IP Services.

Additionally, if you are using a SPARC system, configure a unique MAC address for each interface. For procedures, see SPARC: How to Ensure That the MAC Address of an Interface Is Unique.

Finally, if you are using DHCP, make sure that the underlying interfaces have infinite leases. Otherwise, in case of a group failure, the test addresses will expire and the IPMP daemon will then revert to link-based failure detection. Such circumstances would trigger errors in the manner the group's failure detection behaves during interface recovery. For more information about configuring DHCP, refer to Chapter 12, Planning for DHCP Service (Tasks), in System Administration Guide: IP Services.

On the system on which you want to configure the IPMP group, assume the Primary Administrator role, or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Create an IPMP interface.
# ifconfig ipmp-interface ipmp [group group-name]
Note –
To configure IPv6 IPMP interfaces, use the same command syntax for configuring IPv6 interfaces by specifying inet6 in the ifconfig command, for example:
# ifconfig ipmp-interface inet6 ipmp [group group-name]
This note applies to all configuration procedures that involve IPv6 IPMP interfaces.

ipmp-interface

Specifies the name of the IPMP interface. You can assign any meaningful name to the IPMP interface. As with any IP interface, the name consists of a string and a number, such as ipmp0.

group-name

Specifies the name of the IPMP group. The name can be any name of your choice. Assigning a group name is optional. By default, the name of the IPMP interface also becomes the name of the IPMP group. Preferably, retain this default setting by not using the group-name option.

Note –
The syntax in this step uses the preferred explicit method of creating an IPMP group by creating the IPMP interface.

An alternative method to create an IPMP group is implicit creation, in which you use the syntax ifconfig interface group group-name. In this case, the system creates the lowest available ipmpN to become the group's IPMP interface. For example, if ipmp0 already exists for group acctg, then the syntax ifconfig ce0 group fieldops causes the system to create ipmp1 for group fieldops. All UP data addresses of ce0 are then assigned to ipmp1.

However, implicit creation of IPMP groups is not encouraged. Support for implicit creation is provided only to have compatible implementation with previous Solaris releases. Explicit creation provides optimal control over the configuration of IPMP interfaces.

Add underlying IP interfaces that will contain test addresses to the IPMP group, including the standby interface.
# ifconfig interface group group-name -failover [standby] up

Have DHCP configure and manage the data addresses on the IPMP interface.

You need to plumb as many logical IPMP interfaces as data addresses, and then have DHCP configure and manage the addresses on these interfaces as well.
# ifconfig ipmp-interface dhcp start primary # ifconfig ipmp-interface:n plumb # ifconfig ipmp-interface:n dhcp start

Have DHCP manage the test addresses in the underlying interfaces.

You need to issue the following command for each underlying interface of the IPMP group.
# ifconfig interface dhcp start

Example 8–1 Configuring an IPMP Group With DHCP

This example shows how to configure an active-standby IPMP group with DHCP. This example is based on Figure 7–1, which contains the following information:

Three underlying interfaces, subitops0, subitops1, and subitops2 are designated members of the IPMP group.
The IPMP interface itops0 shares the same name with the IPMP group.
subitops2 is the designated standby interface.
To use probe-based failure detection, all the underlying interfaces are assigned test addresses.

# ifconfig itops0 ipmp

# ifconfig subitops0 plumb group itops0 -failover up
# ifconifg subitops1 plumb group itops0 -failover up
# ifconfig subitops2 plumb group itops0 -failover standby up

# ifconfig itops0 dhcp start primary
# ifconfig itops0:1 plumb
# ifconfig itops0:1 dhcp start

# ifconfig subitops0 dhcp start
# ifconfig subitops1 dhcp start
# ifconfig subitops2 dhcp start

To make the test address configuration persistent, you would need to type the following commands:

# touch /etc/dhcp.itops0 /etc/dhcp.itops0:1
# touch /etc/dhcp.subitops0 /etc/dhcp.subitops1 /etc/dhcp.subitops2
	
# echo group itops0 -failover up > /etc/hostname.subitops0
# echo group itops0 -failover up > /etc/hostname.subitops1
# echo group itops0 -failover standby up > /etc/hostname.subitops2
# echo ipmp > /etc/hostname.itops0

How to Manually Configure an Active-Active IPMP Group

The following procedure describes steps to manually configure an active-active IPMP group.

Before You Begin

Additionally, if you are using a SPARC system, configure a unique MAC address for each interface. For procedures, see SPARC: How to Ensure That the MAC Address of an Interface Is Unique.

On the system on which you want to configure the IPMP group, assume the Primary Administrator role, or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Create an IPMP interface.
# ifconfig ipmp-interface ipmp [group group-name]
ipmp-interface

Specifies the name of the IPMP interface. You can assign any meaningful name to the IPMP interface. As with any IP interface, the name consists of a string and a number, such as ipmp0.

group-name

Specifies the name of the IPMP group. The name can be any name of your choice. Any nun-null name is valid, provided that the name does not exceed 31 characters. Assigning a group name is optional. By default, the name of the IPMP interface also becomes the name of the IPMP group. Preferably, retain this default setting by not using the group-name option.

Note –
The syntax in this step uses the preferred explicit method of creating an IPMP group by creating the IPMP interface.

An alternative method to create an IPMP group is implicit creation, in which you use the syntax ifconfig interface group group-name. In this case, the system creates the lowest available ipmpN to become the group's IPMP interface. For example, if ipmp0 already exists for group acctg, then the syntax ifconfig ce0 group fieldops causes the system to create ipmp1 for group fieldops. All UP data addresses of ce0 are then assigned to ipmp1.

However, implicit creation of IPMP groups is not encouraged. Support for implicit creation is provided only to have compatible implementation with previous Solaris releases. Explicit creation provides optimal control over the configuration of IPMP interfaces.

Add underlying IP interfaces to the group.
# ifconfig ip-interface group group-name
Note –
In a dual-stack environment, placing the IPv4 instance of an interface under a particular group automatically places the IPv6 instance under the same group as well.

Add data addresses to the IPMP interface.
# ifconfig plumb ipmp-interface ip-address up # ifconfig ipmp-interface addif ip-address up
For additional options that you can use with the ifconfig command while adding addresses, refer to the ifconfig(1M) man page.

Configure test addresses on the underlying interfaces.
# ifconfig interface -failover ip-address up
Note –
You need to configure a test address only if you want to use probe-based failure detection on a particular interface.

All test IP addresses in an IPMP group must use the same network prefix. The test IP addresses must belong to a single IP subnet.

(Optional) Preserve the IPMP group configuration across reboots.

To configure an IPMP group that persists across system reboots, you would edit the hostname configuration file of the IPMP interface to add data addresses. Then, if you want to use test addresses, you would edit the hostname configuration file of one of the group's underlying IP interface. Note that data and test addresses can be both IPv4 and IPv6 addresses. Perform the following steps:
1. Edit the /etc/hostname.ipmp-interface file by adding the following lines:
  ipmp group group-name data-address up addif data-address ...
  You can add more data addresses on separate addif lines in this file.
2. Edit the /etc/hostname.interface file of the underlying IP interfaces that contain the test address by adding the following line:
  group group-name -failover test-address up
  Follow this same step to add test addresses to other underlying interfaces of the IPMP group.
  
  Caution –
  When adding test address information on the /etc/hostname.interface file, make sure to specify the -failover option before the up keyword. Otherwise, the test IP addresses will be treated as data addresses and would cause problems for system administration. Preferably, set the -failover option before specifying the IP address.

How to Manually Configure an Active-Standby IPMP Group

For more information about standby interfaces, see Types of IPMP Interface Configurations. The following procedure configures an IPMP group where one interface is kept as a reserve. This interface is deployed only when an active interface in the group fails.

On the system on which you want to configure the IPMP group, assume the Primary Administrator role, or become superuser.

The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

Create an IPMP interface.
# ifconfig ipmp-interface ipmp [group group-name]
ipmp-interface

Specifies the name of the IPMP interface. You can assign any meaningful name to the IPMP interface. As with any IP interface, the name consists of a string and a number, such as ipmp0.

group-name

Specifies the name of the IPMP group. The name can be any name of your choice. Any nun-null name is valid, provided that the name does not exceed 31 characters. Assigning a group name is optional. By default, the name of the IPMP interface also becomes the name of the IPMP group. Preferably, retain this default setting by not using the group-name option.

Note –
The syntax in this step uses the preferred explicit method of creating an IPMP group by creating the IPMP interface.

An alternative method to create an IPMP group is implicit creation, in which you use the syntax ifconfig interface group group-name. In this case, the system creates the lowest available ipmpN to become the group's IPMP interface. For example, if ipmp0 already exists for group acctg, then the syntax ifconfig ce0 group fieldops causes the system to create ipmp1 for group fieldops. All UP data addresses of ce0 are then assigned to ipmp1.

However, implicit creation of IPMP groups is not encouraged. Support for implicit creation is provided only to have compatible implementation with previous Solaris releases. Explicit creation provides optimal control over the configuration of IPMP interfaces.

Add underlying IP interfaces to the group.
# ifconfig ip-interface group group-name
Note –
In a dual-stack environment, placing the IPv4 instance of an interface under a particular group automatically places the IPv6 instance under the same group as well.

Add data addresses to the IPMP interface.
# ifconfig plumb ipmp-interface ip-address up # ifconfig ipmp-interface addif ip-address up
For additional options that you can use with the ifconfig command while adding addresses, refer to the ifconfig(1M) man page.

Configure test addresses on the underlying interfaces.
- To configure a test address on an active interface, use the following command:
  # ifconfig interface -failover ip-address up
- To configure a test address on a designated standby interface, use the following command:
  # ifconfig interface -failover ip-address standby up
Note –
You need to configure a test address only if you want to use probe-based failure detection on a particular interface.

All test IP addresses in an IPMP group must use the same network prefix. The test IP addresses must belong to a single IP subnet.

(Optional) Preserve the IPMP group configuration across reboots.

To configure an IPMP group that persists across system reboots, you would edit the hostname configuration file of the IPMP interface to add data addresses. Then, if you want to use test addresses, you would edit the hostname configuration file of one of the group's underlying IP interface. Note that data and test addresses can be both IPv4 and IPv6 addresses. Perform the following steps:
1. Edit the /etc/hostname.ipmp-interface file by adding the following lines:
  ipmp group group-name data-address up addif data-address ...
  You can add more data addresses on separate addif lines in this file.
2. Edit the /etc/hostname.interface file of the underlying IP interfaces that contain the test address by adding the following line:
  group group-name -failover test-address up
  Follow this same step to add test addresses to other underlying interfaces of the IPMP group. For a designated standby interface, the line must be as follows:
  group group-name -failover test-address standby up
  Caution –
  When adding test address information on the /etc/hostname.interface file, make sure to specify the -failover option before the up keyword. Otherwise, the test IP addresses will be treated as data addresses and would cause problems for system administration. Preferably, set the -failover option before specifying the IP address.

Example 8–2 Configuring an Active-Standby IPMP Group

This example shows how to manually create the same persistent active-standby IPMP configuration that is provided in Example 8–1.

# ifconfig itops0 ipmp

# ifconfig subitops0 group itops0
# ifconfig subitops1 group itops0
# ifconfig subitops2 group itops0

# ifconfig itops0 192.168.10.10/24 up
# ifconfig itops0 addif 192.168.10.15/24 up

# ifconfig subitops0 -failover 192.168.85.30/24 up
# ifconfig subitops1 -failover 192.168.86.32/24 up
# ifconfig subitops2 -failover 192.168.86.34/24 standby up

# ipmpstat -g
GROUP     GROUPNAME   STATE      FDT        INTERFACES
itops0    itops0      ok         10.00s     subitops0 subitops1 (subitops2)

# ipmpstat -t
INTERFACE      MODE     TESTADDR        TARGETS
subitops0      routes   192.168.10.30   192.168.10.1
subitops1      routes   192.168.10.32   192.168.10.1
subitops2      routes   192.168.10.34   192.168.10.5

# vi /etc/hostname.itops0
ipmp group itops0 192.168.10.10/24 up
addif 192.168.10.15/24 up

# vi /etc/hostname.subitops0
group itops0 -failover 192.168.10.30/24 up

# vi /etc/hostname.subitops1
group itops0 -failover 192.168.10.32/24 up

# vi /etc/hostname.subitops2
group itops0 -failover 192.168.10.34/24 standby up