Monitoring IPMP Information

The following procedures use the ipmpstat command, enabling you to monitor different aspects of IPMP groups on the system. You can observe the status of the IPMP group as a whole or its underlying IP interfaces. You can also verify the configuration of data and test addresses for the group. Information about failure detection is also obtained by using the ipmpstat command. For more details about the ipmpstat command and its options, see the PLACEHOLDER IPMPSTAT MAN PAGE.

By default, host names are displayed on the output instead of the numeric IP addresses, provided that the host names exist. To list the numeric IP addresses in the output, use the -n option together with other options to display specific IPMP group information.

In the following procedures, use of the ipmpstat command does not require system administrator privileges, unless stated otherwise.

How to Obtain IPMP Group Information

Use this procedure to list the status of the various IPMP groups on the system, including the status of their underlying interfaces. If probe-based failure detection is enabled for a specific group, the command also includes the failure detection time for that group.

1. Display the IPMP group information.

   $ ipmpstat -g GROUP GROUPNAME STATE FDT INTERFACES itops0 itops0 ok 10.00s subitops0 subitops1 acctg1 acctg1 failed -- [hme0 hme1] field2 field2 degraded 20.00s fops0 fops3 (fops2) [fops1]

   GROUP

   Specifies the IPMP interface name. In the case of an anonymous group, this field will be empty. For more information about anonymous groups, see the in.mpathd(1M) man page.

   GROUPNAME

   Specifies the name of the IPMP group. In the case of an anonymous group, this field will be empty.

   STATE

   Indicates a group's current status, which can be one of the following:

   • ok indicates that all underlying interfaces of the IPMP group are usable.

   • degraded indicates that some of the underlying interfaces in the group are unusable.

   • failed indicates that all of the group's interfaces are unusable.

   FDT

   Specifies the failure detection time, if failure detection is enabled. If failure detection is disabled, this field will be empty.

   INTERFACES

   Specifies the underlying interfaces that belong to the group. In this field, active interfaces are listed first, then inactive interfaces, and finally unusable interfaces.The status of the interface is indicated by the manner in which it is listed:

   • interface (without parentheses or brackets) indicates an active interface. Active interfaces are those interfaces that being used by the system to send or receive data traffic.

   • (interface) (with parentheses) indicates a functioning but inactive interface. The interface is not in use as defined by administrative policy.

   • [interface] (with brackets) indicates that the interface is unusable because it has either failed or been taken offline.

How to Obtain IPMP Data Address Information

Use this procedure to display data addresses and the group to which each address belongs. The displayed information also includes which address is available for use, depending on whether the address has been toggled by the ifconfig [up/down] command. You can also determine on which inbound or outbound interface an address can be used.

1. Display the IPMP address information.

   $ ipmpstat -an ADDRESS STATE GROUP INBOUND OUTBOUND 192.168.10.10 up itops0 subitops0 subitops0 subitops1 192.168.10.15 up itops0 subitops1 subitops0 subitops1 192.0.0.100 up acctg1 -- -- 192.0.0.101 up acctg1 -- -- 128.0.0.100 up field2 fops0 fops0 fops3 128.0.0.101 up field2 fops3 fops0 fops3 128.0.0.102 down field2 -- --

   ADDRESS

   Specifies the hostname or the data address, if the -n option is used in conjunction with the -a option.

   STATE

   Indicates whether the address on the IPMP interface is up, and therefore usable, or down, and therefore unusable.

   GROUP

   Specifies the IPMP IP interface that hosts a specific data address.

   INBOUND

   Identifies the interface that receives packets for a given address. The field information might change depending on external events. For example, if a data address is down, or if no active IP interfaces remain in the IPMP group, this field will be empty. The empty field indicates that the system is not accepting IP packets that are destined for the given address.

   OUTBOUND

   Identifies the interface that sends packets that are using a given address as a source address. As with the INBOUND field, the OUTBOUND field information might also change depending on external events. An empty field indicates that the system is not sending out packets with the given source address. The field might be empty either because the address is down, or because no active IP interfaces remain in the group.

How to Obtain Information About Underlying IP Interfaces of a Group

Use this procedure to display information about an IPMP group's underlying IP interfaces. For a description of the corresponding relationship between the NIC, data link, and IP interface, see Overview of the Networking Stack.

1. Display the IPMP interface information.

   $ ipmpstat -i INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE subitops0 yes itops0 --mb--- up ok ok subitops1 yes itops0 ------- up disabled ok hme0 no acctg1 ------- unknown disabled offline hme1 no acctg1 is----- down unknown failed fops0 yes field2 --mb--- unknown ok ok fops1 no field2 -i----- up ok ok fops2 no filed2 ------- up failed failed fops3 yes field2 --mb--- up ok ok

   INTERFACE

   Specifies each underlying interface of each IPMP group.

   ACTIVE

   Indicates whether the interface is functioning and is in use (yes) or not (no).

   GROUP

   Specifies the IPMP interface name. In the case of anonymous groups, this field will be empty. For more information about anonymous groups, see the in.mpathd(1M) man page.

   FLAGS

   Indicates the status of the underlying interface, which can be one or any combination of the following:

   • i indicates that the INACTIVE flag is set for the interface and therefore the interface is not used to send or receive data traffic.

   • s indicates that the interface is configured to be a standby interface.

   • m indicates that the interface is designated by the system to send and receive IPv4 multicast traffic for the IPMP group.

   • b indicates that the interface is designated by the system to receive broadcast traffic for the IPMP group.

   • M indicates that the interface is designated by the system to send and receive IPv6 multicast traffic for the IPMP group.

   • d indicates that the interface is down and therefore unusable.

   • h indicates that the interface shares a duplicate physical hardware address with another interface and has been taken offline. The h flag indicates that the interface is unusable.

   LINK

   Indicates the state of link-based failure detection, which is one of the following states:

   • up or down indicates the availability or unavailability of a link.

   • unknown indicates that the driver does not support notification of whether a link is up or down and therefore does not detect link state changes.

   PROBE

   Specifies the state of the probe–based failure detection for interfaces that have been configured with a test address, as follows:

   • ok indicates that the probe is functional and active.

   • failed indicates that probe-based failure detection has detected that the interface is not working.

   • unknown indicates that no suitable probe targets could be found, and therefore probes cannot be sent.

   • disabled indicates that no IPMP test address is configured on the interface. Therefore probe-based failure detection is disabled.

   STATE

   Specifies the overall state of the interface, as follows:

   • ok indicates that the interface is online and working normally based on the configuration of failure detection methods.

   • failed indicates that the interface is not working because either the interface's link is down, or the probe detection has determined that the interface cannot send or receive traffic.

   • offline indicates that the interface is not available for use. Typically, the interface is switched offline under the following circumstances:

     • The interface is being tested.

     • Dynamic reconfiguration is being performed.

     • The interface shares a duplicate hardware address with another interface.

   • unknown indicates the IPMP interface's status cannot be determined because no probe targets can be found for probe-based failure detection.

How to Obtain IPMP Probe Target Information

Use this procedure to monitor the probe targets that are associated with each IP interface in an IPMP group.

1. Display the IPMP probe targets.

   $ ipmpstat -nt INTERFACE MODE TESTADDR TARGETS subitops0 routes 192.168.85.30 192.168.85.1 192.168.85.3 subitops1 disabled -- -- hme0 disabled -- -- hme1 routes 192.1.2.200 192.1.2.1 fops0 multicast 128.9.0.200 128.0.0.1 128.0.0.2 fops1 multicast 128.9.0.201 128.0.0.2 128.0.0.1 fops2 multicast 128.9.0.202 128.0.0.1 128.0.0.2 fops3 multicast 128.9.0.203 128.0.0.1 128.0.0.2

   INTERFACE

   Specifies the underlying interfaces of the IPMP group.

   MODE

   Specifies the method for obtaining the probe targets.

   • routes indicates that the system routing table is used to find probe targets.

   • mcast indicates that multicast ICMP probes are used to find targets.

   • disabled indicates that probe-based failure detection has been disabled for the interface.

   TESTADDR

   Specifies the hostname or, if the -n option is used in conjunction with the -t option, the IP address that is assigned to the interface to send and receive probes. This field will be empty if a test address has not been configured.

   ---

   Note –

   If an IP interface is configured with both IPv4 and IPv6 test addresses, the probe target information is displayed separately for each test address.

   ---

   TARGETS

   Lists the current probe targets in a space-separated list. The probe targets are displayed either as hostnames or IP addresses, if the -n is used in conjunction with the -t option.

How to Observe IPMP Probes

Use this procedure to observe ongoing probes. When you issue the command to observe probes, information about probe activity on the system is continuously displayed until you terminate the command with Ctrl-C. You must have Primary Administrator privileges to run this command.

1. Assume the role of Primary Administrator, or become superuser.

   The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

2. Display the information about ongoing probes.

   # ipmpstat -pn TIME INTERFACE PROBE TARGET NETRTT RTT RTTAVG RTTDEV 0.11s subitops0 589 192.168.85.1 0.51ms 0.76ms 0.76ms -- 0.17s hme1 612 192.1.2.1 -- -- -- -- 0.25s fops0 602 128.0.0.1 0.61ms 1.10ms 1.10ms -- 0.26s fops1 602 128.0.0.2 -- -- -- -- 0.25s fops2 601 128.0.0.1 0.62ms 1.20ms 1.00ms -- 0.26s fops3 603 128.0.0.1 0.79ms 1.11ms 1.10ms -- 1.66s hme1 613 192.1.2.1 -- -- -- -- 1.70s subitops0 603 192.168.85.3 0.63ms 1.10ms 1.10ms -- ^C

   TIME

   Specifies the time a probe was sent relative to when the ipmpstat command was issued. If a probe was initiated prior to ipmpstat being started, then the time is displayed with a negative value, relative to when the command was issued.

   PROBE

   Specifies the identifier that represents the probe.

   INTERFACE

   Specifies the interface on which the probe is sent.

   TARGET

   Specifies the hostname or, if the -n option is used in conjunction with -p, the target address to which the probe is sent.

   NETRTT

   Specifies the total network round-trip time of the probe and is measured in milliseconds. NETRTT covers the time between the moment when the IP module sends the probe and the moment the IP module receives the ack packets from the target. If the in.mpathd daemon has determined that the probe is lost, then the field will be empty.

   RTT

   Specifies the total round-trip time for the probe and is measured in milliseconds. RTT covers the time between the moment the daemon executes the code to send the probe and the moment the daemon completes processing the ack packets from the target. If the in.mpathd daemon has determined that the probe is lost, then the field will be empty. Spikes that occur in the RTT which are not present in the NETRTT might indicate that the local system is overloaded.

   RTTAVG

   Specifies the probe's average round-trip time over the interface between local system and target. The average round-trip time helps identify slow targets. If data is insufficient to calculate the average, this field will be empty.

   RTTDEV

   Specifies the standard deviation for the round-trip time to the target over the interface. The standard deviation helps identify jittery targets whose ack packets are being sent erratically. For jittery targets, the in.mpathd daemon is forced to increase the failure detection time. Consequently, the daemon would take a longer time before it can detect such a target's outage. If data is insufficient to calculate the standard deviation, this field will be empty.

How to Customize the Output of the ipmpstat Command in a Script

When you use the ipmpstat, by default, the most meaningful fields that fit in 80 columns are displayed. In the output, all the fields that are specific to the option that you use with the ipmpstat command are displayed, except in the case of the ipmpstat -p syntax. If you want to specify the fields to be displayed, then you use the -o option in conjunction with other options that determine the output mode of the command. This option is particularly useful when you issue the command from a script or by using a command alias

1. To customize the output, issue one of the following commands:

   • To display selected fields of the ipmpstat command, use the -o option in combination with the specific output option. For example, to display only the GROUPNAME and the STATE fields of the group output mode, you would type the following:

     $ ipmpstat -g -o groupname,state GROUPNAME STATE itops0 ok accgt1 failed field2 degraded

   • To display all the fields of a given ipmpstat command, use the following syntax:

     # ipmpstat -o all

How to Generate Machine Parseable Output of the ipmpstat Command

You can generate machine parseable information by using the ipmpstat -P syntax. The -P option is intended to be used particularly in scripts. Machine-parseable output differs from the normal output in the following ways:

• Headers are omitted.

• Fields are separated by colons (:).

• Fields with empty values are empty rather than being filled with the double dash (--).

• In the case of multiple fields being requested, if a field contains a literal colon (:) or back slash (\), these can be escaped or excluded by prefixing these characters with a back slash (\) .

To correctly use the ipmpstat -P syntax, observe the following rules:

• Use the -o option fields together with the -P option.

• Never use -o all with the -P option.

Ignoring either one of these rules will cause ipmpstat -P to fail.

1. To display in machine parseable format the group name, the failure detection time, and the underlying interfaces, you would type the following:

   $ ipmpstat -P -o -g groupname,fdt,interfaces itops0:10.00s:subitops0 subitops1 acctg1::[hme0 hme1] field2:20.00s:fops0 fops3 (fops2) [fops1]

   The group name, failure detection time, and underlying interfaces are group information fields. Thus, you use the -o -g options together with the -P option.

Example 8–8 Using ipmpstat -P in a Script

This sample script displays the failure detection time of a particular IPMP group.

getfdt() {
         ipmpstat -gP -o group,fdt | while IFS=: read group fdt; do
             [[ "$group" = "$1" ]] && { echo "$fdt"; return; }
         done
     }