Sun Cluster 3.0 System Administration Guide

5.2 Administering the Public Network

If you need to alter a public network configuration, you can use the software procedures in this section.

When administering public network adapters pay attention to the following:

Avoid unconfiguring (unplumbing) or bringing down the active adapter of a NAFO group without first switching over the active adapter to a backup adapter in the group. See "5.2.5 How to Switch a NAFO Group's Active Adapter".
Avoid rewiring backup adapters to different subnets without first removing them from their respective NAFO groups.
Logical adapter operations can be done on the active adapter even if monitoring is on for the group.
You must maintain at least one public network connection for each node in the cluster. The cluster is inaccessible without a public network connection.

For cluster software installation procedures, see the Sun Cluster 3.0 Installation Guide. For procedures about servicing public networking hardware components, see the Sun Cluster 3.0 Hardware Guide.

Table 5-2 Task Map: Administering the Public Network


Task	For Instructions, Go To...
Create a NAFO group on a node.	"5.2.1 How to Create a NAFO Group"
Add more public network adapters to a node for increased availability of the public network.	"5.2.3 How to Add an Adapter to a NAFO Group"
Delete a NAFO group to not have any monitoring or failover for any adapter in the group.	"5.2.2 How to Delete a NAFO Group"
Remove backup adapters from an existing NAFO group to enable the adapter to be removed from the system, to be replaced, or to be reconnected to a different subnet and used as backup for another NAFO group.	"5.2.4 How to Remove an Adapter From a NAFO Group"
Switch the active adapter to a backup adapter so that the current active adapter can be removed from the NAFO group.	"5.2.5 How to Switch a NAFO Group's Active Adapter"
Check the status of NAFO groups.	"5.2.6 How to Check the Status of NAFO Groups"
Change parameters to tune the PNM fault detection and failover process.	"5.2.7 How to Change Public Network Management Tunable Parameters"

5.2.1 How to Create a NAFO Group

Note the following requirements for creating a NAFO group:
- All public network adapters must be configured to belong to a NAFO group.
- For any given node, there can be at most one NAFO group on a given subnet.
- All adapters in a given NAFO group must be connected to the same subnet.
- Only one adapter in a given NAFO group can have a hostname association, that is, an /etc/hostname.adapter file.
- A public network adapter can belong to only one NAFO group.

Become superuser on the node being configured for a NAFO group.

For this node, find out the public network adapters that are physically connected to the same subnet.

These adapters form the backup adapters for the NAFO group.

Determine if you need to create an /etc/hostname.adapter file for one of the public network adapters.
- If no (the file already exists), proceed to Step 5.
- If yes (the file does not exist), create the file and add the hostname of the public network adapter's IP address to the file. For example:
# vi /etc/hostname.hme0 phys-schost-1

Edit the /etc/inet/hosts file to add the IP address and corresponding hostname assigned to the public network adapter.

For example, the following shows the IP address 192.29.75.101 and hostname phys-schost-1 added to the /etc/inet/hosts file.
# vi /etc/inet/hosts 192.29.75.101 phys-schost-1
Note -
If a naming service is used, this information should also exist in the naming service database.

Create the NAFO group.
# pnmset -c nafo-group -o create adapter [adapter ...]
-c nafo-group

Performs a configuration subcommand for the specified NAFO group. NAFO groups must be named nafoN, where N is a nonnegative integer identifier for the group. Group names are local to each node. Thus, the same NAFO group name can be used on multiple nodes.

-o create

Creates the new NAFO group.

adapter [ adapter ... ]

Specifies the public network adapter(s) that serves as the backup adapter. See Step 3 above.

Note -
If an adapter is already configured, it will be chosen as the active adapter and the pnmset command does not alter its state. Otherwise, one of the backup adapters will be configured and assigned the IP address found in the /etc/hostname.adapter file for the NAFO group.

Verify the status of the NAFO group.
# pnmstat -l

5.2.1.1 Example--Creating a NAFO Group

The following example shows the creation of a NAFO group (nafo0) configured with two network adapters (qfe0 and qfe1).

# pnmstat -l
# pnmset -c nafo0 -o create qfe0 qfe1
# pnmstat -l
group   adapters       status   fo_time   act_adp
nafo0   qfe0:qfe1       OK      NEVER     qfe0

5.2.2 How to Delete a NAFO Group

Delete a NAFO group when you do not want monitoring and failover for any adapter in the group. To be deleted, a NAFO group cannot be in use by logical host resource groups or shared address resource groups.

Become superuser on the node that contains the NAFO group that is being deleted.

Identify whether the NAFO group is being used by any logical host or shared address resources.
# scrgadm -pv
You can also use scrgadm -pvv (with two v flags) to locate the resources that are using the NAFO group you are going to delete.

Switch logical host resource groups and shared address resource groups using this NAFO group to another node.
# scswitch -z -g resource-group -h node
-z -g resource-group

Switches the specified resource group.

-h node

Specifies the name of the node to switch the resource group to.

Delete the NAFO group.
# pnmset -c nafo-group -o delete
-c nafo-group

Specifies the NAFO group to be deleted.

-o delete

Deletes the NAFO group.

Verify the status of the NAFO group.

The deleted NAFO group should not appear in the listing.
# pnmstat -l

5.2.2.1 Example--Deleting a NAFO Group

The following example shows the NAFO group named nafo1 deleted from the system. Logical host resource group lh-rg-1, which uses this NAFO group, is first switched to a different node.

# scswitch -z -g lh-rg-1 -h phys-schost-2
# pnmstat -l
group   adapters       status   fo_time  act_adp
nafo0   qfe0:qfe1       OK      NEVER    qfe0
nafo1   qfe2            OK      NEVER    qfe2
# pnmset -c nafo1 -o delete
# pnmstat -l
group   adapters        status  fo_time  act_adp
nafo0   qfe0:qfe1        OK     NEVER    qfe0

5.2.3 How to Add an Adapter to a NAFO Group

You can add adapters to an existing NAFO group to provide additional backup adapters for the NAFO group and thereby increase the availability of public network connectivity for the cluster node.

Do you need to install the new public network adapter card(s) in the node(s)?
- If yes, see the Sun Cluster 3.0 Hardware Guide for instructions.
- If no, proceed to Step 2.

Make sure the adapter to be added to the NAFO group is connected to the same subnet as the active adapter for the NAFO group.

Make sure the adapter is not plumbed, and that it does not have an associated /etc/hostname.adapter file.

Become superuser on the node that contains the NAFO group to which the new adapter is being added.

Add the adapter to the NAFO group.
# pnmset -c nafo-group -o add adapter
-c nafo-group

Specifies the NAFO group to which the new adapter is being added.

-o add adapter

Specifies the public network adapter being added to the named NAFO group.

Verify the status of the NAFO group.
# pnmstat -l

5.2.3.1 Example--Adding an Additional Public Network Adapter to a NAFO Group

The following example shows the addition of adapter qfe2 to NAFO group nafo0 which already contained two adapters (qfe0, qfe1).

# pnmstat -l
group   adapters        status   fo_time  act_adp
nafo0   qfe0:qfe1        OK      NEVER    qfe0
# pnmset -c nafo0 -o add qfe2
# pnmstat -l
group   adapters        status   fo_time  act_adp
nafo0   qfe0:qfe1:qfe2   OK      NEVER    qfe0

5.2.4 How to Remove an Adapter From a NAFO Group

Remove backup adapters from an existing NAFO group to enable the adapter to be removed from the system, to be replaced, or to be reconnected to a different subnet and used as backup for another NAFO group.

Caution -

Removing the last backup adapter from a NAFO group results in no protection against faults detected on the active adapter, reducing public network availability for the cluster node.

If you want to remove the active adapter, first switch to another adapter in the group.

See "5.2.5 How to Switch a NAFO Group's Active Adapter".

As superuser, remove the adapter from the NAFO group.
# pnmset -c nafo-group -o remove adapter
-c nafo-group

Specifies the NAFO group from which to remove the adapter.

-o remove adapter

Removes the adapter from the NAFO group.

Verify the status of the NAFO group.

The deleted adapter should not appear in the listing for the NAFO group.
# pnmstat -l

5.2.4.1 Example--Removing an Adapter From a NAFO Group

The following example removes adapter qfe2 from NAFO group nafo0.

# pnmstat -l
group   adapters       status   fo_time  act_adp
nafo0   qfe0:qfe1:qfe2  OK      NEVER    qfe0
# pnmset -c nafo0 -o remove qfe2
# pnmstat -l
group   adapters       status   fo_time  act_adp
nafo0   qfe0:qfe1       OK      NEVER    qfe0

5.2.5 How to Switch a NAFO Group's Active Adapter

Switch the active adapter to a backup adapter so that the current active adapter can be removed from the NAFO group. The pnmd(1M) daemon moves all IP addresses hosted by the current active adapter to the new active adapter in a similar fashion as a fault-triggered adapter failover.

Note -

Connections can experience a brief delay while the switchover is taking place. Otherwise, the operation is transparent to higher-level applications.

Ensure the physical connectivity of the new active adapter is identical to that of the current active adapter.

If the new active adapter fails to host some of the IP addresses as the current active adapter, network and data services that depend on those IP addresses are interrupted until the physical connectivity is fixed or a subsequent successful failover occurs.

Become superuser on the node that contains the NAFO group whose active adapter you want to switch.

Switch the active adapter.
# pnmset -c nafo-group -o switch adapter
-c nafo-group

Specifies the NAFO group containing the adapter to switch.

-o switch adapter

Makes the specified adapter the active adapter in the NAFO group.

Verify the status of the NAFO group.

The "switched-to" adapter should now appear as the active adapter.
# pnmstat -l

5.2.5.1 Example--Switching a NAFO Group's Active Adapter

The following example switches the active adapter to qfe1 from qfe0.

# pnmstat -l
group   adapters       status   fo_time  act_adp
nafo0   qfe0:qfe1       OK      NEVER    qfe0
# pnmset -c nafo0 -o switch qfe1
# pnmstat -l
group   adapters       status   fo_time  act_adp
nafo0   qfe0:qfe1       OK      11       qfe1

5.2.6 How to Check the Status of NAFO Groups

Run the pnmstat(1M) command to list information about the current setup and status of all NAFO groups on a node.

# pnmstat -l

You can also use the pnmptor(1M) and pnmrtop(1M) commands to get information on adapters.

5.2.6.1 Examples--Checking the Status of NAFO Groups

The following example shows the status of a node's three NAFO groups.

# pnmstat -l
Group   adapters        status  fo_time act_adp 
nafo0   qfe5            OK      NEVER   qfe5 
nafo1   qfe6            OK      NEVER   qfe6 
nafo2   qfe7            OK      NEVER   qfe7

5.2.6.2 Examples--Finding the Active Adapter For a NAFO Group

The following example shows that the active adapter in NAFO group nafo0 is adapter qfe5.

# pnmptor nafo0
qfe5

5.2.6.3 Examples--Finding an Adapter's NAFO Group

The following example shows that adapter qfe5 belongs to NAFO group nafo0.

# pnmrtop qfe5
nafo0

5.2.7 How to Change Public Network Management Tunable Parameters

There are four tunable parameters in this algorithm: inactive_time, ping_timeout, repeat_test, and slow_network. These parameters provide an adjustable trade-off between speed and correctness of fault detection. See Table 5-3 for more information.

Use this procedure to change the default Public Network Management (PNM) values for the pnmd(1M) daemon.

Become superuser on a node in the cluster.

If it doesn't already exist, create the pnmparams file.
# vi /etc/cluster/pnmparams

Use the following table to set PNM parameters.

Note -

Settings in the /etc/cluster/pnmparams file apply to all NAFO groups on the node. Lines started with a pound sign (#) are ignored. Other lines in the file must be of the format: variable=value.

Table 5-3 Public Network Management Tunable Parameters


Parameter	Description
`inactive_time`	Number of seconds between successive probes of the packet counters of the current active adapter. Default is 5.
`ping_timeout`	Time-out value in seconds for the `ALL_HOST_MULTICAST` and subnet broadcast pings. Default is 4.
`repeat_test`	Number of times to do the ping sequence before declaring that the active adapter is faulty and failover is triggered. Default is 3.
`slow_network`	Number of seconds waited after each ping sequence before checking packet counters for any change. Default is 2.
`warmup_time`	Number of seconds waited after failover to a backupadapter before resuming fault monitoring. This allows extra time for any slow driver or port initialization. Default is 0.

Note -

The changes don't take affect until the next time the pnmd daemon starts up.

5.2.7.1 Example--Changing Public Network Management Tunable Parameters

The following shows a sample/etc/cluster/pnmparams file, with two parameters changed from their default values.

inactive_time=3
repeat_test=5