| Netra™ High Availability Suite 3.0 1/08 Foundation Services Cluster Administration Guide | 
   |
| C H A P T E R 4 |
For information about peer nodes and the network interfaces on those nodes, see the following sections. For a description of addressing and networking, see the Netra High Availability Suite 3.0 1/08 Foundation Services Overview.
Use the following tools to examine the cluster network configuration:
Use these tools to ensure that your cluster network is correctly configured after maintenance, or to obtain information about cluster membership problems. For information about solving cluster membership problems, see the Netra High Availability Suite 3.0 1/08 Foundation Services Troubleshooting Guide.
Before collecting data about the cluster network configuration, perform the following procedure to test whether the interfaces of a node are configured correctly.
Log in as superuser to the node whose interfaces you want to examine.
Check the network configuration:
# nhadm check configuration |
The tests this command performs include:
Whether the network interfaces are configured. For example if the current node is running the Solaris OS, the command checks that /etc/hostname.NIC0 and /etc/hostname.NIC1 exist. If the current node is running MontaVista Linux, the command checks that NIC0 and NIC1 are configured in the file /etc/network/interfaces
Whether the /etc/hosts file contains IP addresses that correspond to the NIC0, NIC1, and cgtp0 interfaces for each peer node
If the current node is running Wind River CGL, the command checks that the following directories exist:
The nhadm tool displays the success or failure of each test it performs.
A network topology map contains information about each node in a cluster and each interface on a node. A network topology map can also include information about nonpeer nodes that are communicating with the cluster.
When investigating your network topology, you can create a network topology map. The map is a helpful reference when using the cluster or changing the cluster configuration. The following figure illustrates an example of the information that you can include in a network topology map.
As you work through the procedures in this chapter, add information to the network topology map for the nodes in your cluster. You can link all the nodes in the network topology map using the routing information described in Examining the Routes on a Node.
This section describes how to obtain network information about a peer node.
|
This procedure determines the IP addresses, netmask value, network IDs, node IDs, Ethernet address, and interface names of a peer node.
Log in to the node whose network interfaces you want to examine.
# ifconfig -a |
The ifconfig command displays configuration information about the network interfaces.
Each peer node has at least three configured network interfaces. The physical interfaces, NIC0 and NIC1, and the CGTP interface. Additional network interfaces are displayed for the master node, the vice-master node, and for interfaces that are configured for external access.
If the node is running the Solaris OS and is using the class B addressing scheme, output similar to the following is displayed:
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 cgtp0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 172.17.0.25 netmask ffff0000 broadcast 172.17.255.255 ether 0:0:0:0:0:0 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 172.15.0.25 netmask ffff0000 broadcast 172.15.255.255 ether 8:0:20:f9:a5:56 hme1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 172.16.0.25 netmask ffff0000 broadcast 172.16.255.255 ether 8:0:20:f9:a5:57 |
In this example, the nodeid is 25, the netmask is ffff0000, and the subnetworks are 172.15.0.0, 172.16.0.0, and 172.17.0.0.
If the node is using the default class C addressing scheme, output similar to the following is displayed:
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 cgtp0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.25.3.25 netmask ffffff00 broadcast 10.25.3.255 ether 0:0:0:0:0:0 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.25.1.25 netmask ffffff00 broadcast 10.25.1.255 ether 8:0:20:f9:a5:56 hme1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 10.25.2.25 netmask ffffff00 broadcast 10.25.2.255 ether 8:0:20:f9:a5:57 |
In this example, the nodeid is 25, the netmask is ffffff00, and the subnetworks are 10.25.1.0, 10.25.2.0, and 10.25.3.0.
If the node is running Linux and is using the default class C addressing scheme, output similar to the following is displayed:
cgtp0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:10.125.3.20 Bcast:10.125.3.255 Mask:255.255.255.0 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:80209 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:9631182 (9.1 MiB) eth0 Link encap:Ethernet HWaddr 00:03:BA:F1:7B:58 inet addr:10.125.1.20 Bcast:10.125.1.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:7b58/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:64908 errors:0 dropped:0 overruns:0 frame:0 TX packets:56376 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:47373431 (45.1 MiB) TX bytes:6795936 (6.4 MiB) Interrupt:32 eth1 Link encap:Ethernet HWaddr 00:03:BA:F1:7B:59 inet addr:10.125.2.20 Bcast:10.125.2.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:7b59/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:25575 errors:0 dropped:0 overruns:0 frame:0 TX packets:23833 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:18522762 (17.6 MiB) TX bytes:2835246 (2.7 MiB) Interrupt:33 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:20877 errors:0 dropped:0 overruns:0 frame:0 TX packets:20877 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3042793 (2.9 MiB) TX bytes:3042793 (2.9 MiB) |
In this example, the nodeid is 20, the netmask is ffffff00, and the subnetworks are 10.125.1.0, 10.125.2.0, and 10.125.3.0.
This section describes how to create a map of the external network configuration of a peer node. For further information about external addresses, see External Addressing and Networking in Netra High Availability Suite 3.0 1/08 Foundation Services Overview.
Log in to the node whose network interfaces you want to examine.
# ifconfig -a |
The ifconfig command displays configuration information about the network interfaces of a node. The following output is for a diskless or a dataless node running the Solaris OS and using the class C addressing scheme, and configured for external communication through additional logical interfaces.
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 cgtp0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.25.3.25 netmask ffffff00 broadcast 10.25.3.255 ether 0:0:0:0:0:0 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.25.1.25 netmask ffffff00 broadcast 10.25.1.255 ether 8:0:20:f9:a5:56 hme0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.17.1.125 netmask ffffff00 broadcast 10.17.1.255 hme1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 10.25.2.25 netmask ffffff00 broadcast 10.25.2.255 ether 8:0:20:f9:a5:57 |
Search in the output of Step 2 for an interface to an external network.
In this example, the logical interface hme0:1 is an additional interface configured for external addressing.
The master node and vice-master node have three interfaces for the floating address triplet. For example, NIC0:1, NIC1:1, and cgtp0:1. The interfaces are assigned to the master node and vice-master node, but are configured as up on the master node only. If a switchover or failover occurs, the floating address triplet is configured down on the old master node and up on the new master node. For more information see “Cluster Addressing and Networking” in the Netra High Availability Suite 3.0 1/08 Foundation Services Overview.
# nhcmmstat -c all |
The nhcmmstat command also displays information in the console window about each peer node.
# ifconfig -a |
The following table shows sample output for a Solaris OS node using the class C addressing scheme:
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 cgtp0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.25.3.25 netmask ffffff00 broadcast 10.25.3.255 ether 0:0:0:0:0:0 cgtp0:1: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 2 inet 10.25.3.1 netmask ffffff00 broadcast 10.25.3.255 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.25.1.25 netmask ffffff00 broadcast 10.25.1.255 ether 8:0:20:f9:a5:56 hme0:2: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 3 inet 10.25.1.1 netmask ffffff00 broadcast 10.25.1.255 hme1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 10.25.2.25 netmask ffffff00 broadcast 10.25.2.255 ether 8:0:20:f9:a5:57 hme1:1: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 4 inet 10.25.2.1 netmask ffffff00 broadcast 10.25.2.255 |
The floating address triplet has the logical interfaces hme0:2, hme1:1, and cgtp0:1. Note the IP addresses and netmask of the interfaces for the floating address triplet.
The following table shows sample output:
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 cgtp0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.25.3.26 netmask ffffff00 broadcast 10.25.3.255 ether 0:0:0:0:0:0 cgtp0:1: flags=1040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 2 inet 10.25.3.1 netmask ffffff00 broadcast 10.25.3.255 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.25.1.26 netmask ffffff00 broadcast 10.25.1.255 ether 8:0:20:fa:3f:70 hme0:2: flags=1040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 3 inet 10.25.1.1 netmask ffffff00 broadcast 10.25.1.255 hme1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 10.25.2.26 netmask ffffff00 broadcast 10.25.2.255 ether 8:0:20:fa:3f:71 hme1:1: flags=1040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 4 inet 10.25.2.1 netmask ffffff00 broadcast 10.25.2.255 |
Note - Note that the hme0:2, hme1:1, and cgtp0:1 interfaces of the vice-master are configured but are not marked UP. |
The External Address Manager (EAM) and IP MultiPathing (IPMP) both manage the external floating addresses. The external floating addresses are configured, but are initially in DOWN state. EAM is the program that changes the state to UP on the master node.
# nhcmmstat -c all |
The nhcmmstat command also displays information about each peer node in the console window.
Verify that the EAM has been configured in the nhfs.conf file.
For a node using the class C addressing scheme, the EAM configuration is similar to the following examples.
Node.External.FloatingAddress.0=12.10.10.1 Node.External.Monitor.Group.0=ext_group |
In this example, a floating external address is configured on the master node. The floating external address is 12.10.10.1, and the physical interface belongs to the IPMP group ext_group. The logical interface hme2 has IPMP’s test address, which is 12.10.10.225. Note that both addresses belong to the same subnetwork. After failover or switchover, the floating external address is configured on the new master node.
| Node.External.FloatingAddress.0=12.10.10.1 |
| Node.External.Monitor.Group.0=bond0 eth2 eth3 |
In this example, a floating external address is configured on the master node. The floating external address is 12.10.10.1, and the physical interfaces eth2 and eth3 are attached to the interface bond0. All three interfaces have the same IP address, 12.10.10.1.
Run the ifconfig command on the master node:
# ifconfig -a |
The following table shows sample output for the Solaris OS:
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 cgtp0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.25.3.25 netmask ffffff00 broadcast 10.25.3.255 ether 0:0:0:0:0:0 cgtp0:1: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 2 inet 10.25.3.1 netmask ffffff00 broadcast 10.25.3.255 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.25.1.25 netmask ffffff00 broadcast 10.25.1.255 ether 8:0:20:f9:a5:56 hme0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.17.1.125 netmask ffffff00 broadcast 10.17.1.255 hme0:2: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 3 inet 10.25.1.1 netmask ffffff00 broadcast 10.25.1.255 hme1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 10.25.2.25 netmask ffffff00 broadcast 10.25.2.255 ether 8:0:20:f9:a5:57 hme1:1: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 4 inet 10.25.2.1 netmask ffffff00 broadcast 10.25.2.255 hme2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 5 inet 12.10.10.225 netmask ffffff00 broadcast 12.10.10.255 groupname ext_group ether 0:3:ba:31:c2:5e hme2:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5 inet 12.10.10.1 netmask ffffff00 broadcast 12.10.10.255 hme3: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 6 inet 12.10.10.235 netmask ffffff00 broadcast 12.10.10.255 groupname ext_group ether 0:3:ba:31:c2:5f |
This output shows the hme2:1 interface as configured in the nhfs.conf file.
On Linux, the sample output for a master node appears as follows:
bond0 Link encap:Ethernet HWaddr 00:03:BA:F1:76:1A inet addr:12.10.10.1 Bcast:12.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:761a/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) cgtp0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:10.125.3.10 Bcast:10.125.3.255 Mask:255.255.255.0 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:238077 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:310149562 (295.7 MiB) cgtp0:0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:10.125.3.1 Bcast:10.125.3.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0 Link encap:Ethernet HWaddr 00:03:BA:F1:76:18 inet addr:10.125.1.10 Bcast:10.125.1.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:7618/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:615003 errors:0 dropped:21216 overruns:0 frame:0 TX packets:124213 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:840327200 (801.3 MiB) TX bytes:155735404 (148.5 MiB) Interrupt:32 eth0:0 Link encap:Ethernet HWaddr 00:03:BA:F1:76:18 inet addr:10.125.1.1 Bcast:10.125.1.255 Mask:255.255.255.0 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 Interrupt:32 eth1 Link encap:Ethernet HWaddr 00:03:BA:F1:76:19 inet addr:10.125.2.10 Bcast:10.125.2.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:7619/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:722144 errors:0 dropped:17459 overruns:0 frame:0 TX packets:113864 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:843923725 (804.8 MiB) TX bytes:154414158 (147.2 MiB) Interrupt:33 |
eth1:0 Link encap:Ethernet HWaddr 00:03:BA:F1:76:19 inet addr:10.125.2.1 Bcast:10.125.2.255 Mask:255.255.255.0 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 Interrupt:33 eth2 Link encap:Ethernet HWaddr 00:03:BA:F1:76:1B inet addr:12.10.10.1 Bcast:12.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:761b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:247 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:15826 (15.4 KiB) TX bytes:158 (158.0 b) Interrupt:26 eth2 Link encap:Ethernet HWaddr 00:03:BA:F1:76:1B inet addr:12.10.10.1 Bcast:12.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:761b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:247 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:15826 (15.4 KiB) TX bytes:158 (158.0 b) Interrupt:26 eth3 Link encap:Ethernet HWaddr 00:03:BA:F1:76:1B inet addr:12.10.10.1 Bcast:12.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:761b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:248 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:15872 (15.5 KiB) TX bytes:462 (462.0 b) Interrupt:27 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:7149 errors:0 dropped:0 overruns:0 frame:0 TX packets:7149 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:956060 (933.6 KiB) TX bytes:956060 (933.6 KiB) |
This output shows the bond0 interface as configured in the nhfs.conf file.
Run the ifconfig command on the vice-master node:
# ifconfig -a |
The following table shows sample output for the Solaris OS:
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 cgtp0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.25.3.26 netmask ffffff00 broadcast 10.25.3.255 ether 0:0:0:0:0:0 cgtp0:1: flags=1040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 2 inet 10.25.3.1 netmask ffffff00 broadcast 10.25.3.255 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.25.1.26 netmask ffffff00 broadcast 10.25.1.255 ether 8:0:20:fa:3f:70 hme0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 10.17.1.126 netmask ffffff00 broadcast 10.17.1.255 hme0:2: flags=1040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 3 inet 10.25.1.1 netmask ffffff00 broadcast 10.25.1.255 hme1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 10.25.2.26 netmask ffffff00 broadcast 10.25.2.255 ether 8:0:20:fa:3f:71 hme1:1: flags=1040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 4 inet 10.25.2.1 netmask ffffff00 broadcast 10.25.2.255 hme2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 5 inet 12.10.10.226 netmask ffffff00 broadcast 12.10.10.255 groupname ext_group ether 0:3:ba:31:c2:77 hme2:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5 inet 12.10.10.1 netmask ffffff00 broadcast 12.10.10.255 hme3: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 5 inet 12.10.10.236 netmask ffffff00 broadcast 12.10.10.255 groupname ext_group ether 0:3:ba:31:c2:78 |
This output shows that the external floating address is configured but in a DOWN state.
On Linux, the sample output for a vice-master node appears as follows:
bond0 Link encap:Ethernet HWaddr 00:03:BA:F1:7B:5A inet addr:12.10.10.1 Bcast:12.10.10.255 Mask:255.255.255.0 BROADCAST MASTER MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) cgtp0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:10.125.3.20 Bcast:10.125.3.255 Mask:255.255.255.0 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:137149 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:17165584 (16.3 MiB) eth0 Link encap:Ethernet HWaddr 00:03:BA:F1:7B:58 inet addr:10.125.1.20 Bcast:10.125.1.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:7b58/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:16230 errors:0 dropped:0 overruns:0 frame:0 TX packets:68990 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2442974 (2.3 MiB) TX bytes:8636946 (8.2 MiB) Interrupt:32 eth1 Link encap:Ethernet HWaddr 00:03:BA:F1:7B:59 inet addr:10.125.2.20 Bcast:10.125.2.255 Mask:255.255.255.0 inet6 addr: fe80::203:baff:fef1:7b59/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:190888 errors:0 dropped:0 overruns:0 frame:0 TX packets:68159 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:260118091 (248.0 MiB) TX bytes:8528638 (8.1 MiB) Interrupt:33 eth2 Link encap:Ethernet HWaddr 00:03:BA:F1:7B:5B inet addr:12.10.10.1 Bcast:12.10.10.255 Mask:255.255.255.0 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:26 |
eth3 Link encap:Ethernet HWaddr 00:03:BA:F1:7B:5B inet addr:12.10.10.1 Bcast:12.10.10.255 Mask:255.255.255.0 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:27 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:840 errors:0 dropped:0 overruns:0 frame:0 TX packets:840 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:131099 (128.0 KiB) TX bytes:131099 (128.0 KiB) |
This output shows that the external floating address is configured, but in a DOWN state on eth2 and eth3.
Each interface of each peer node must have the following configuration information:
An /etc/hostname.interface-name file or an /etc/nodename file if the node is running the Solaris OS
An entry in /etc/network/interfaces if the node is running Linux
Examine these files to understand the network configuration of a peer node. To find the name of the current node, see Mapping the Network Configuration of a Peer Node.
The network configuration files contain the following example information. In the following examples, the node is running the Solaris OS and the name is node25.
/etc/hostname.hme0 and /etc/hostname.hme1
These files define the host names, IP addresses, and attributes for the hme0 and hme1 interface on a node.
This file defines the host name of the cgtp0 interface on a node.
This file contains a list of host names. For a cluster of three nodes, using the default class C addressing scheme, the /etc/hosts file is as follows:
| 127.0.0.1 localhost |
Peer node 1 address triplet (current node):
| 10.250.1.20 MEN-C250-N20 MEN-C250-N20.localdomain loghost |
| 10.250.1.10 MEN-C250-N10 |
| 10.250.1.30 NMEN-C250-N30 |
| 172.36.128.51 sol52 |
| 10.250.1.1 master-nic0 |
At startup, the Cluster Membership Manager (CMM) creates a routing table for the interfaces on each peer node. Carrier Grade Transport Protocol (CGTP) uses the routing table for data replication. To examine the routing table for a peer node, perform the following procedure. For simplicity, this procedure does not show external network access.
Note - Because CTGP on Linux uses ARP requests to resolve the CGTP destination and does not require entries in the routing table, the following section is applicable only for Solaris nodes. |
Display the routing table for the node:
# netstat -r |
The following table shows sample output for a node using the class C addressing scheme:
Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ------ --------- node26-cgtp node26 UGHMS 1 8 node26-cgtp node26-nic1 UGHMS 1 8 10.25.3.255 10.25.1.255 UGHMS 1 0 10.25.3.255 10.25.2.255 UGHMS 1 0 node32-cgtp node32 UGHMS 1 4 node32-cgtp node32-nic1 UGHMS 1 4 master-cgtp master-nic0 UGHMS 1 1 master-cgtp master-nic1 UGHMS 1 1 12.10.10.0 ipmp-host U 1 9826 hme2:1 12.10.10.0 ipmp-host U 1 0 hme2 10.25.3.0 node25-cgtp U 1 0 cgtp0 10.25.3.0 node25-cgtp U 1 0 cgtp0:1 10.25.2.0 node25-nic1 U 1 7 hme1 10.25.2.0 node25-nic1 U 1 0 hme1:1 10.25.1.0 node25 U 1 7 hme0 10.25.1.0 node25 U 1 0 hme0:2 10.17.1.0 l6-t1-25 U 1 3 hme0:1 224.0.0.0 node25-cgtp U 1 0 cgtp0 default 10.17.1.1 UG 1 4 localhost localhost UH 4 40 lo0 |
For a description of the column headings, see Output of the netstat r Command.
From the sample output, you can conclude the following facts:
This is a three-node cluster. The cluster contains the nodes node26-cgtp, node32-cgtp, and the current node.
The CGTP routes to broadcast 10.25.3.255 are as follows:
Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ------ --------- 10.25.3.255 10.25.1.255 UGHMS 1 0 10.25.3.255 10.25.2.255 UGHMS 1 0 |
The CGTP routes to node 26 are as follows:
Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ------ --------- node26-cgtp node26 UGHMS 1 8 node26-cgtp node26-nic1 UGHMS 1 8 |
The CGTP routes to the CGTP floating address, cgtp0:1, are as follows:
Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ------ --------- master-cgtp master-nic0 UGHMS 1 1 master-cgtp master-nic1 UGHMS 1 1 |
The CGTP routes to node 32 are as follows:
Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ------ --------- node32-cgtp node32 UGHMS 1 4 node32-cgtp node32-nic1 UGHMS 1 4 |
The other entries in the table are relative to the external addresses (floating and static) and the standard Solaris routes.
To change the entries in the routing table, use the route command as described in the route1M man page.
The following table explains the output of the -netstat -r command.
The flags D, G, H, and U are part of standard Solaris routing. The flags M and S are used for CGTP. The flags configure the way in which data is replicated, as follows:
When debugging diskless nodes, use the snoop command for Solaris systems, or the tcpdump command for Linux systems, as follows.
Find the IP address of this node:
% ifconfig -a |
The ifconfig command returns the IP address, the interface type, and the Ethernet address.
From a console window, examine the first Ethernet address.
% snoop -d NIC0 ether diskless-node-NIC0-Ethernet-address |
| % tcpdump -i NIC0 ether host diskless-node-NIC0-Ethernet-address |
The snoop or tcpdump command captures packets from the network and displays their contents.
From another console window, examine the next Ethernet address.
% snoop -d NIC1 ether diskless-node-NIC1-Ethernet-address |
| % tcpdump -i NIC1 ether host diskless-node-NIC1-Ethernet-address |
| Netra™ High Availability Suite 3.0 1/08 Foundation Services Cluster Administration Guide | 819-5235-13 |
|
Copyright © 2008, Sun Microsystems, Inc. All rights reserved.
To Verify the Network Interfaces
of a Node