Introducing CGTP Configuration

This chapter describes the steps that you perform on nodes that are to run standalone CGTP. All examples in this chapter describe the configuration of two nodes, machine A and machine B, as shown in FIGURE 5-1. When configured, these nodes communicate through dual network links that form a CGTP network. You must decide the number of nodes that you want to configure to be part of this CGTP network.

FIGURE 5-1 CGTP Configuration for machine A and machine B

Figure shows CGTP configuration between two
machines, machine A and machine B.

Preparing to Configure Standalone CGTP on Linux

To have a CGTP network, each node on the network must be configured to have three interfaces:

Two physical interfaces, NIC0 and NIC1.

One virtual physical interface, cgtp0.

Configure CGTP on the node's local file system. Do not configure CGTP on a remote shared file system because a remote file system might become unreachable if the network fails. Before configuring CGTP interfaces and routes, configure the interface names, addresses, netmasks, and the database search on each node.

To Configure Node Addresses on MontaVista Linux 4.0

Perform the following procedure on each node that you are configuring for standalone CGTP.

Manually configure the node addresses so that the interfaces are created at boot.

Note - This procedure is valid only for MontaVista Linux 4.0. If you are using any other distribution of Linux, follow the procedures provided in the documentation that accompanied the distribution.

Log in to the node that you want to configure for CGTP.

Define the configuration for NIC0, NIC1, and cgtp0 in the /etc/network/interfaces file. The following interfaces and addresses are examples that you can modify to meet your needs.

# CGTP’s NIC0
auto eth2
iface eth2 inet static
address 10.217.1.10
network 10.217.1.0
netmask 255.255.255.0
broadcast 10.217.1.255
 
# CGTP’s NIC1
auto eth3
iface eth3 inet static
address 10.217.2.10
 network 10.217.2.0
netmask 255.255.255.0
broadcast 10.217.2.255
 
# CGTP interface
auto cgtp0
iface cgtp0 inet static
address 10.217.3.10
network 10.217.3.0
netmask 255.255.255.0
broadcast 10.217.3.255
up echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter || true
up /opt/sun/sbin/cgtptool cgtp0 eth2 eth3

Configure the autoloading at boot time of the CGTP module.

Add one line to the /etc/modules file with the module name.
cgtp

Reboot the node.

Check that the node addresses are automatically assigned to the interface:
# ifconfig -a
For more information on the ifconfig command, see the ifconfig8 man page.

To Configure Node Addresses on Wind River Linux CGL 1.4

Manually configure the node addresses so that the interfaces are created at boot.

Note - This procedure is valid only for Wind River Linux CGL 1.4. If you are using any other distribution of Linux, follow the procedures provided in the documentation that accompanied the distribution.

Log in to the node that you want to configure for CGTP.

Define the configuration for NIC0, NIC1, and cgtp0 in the /etc/network/interfaces file. The following interfaces and addresses are examples that you can modify to meet your needs.

 File /etc/sysconfig/network-scripts/ifcfg-eth0
# CGTP’s NIC0
DEVICE=eth0
File  /etc/sysconfig/network-scripts/ifcfg-eth1
BOOTPROTO=static
# CGTP’s NIC1DEVICE=eth1
BOOTPROTO=static
BROADCAST=10.217.2.255
IPADDR=10.217.2.10
NETMASK=255.255.255.0
NETWORK=10.217.2.0
STARTMODE=onboot
BROADCAST=10.217.1.255
File /etc/sysconfig/network-scripts/ifcfg-cgtp0
IPADDR=10.217.1.10
# CGTP interfaceDEVICE=cgtp0
BOOTPROTO=static
BROADCAST=10.217.3.255
IPADDR=10.217.3.10
NETMASK=255.255.255.0
NETWORK=10.217.3.0STARTMODE=onboot
NETMASK=255.255.255.0
NETWORK=10.217.1.0STARTMODE=onboot

Configure the autoloading at boot time of the CGTP module. Add one line to the /etc/modules file with the module name.
cgtp

Reboot the node.

Check that the node addresses are automatically assigned to the interface:
# ifconfig -a
For more information on the ifconfig command, see the ifconfig8 man page.

Configuring Additional CGTP Virtual Logical Interfaces on Linux

In addition to the cgtp0 interface, you can configure additional CGTP interfaces. These additional interfaces are virtual logical interfaces that are associated with cgtp0, such as cgtp0:1 and cgtp0:2. You might want to create such an additional interface if you want to have services using different interfaces, but want to maintain the redundancy.

The cgtp0 and cgtp0:x interfaces on a node must have the same subnet prefix. This means that the primary CGTP address and the logical, or aliased, addresses must only differ by their hostid. Routes to the aliased interfaces can accordingly be set up on the other cluster nodes.

To Configure Additional CGTP Interfaces

As superuser, log in to machine A.

Create the logical interface for CGTP:
# ifconfig cgtp0 add ip-address \netmask netmask broadcast broadcast-address
The add option creates the next unused logical interface associated with the interface you specified. For example, if you specify cgtp0, the next unused logical interface is cgtp0:0. For further information, see the ifconfig8 man page.

Alternatively, you can still specify the logical interface name by explicitly configuring it:
# ifconfig cgtp0:10 ip-address \netmask netmask broadcast broadcast-address up

As superuser, repeat Step 2 for machine B.

Ensuring Interoperability of the Solaris OS and Linux

CGTP for Linux provides a way to resolve the differences that exist in the way the Solaris OS and Linux manage the Address Resolution Protocol (ARP). This section describes how to set up CGTP on Linux when Linux nodes must interoperate with Solaris nodes using CGTP.

Because of the differences in the way the Solaris OS and Linux handle the ARP, CGTP is unable to find the MAC addresses of remote Solaris nodes. To solve this problem, the redundant gateways table must be populated with information that explains how to reach those nodes. An entry in this table is made of the following, globally unique, triplet:

Destination CGTP IP address
Gateway IP address
Network interface through which the gateway address must be resolved.

Generally, there should be two entries associated to the same destination CGTP IP address. The redundant gateways table is accessible through the /proc/net/cgtp interface/gateway file.

To Add Nodes to the Redundant Gateways Table

As superuser, log in to the Linux node.

Add an entry to the file as follows:

# echo "add dest IP addr gateway
IP addr slave interface" > \
/proc/net/cgtp0/gateway

To Remove Nodes to the Redundant Gateways Table

As superuser, log in to the Linux node.

Add an entry to the file as follows:

# echo "del <dest IP addr> <gateway IP addr> <slave interface>" > \
/proc/net/cgtp0/gateway

To Change the IP Address of an Existing Entry

As superuser, log in to the Linux node.

Change the entry in the file as follows:

# echo "change <dest IP addr> <gateway IP addr> <slave int.>" > \
/proc/net/cgtp0/gateway

To Get the Contents of the Redundant Gateways Table

As superuser, log in to the Linux node.

Read the contents of the file, as follows:

# cat /proc/net/cgtp0/gateway
Device     Dest. IP addr  Gateway IP addr
-----------------------------------------
eth0           11.0.3.15        11.0.1.15
eth1           11.0.3.15        11.0.2.15

In this example, a packet being sent over the CGTP link to the destination IP address 11.0.3.15 will be duplicated on the interfaces eth0 and eth1 as if it was a packet being sent to 11.0.1.15 and 11.0.2.15, respectively.

Note - Addresses must be supplied in numeric format. No name resolution is supported.

Disabling CGTP on Linux

This section describes how to disable the CGTP interfaces on two machines where each machine is attached to two subnets.

To Disable the Virtual CGTP Interface cgtp0

As superuser, log in to machine A.

Take the interface down:
# ifdown cgtp0

As superuser, repeat Step 2 for machine B.

Configuring Standalone CGTP on Linux

Introducing CGTP Configuration

Preparing to Configure Standalone CGTP on Linux

Configuring Additional CGTP Virtual Logical Interfaces on Linux

Ensuring Interoperability of the Solaris OS and Linux

Disabling CGTP on Linux