Sun Fire B1600 Blade System Chassis Software Setup Guide
|
   
|
| Sun Fire B1600 Blade System Chassis Software Setup Guide
|
|
| C H A P T E R 5 |
|
Installing a Chassis Containing B100s Blades into Separated Data and Management Networks |
This chapter contains the following sections:
|
Note - For information about setting up network redundancy on a chassis containing Linux and/or Solaris x86 blades, refer to the Sun Fire B100x and B200x Server Blade Installation and Setup Guide. |
This chapter tells you how to set up the Sun Fire B1600 blade system chassis for use in an environment that separates the data and management networks. If you have dual SSCs installed in the chassis, the instructions enable you to take advantage of the presence of two switches to give your SPARC server blades two connections each to your network.
FIGURE 5-1 shows a sample network containing a Sun Fire B1600 blade system chassis, and the subsequent sections use this diagram and the IP addresses marked on it to illustrate the steps you need to perform.
If you have Solaris blades installed in the chassis, the sample /etc/hosts (CODE EXAMPLE 5-1) and /etc/netmasks (CODE EXAMPLE 5-2) files provided in this chapter illustrate how to edit the files on your Name Server to simplify the process of configuring Solaris on the blades. Use these sample administration files for guidance, substituting your own IP addresses and host names for the ones used in the sample network illustrated in FIGURE 5-1.
|
Note - As noted in Chapter 3, if you have dual SSCs installed, then, when you are considering how to integrate the chassis into your network environment, you need to remember that the chassis contains two switches. Although only one of its System Controllers is active at any one time, both of its switches are active all the time. This means that, in a system chassis that is working normally, both switches are providing the server blades with continuous network connectivity. However, if for any reason one switch fails, the other switch continues to provide network connectivity. (Also, if either System Controller fails, the switch inside the same SSC module continues to provide network connectivity; the switches operate independently of the System Controllers even though they are physically located in the same enclosure.) |
This chapter now tells you how to take advantage of the presence of two switches by using VLANs in conjunction with IPMP (IP Network Multipathing) on your B100s (SPARC Solaris) blades to provide fully redundant connections from Solaris server blades to the data and management networks.
If you have Linux blades installed, you cannot currently take advantage of the presence of two switches because the bonding facility (this is the Linux equivalent of IPMP) is not currently available.
To take advantage of the redundancy offered by the second switch inside the system chassis, we recommend you to:
If you are using DHCP, make sure that the DHCP server for the System Controllers and switches is on the management network, and that the DHCP server for the blades is on the data network.
For information about setting up the Network Install Servers and DHCP servers for B100s blades, see Chapter 1, Chapter 3, and Appendix B.
FIGURE 5-1 shows a network similar to the sample configuration in Chapter 3 but with the 100Mbps network management port (NETMGT) on both SSCs now connected to a different switch from the data uplink ports. This new external switch is on a different subnet from the switch that the data uplink ports on the chassis are connected to. It is a subnet dedicated to network management traffic and it therefore also contains both of the System Controllers and switches in the chassis. A management VLAN (VLAN 2) contains both System Controller interfaces and both switch management ports. All the server blades and uplink ports are on VLAN 1.
FIGURE 5-1 also shows the connection of the ce0 interface on each blade to the switch in SSC0, and the connection of the ce1 interface on each blade to the switch in SSC1. Note that each server blade interface now has four IP addresses associated with it instead of one. These four addresses are used by the IPMP driver to enable the interfaces to function as redundant connections (see Section 5.5, Setting up SPARC Solaris Server Blades Using IPMP for Network Resiliency).
As in FIGURE 3-1 (see Chapter 3), one or more of the eight uplink ports on each switch in FIGURE 5-1 are connected to an external switch that has an Install Server (also containing a Name Server) connected to it. This external switch also has a router (with IP address 192.168.1.1) connected to it that acts as the default gateway from the chassis to the wider network.
|
Note - that there is no direct network connection in FIGURE 5-1 from the management port (NETMGT) in the switch to the server blade ports. This means that, by default, you cannot manage the server blades directly from the management network. This is a security feature to protect the management network from the possibility of hostile attack from the data network. For information about permitting specified traffic from the server blades to the management port, see Appendix A and Chapter 6. |
Before installing the Sun Fire B1600 blade system chassis into an environment like the one illustrated in FIGURE 5-1 (in other words, one that separates the data and management networks), you need to edit the /etc/hosts, /etc/ethers, and /etc/netmasks files on your Solaris Name Servers on the data and management networks:
|
Note - For each B100s server blade, only the published IP addresses (not the test IP addresses used by IPMP) need to be registered in the /etc/hosts file on the name server. However, the test addresses for each blade must be clearly reserved in a comment so that other network administrators know they are not available for use (see CODE EXAMPLE 5-1). |
[ D ]
To configure the System Controllers and switches for the type of configuration illustrated in FIGURE 5-1, follow the instructions in Section 3.4, Configuring the System Controllers and Switches. However, remember that the IP addresses you assign to the System Controllers and switches need to be on the management subnet.
The instructions in this section tell you how to use the Solaris IP Network Multipathing (IPMP) facility to take advantage of the redundant connections from each server blade to the switches in the chassis. A server blade's two 1000Mbps Ethernet interfaces are labeled respectively ce0 and ce1 (ce0 is connected to the switch in SSC0, and ce1 is connected to the switch in SSC1). When the Sun Fire B1600 blade system chassis is fully operational, both switches are constantly active.
The IPMP driver on the server blade works by periodically pinging the default gateway from both Ethernet interfaces. If for any reason one of the pings fails (indicating that the path to the network is no longer available on the interface that was used to perform the ping) the IPMP driver ensures that network traffic uses only the interface that remains valid. Both interfaces can be active (in which case they each require a separate IP address). Alternatively one can be a standby interface that takes over the IP address of the active one if that interface fails.
The active/active configuration requires four IP addresses: one for each interface plus one test address for each interface. The active/standby configuration requires three IP addresses. In either case two test addresses are used privately by the IPMP driver to perform the ping process. If it does not receive a reply from a ping on the test address associated with one interface it knows that that interface has failed and it directs all network traffic for either interface over the valid one. If you have an active/active configuration, it simply stops using the non-valid interface. If you have an active/standby configuration and the non-valid interface is the active one, it assigns the IP address to the standby interface which now becomes the active interface.
Because both switches inside the chassis are active (when the chassis is working normally), the instructions in this chapter tell you how to perform an active/active configuration. For information about performing an active/standby configuration, refer to the IP Network Multipathing Administration Guide (816-0850).
The IP addresses you require for each physical interface on a blade are as follows:
The primary and secondary IP addresses are (or can) both be registered on a Name Server. They are the addresses by which other devices on the network communicate with a blade.
In this chapter, the instructions tell you how to set up IPMP for two physical interfaces. In the next chapter, instructions are provided for setting up multiple pairs of virtual IPMP interfaces, each pair providing redundant interfaces to separate VLANs.
This section tells you how to configure IPMP on a server blade so that the two Ethernet interfaces both actively transmit and receive data. For purposes of illustration the instructions use sample configuration input from the network scenario described in Section 5.3, Preparing the Network Environment Using Static IP Addresses.
TABLE 5-1 summarizes the information you would need to give the IPMP driver on the server blade in Slot 0 of the system chassis illustrated in FIGURE 5-1.
|
Note - You need to perform the instructions in this section on each server blade that requires a redundant connection to the network. |
1. Perform a preliminary setup of Solaris by following the instructions in Chapter 3.
When you have done this, type #. to return from a server blade console to the sc> prompt.
2. Log in as root to the console of the server blade whose interfaces you want to configure.
Type the following at the sc> prompt:
where n is the number of the slot containing the server blade you want to log into.
3. Edit the /etc/hosts file on the server blade itself to add the blade's two test IP addresses.
For a blade using the sample addresses in TABLE 5-1, you would need to add the last two lines of the following file:
4. Set the netmask in the server blade's /etc/netmasks file.
For a blade using the sample addresses in TABLE 5-1, you would need to add the folowing line:
5. Disable routing, because the server blade is not being used to perform routing.
6. Create the network interfaces by typing:
7. Create an IPMP group named medusa_grp0 containing network interfaces ce0 and ce1:
When you execute these commands, you might see the following syslog messages:
Sep 3 00:49:58 medusa-s0 in.mpathd[298]: Failures cannot be detected on ce0 as no IFF_NOFAILOVER address is available |
These messages simply warn that failures cannot be detected until test addresses have been established on the interfaces.
8. Create an address on ce0 and ce1 for data transmission and mark it to failover if an interface failure is detected.
9. Configure a test address on each network interface.
These will be used by mpathd to detect interface failures. You need to use the
-failover flag. This causes in.mpathd to use the address as a test address (in other words, an address that cannot pass to the other interface and therefore does not fail over):
10. To enable the new interface configuration to survive a reboot, create a hostname.ce0 and a hostname.ce1 file in the /etc directory.
A sample file for hostname.ce0 is as follows:
medusa-s0 netmask + broadcast + \ group medusa_grp0 up \ addif medusa-s0-0 deprecated -failover \ netmask + broadcast + up |
A sample file for hostname.ce1 is as follows:
medusa-s0-sec netmask + broadcast + \ group medusa_grp0 up \ addif medusa-s0-1 deprecated -failover \ netmask + broadcast + up |
11. Inspect the configuration of the two network adapters.
The output above shows that four addresses have been defined (the sample addresses from TABLE 5-1). The two IPMP test addresses (associated with ce0:1 and ce1:1 respectively) are marked NOFAILOVER. This means that they will not be transferred to the surviving interface in the event of a failure.
12. Test IPMP by temporarily removing one SSC from the chassis.
This will cause the following error messages to be displayed on the console:
Sep 3 01:08:50 medusa-s0 in.mpathd[29]: NIC failure detected on ce0 of group medusa_grp0 Sep 3 01:08:50 medusa-s0 in.mpathd[29]: Successfully failed over from NIC ce0 to NIC ce1 |
| Sun Fire B1600 Blade System Chassis Software Setup Guide
|
817-4603-11 |
|
Copyright © 2004, Sun Microsystems, Inc. All rights reserved.