Sun Fire B100x and B200x Server Blade Installation and Setup Guide

This chapter provides the information you need to install Linux on a B100x or B200x server blade. It contains the following sections:

• Section 4.1, PXE Overview
• Section 4.2, Installing Linux From a Linux PXE Boot Server
• Section 4.3, Installing Linux From a Solaris PXE Boot Server

---

4.1 PXE Overview

The Preboot Execution Environment (PXE) is a method of network booting blade and cluster systems. It is the core technology for Intel's Wired for Management (WfM) initiative and is supported by most commercial network interfaces. You can install a blade operating system image with minimal effort from a central location by using PXE.

To install Linux onto a server blade using PXE you will need the following:

• A PXE boot server machine. This machine must be running one of the following operating systems:
• Red Hat Enterprise Linux, Advanced Server 2.1 update 2
• Red Hat Enterprise Linux, version 3.0
• SuSE Linux Enterprise Server 8, service pack 3
• Solaris, version 9 or later

• A server blade (without the operating system installed).
• The Sun Fire B1600 Platform Documentation, Drivers, and Installation CD supplied by Sun with the server blade.
• Installation CDs for the version of Linux you are installing. You can install one of the following:
• Red Hat Enterprise Linux, Advanced Server 2.1 update 2
• Red Hat Enterprise Linux, version 3.0
• SuSE Linux Enterprise Server 8, service pack 3

4.1.1 PXE Protocols

PXE comprises three distinct network protocols:

• Dynamic Host Configuration Protocol (DHCP)
• Trivial File Transfer Protocol (TFTP)
• Network File System (NFS)

These protocols allow delivery of system configuration information in addition to system software for blades. See TABLE 4-1 for further details.

The PXE standard also specifies a client side BIOS programming interface called UNDI. This API abstracts ethernet devices to allow x86 based systems to implement simple, network-based bootstrap loaders.

Universal Network Driver Interface (UNDI) is a programming API that simplifies network programming. All network interface cards that support PXE network booting can be controlled using the API. This provides the bootstrap mechanism with a universal method for accessing network cards.

---

4.2 Installing Linux From a Linux PXE Boot Server

This section tells you how to install Linux on a B100x or B200x server blade from a PXE boot server running Linux.

The PXE boot server must be running one of the following versions of Linux:

• Red Hat Enterprise Linux, Advanced Server 2.1 update 2
• Red Hat Enterprise Linux, version 3.0
• SuSE Linux Enterprise Server 8 service pack 3

4.2.1 Files Relevant to PXE Boot Installation

TABLE 4-2 provides a summary of the files required during the PXE boot installation:

4.2.2 Configuring the PXE Boot Servers

Linux is installed on the server blade using the PXE boot system. Three server processes are required to perform the installation:

• DHCP
• TFTP
• NFS

This section provides information on how to configure the DHCP, TFTP and NFS servers for use with the PXE boot installation.

4.2.2.1 Configuring the DHCP Server

The DHCP server supplies the PXE boot plug-in with:

• IP address
• TFTP server address
• Stage 1 image boot-loader name from which to download and execute the image.

Use the dhcp package provided with the version of Linux you are installing to provide DHCP services.

1. Update the /etc/dhcpd.conf file:

a. Add a subnet section with next-server referring to your TFTP server.

b. Change the filename entry to /<Linux_dir>/sun/pxelinux.bin

where <Linux_dir> is either as-2.1u2, el-3.O, or sles-8sp3, depending on the version of Linux you are installing.

c. If you are installing Red Hat Enterprise Linux Advanced Server 2.1 update 2, remove the line ddns-update-style none;. (This line is required when installing all other versions of Linux).

2. Enable the DHCP server.

For Red Hat, type:

/sbin/chkconfig --level 345 dhcpd on

For SuSE, type:

chkconfig dhcpd on

3. Restart the DHCP server:

/etc/init.d/dhcpd restart

4. Validate the configuration:

# netstat -an | fgrep -w 67

The output should be:

udp        0      0 0.0.0.0:67              0.0.0.0:*

Example of the dhcpd.conf File

CODE EXAMPLE 4-1 shows a sample /etc/dhcpd.conf file

ddns-update-style none;

default-lease-time 1800;

max-lease-time 3600;

option domain-name              "linux.sun.com";

option domain-name-servers      172.16.11.2, 172.16.11.8;

option subnet-mask              255.255.0.0;

allow bootp;

allow booting;

option ip-forwarding    false;  # No IP forwarding

option mask-supplier    false;  # Don't respond to ICMP Mask req

get-lease-hostnames     on;     # DNS lookup hostnames

use-host-decl-names     on;     # And supply them to clients

option routers 172.16.11.6;

# WARNING: This is a default configuration -- any system PXE booting will

#          wipe out all existing data on the first hard disk and install

#          Linux 

subnet 172.16.11.0 netmask 255.255.0.0 { 

  next-server 172.16.11.8;                   # name of your TFTP server

  filename "/<linux_dir>/sun/pxelinux.bin";    # name of the boot-loader program 

  range 172.16.11.100 172.16.11.200;         # dhcp clients IP range

}

The important areas in this example are the address of the TFTP server
(next-server 172.16.11.8) and the filename of the stage 1 bootloader image (filename "/<linux_dir>/sun/pxelinux.bin").

4.2.2.2 Configuring the TFTP Server

The TFTP server supplies the PXE boot with the stage 1 bootloader image. This image loads the installation kernel which performs the actual installation on the hard disk through the use of the custom initrd.img supplied by Red Hat.

Use the tftp-server package provided with your Linux distribution to provide TFTP services.

1. Create the TFTP directory. Ensure that all users have read/execute access to the TFTP directory:

umask 022

mkdir /tftp

chmod 755 /tftp

2. Modify the /etc/xinetd.d/tftp file (for Red Hat) or the /etc/inetd.conf file (for SuSE) to allow TFTP services:

• If you are installing Red Hat, update the /etc/xinetd.d/tftp file. You need to change the server_args entry to -s /tftp. (The /tftp path is the directory in which the PXE images are copied.)
• If you are installing SuSE, update the /etc/inetd.conf file by inserting the following line:

tftp dgram udp wait root /usr/sbin/in.tftpd in.tftpd -s /tftp

3. If you are installing SuSE skip to Step 4. If you are installing Red Hat, configure the TFTP server to be enabled at installation.

Change the disable entry to disable= no.

4. Enable the TFTP server.

• For Red Hat, type:

chkconfig --level 345 xinetd on

• For SuSE, type:

chkconfig inetd on

5. Restart xinetd (for Red Hat) or inetd (for SuSE):

• For Red Hat, type:

/etc/init.d/xinetd restart

• For SuSE, type:

/etc/init.d/inetd restart

6. Validate the configuration:

# netstat -an | fgrep -w 69

The output should be:

udp        0      0 0.0.0.0:69         0.0.0.0:*

Example of the tftp File for Red Hat

shows an example of the /etc/xinetd.d/tftp file for Red Hat:

# default: off# description: The tftp server serves files using the trivial file transfer#       protocol.  The tftp protocol is often used to boot diskless#       workstations, download configuration files to network-aware printers,#       and to start the installation process for some operating systems.

service tftp

{

	socket_type		= dgram                               protocol	   	= udp

	wait			= yes

	user			= root

	server			= /usr/sbin/in.tftpd

	server_args.. 		= -s /tftp

	disable			= no

}

Example /etc/xinetd.d/tftp file for Red Hat

4.2.2.3 Configuring the NFS Server

The NFS server is used by the installation kernel to read all of the packages necessary to the installation process. The NFS server therefore needs to provide access to the directory structure containing the PXE images.

1. Update the /etc/exports file to include the export for the NFS server.

Insert the following line into file /etc/exports:

/tftp 	*(ro)

2. Enable the NFS server.

• For Red Hat, type:

chkconfig --level 2345 nfs on

• For SuSE, type:

chkconfig nfslock on chkconfig nfsserver on

3. Restart the NFS server.

For Red Hat, type::

/etc/init.d/nfs restart

For SuSe, type:

/etc/init.d/nfslock restart

/etc/init.d/nfsserver restart

4. Validate the configuration:

showmount -e

The output should include the line:

/tftp

4.2.3 Installing Linux on a Server Blade from a Linux PXE Boot Server

Caution - Installing Linux will overwrite any data already on the destination server blade.

1. If you have configured a firewall, make sure that the TFTP, NFS, and DHCP protocols are not filtered on the server to be used as the PXE boot server.

2. Alternatively, disable the firewall and prevent it from running on subsequent reboots.

• To do this, for Red Hat, type:

chkconfig --level 2345 iptables off /etc/init.d/iptables stop

• For SuSE, type:

chkconfig iptables off /etc/init.d/iptables stop

3. Ensure that the DHCP server, NFS server and TFTP server have been configured correctly.

See Section 4.2.2, Configuring the PXE Boot Servers for more information.

4. Install the PXE images onto the TFTP server:

a. Copy the required Linux directory from the root of the Sun Fire B1600 Platform Documentation, Drivers, and Installation CD to the /tftp directory on your PXE boot server:

umask 022

mount /mnt/cdrom

cd /mnt/cdrom

egrep '^<Linux_dir>' filenames.txt | cpio -pumd /tftp/.

cd /

umount /mnt/cdrom

where <Linux_dir> is as-2.1u2, el-3.O or sles-8sp3, depending on the version of Linux you are installing.

b. Install the Linux installation CDs to the /tftp directory on your PXE boot server.

• For Red Hat, you need to install the CDs in reverse order. If you have two Red Hat installation CDs, install Disk 2 first; if you have four, install Disk 4 first. After inserting each CD, type the following command:

umask 022 mount /mnt/cdrom cd /mnt/cdrom tar -cf - . |tar -C /tftp/<Linux_dir> -xf - cd / umount /mnt/cdrom

where <Linux_dir> is as-2.1u2 or el-3.O, depending on the version of Linux you are installing.

• For SuSE Linux Enterprise Server 8 service pack 3, you need to load each image into its own directory rather than into the same directory. This allows the SuSE installer to select the correct packages from each ISO image. Use the following commands:

After inserting the SLES-8 disk:

     mount /mnt/cdrom

     mkdir /tftp/sles-8sp3/SLES-8-i386-RC5-CD1

     cd /mnt/cdrom

     pax -rw . /tftp/sles-8sp3/SLES-8-i386-RC5-CD1

     cd /

     umount /mnt/cdrom

After inserting the first UnitedLinux 1.0 disk:

     mount /mnt/cdrom

     mkdir /tftp/sles-8sp3/UnitedLinux-1.0-i386-RC5-CD1

     cd /mnt/cdrom

     pax -rw . /tftp/sles-8sp3/UnitedLinux-1.0-i386-RC5-CD1

     cd /

     umount /mnt/cdrom

After inserting the second UnitedLinux 1.0 disk:

     mount /mnt/cdrom

     mkdir /tftp/sles-8sp3/UnitedLinux-1.0-i386-RC5-CD2

     cd /mnt/cdrom

     pax -rw . /tftp/sles-8sp3/UnitedLinux-1.0-i386-RC5-CD2

     cd /

     umount /mnt/cdrom

After inserting the third UnitedLinux 1.0 disk:

     mount /mnt/cdrom

     mkdir /tftp/sles-8sp3/UnitedLinux-1.0-i386-RC5-CD3

     cd /mnt/cdrom

     pax -rw . /tftp/sles-8sp3/UnitedLinux-1.0-i386-RC5-CD3

     cd /

     umount /mnt/cdrom

After inserting the first United Linux 1.0 SP 3 disk:

     mount /mnt/cdrom

     mkdir /tftp/sles-8sp3/UnitedLinux-1.0-SP-3-i386-RC4-CD1

     cd /mnt/cdrom

     pax -rw . /tftp/sles-8sp3/UnitedLinux-1.0-SP-3-i386-RC4-CD1

     cd /

     umount /mnt/cdrom

After inserting the second UnitedLinux 1.0 SP 3 disk:

     mount /mnt/cdrom

     mkdir /tftp/sles-8sp3/UnitedLinux-1.0-SP-3-i386-RC4-CD2

     cd /mnt/cdrom

     pax -rw . /tftp/sles-8sp3/UnitedLinux-1.0-SP-3-i386-RC4-CD2

     cd /

     umount /mnt/cdrom

When you have copied all the disks, tie the ISO images together:

 cd /tftp/sles-8sp3

     sh ./create-glue

5. Modify the configuration file to specify the address of your NFS server.

• For Red Hat, modify the /tftp/<Linux_dir>/sun/install/ks.cfg file. For example:

nfs --server 172.16.13.8 --dir /tftp/<Linux_dir>/mount -t nfs -o nolock 172.16.13.8:/tftp/<Linux_dir> /mnt

where <Linux_dir> is as-2.1u2 or el-3.O, depending on the version of Red Hat you are installing.

• For SuSE, modify the /tftp/sles-8sp3/sun/install/autoyast.xml file to set the NFS server address. A sample command is as follows:

mount -t nfs -o nolock 172.16.11.8:/tftp/sles-8sp3 $MOUNTPT install: nfs://172.16.11.8/sles-8sp3 <server>172.16.11.8</server>

6. Set your own root password in the Linux configuration file.

• For Red Hat, modify the /tftp/<Linux_dir>/sun/install/ks.cfg file by removing the comment symbol (#) in the rootpw entry and then overwriting changeme with your own password:

#rootpw changeme

For example:

rootpw nnnnnnn

where nnnnnnn is your root password.

• For SuSE, specify the root password in the autoyast.xml file (/tftp/sles-8sp3/sun/install/autoyast.xml) by scrolling to the user password section of the file, removing the existing text between the <user_password> key words, typing the password you want to use:

<user> <encrypted config:type="boolean">true</encrypted> <!-- Define the root password here using the <user_password> --> <!-- tag. The specified password must be encrypted... Use --> <!-- the following command to get the encrypted form of (for --> <!-- example) a password of ``changeme'': --> <!-- perl -e 'print crypt("changeme", "/."), "\n"' --> <user_password>/.hz7/JN74p1I</user_password> <username>root</username>

7. Modify the /tftp/Linux_dir/sun/pxelinux.cfg/default file to include the path to the kernel to be installed, and the location of the PXE server.

The line of the default file containing the IP address of the PXE server and the path to the kernel software is the wrapped line beginning with the word "kernel" and ending "/initrd.img":

serial 0 9600default Enterprise-Linux-3.0

display pxelinux.cfg/bootinfo.txt

prompt 1

timeout 50

label Enterprise-Linux-3.0

kernel ../images/pxeboot/vmlinuzappend ksdevice=eth0 console=ttyS0,9600n8 load_ramdisk=1 network ks=nfs:172.16.11.8:/tftp/<Linux_dir>/sun/install/ks.cfg initrd=install/initrd.img

where <Linux_dir> is as-2.1u2 or el-3.O, depending on the version of Red Hat you are installing. If you are installing SuSE Linux Enterprise Server 8 service pack 3, the Linux directory will be sles-8sp3.

8. Log into the B1600 System Controller.

See the Sun fire B1600 Blade System Chassis Software Setup Guide for further details.

9. Boot the blade to begin the PXE boot from the SC prompt.

sc> bootmode bootscript="boot net" snsc> poweron sn (if the blade is currently off)

sc> reset sn (if the blade is currently on)

where n is the slot number of the server blade on which you want to install the operating system.

10. Access the blade's console to monitor the progress of the installation.

At the SC prompt, type:

sc> console sn

where n is the number of the slot containing the blade.

When the installation is complete the blade automatically reboots.

---

4.3 Installing Linux From a Solaris PXE Boot Server

This section tells you how to install Linux on a server blade from a PXE boot server running Solaris.

4.3.1 Files Relevant to PXE Boot Installation

A summary of the files required by the Solaris PXE boot server during PXE boot installation and their purpose is provided in TABLE 4-3.

4.3.2 Preparing to Install Linux

1. Connect a network port on the SSC to a subnet containing both the Network Install Server you intend to use as the PXE boot server and the DHCP server you intend to use to allocate IP addresses to the server blade.

If you have a redundant SSC in the blade system chassis, duplicate this connection on the second SSC.

2. Find out the MAC address of the first interface on the blade you intend to install Linux onto.

To do this, log into the System Controller, and at the sc> prompt, type:

sc>showplatform -v

:

:    

Domain    Status         MAC Address         Hostname

--------  -----------    -----------------   -------------

S1        Standby        00:03:ba:29:e6:28   chatton-s1-0

S2        Standby        00:03:ba:29:f0:de  

S6        OS Running     00:03:ba:19:27:e9   chatton-s6-0

S7        OS Stopped     00:03:ba:19:27:bd   chatton-s7-0

S10       Standby        00:03:ba:2d:d1:a8   chatton-s10-0

S12       OS Running     00:03:ba:2d:d4:a0   chatton-s12-0

:

SSC0/SWT  OS Running           00:03:ba:1b:6e:a5

SSC1/SWT  OS Running           00:03:ba:1b:65:4d 

SSC0/SC   OS Running (Active)  00:03:ba:1b:6e:be

SSC1/SC   OS Running           00:03:ba:1b:65:66

:   

sc>

where the : character indicates omitted data. The MAC address listed for each blade is the MAC address of the first interface (by default, bge0).

For a basic installation that uses only one active network interface (for example, for setting up a blade to boot Linux from the network), you only need the MAC address of the first network interface.

However, if you are intending to set up redundant connections to the network, you also need to calculate the MAC addresses for bge1, bge2 , and bge3.

Make a note of the MAC addresses for each interface on the blade.

3. Make sure the DHCP server you intend to use is properly set up and functioning.

For information about setting up a Solaris DHCP server, refer to the Solaris DHCP Administration Guide.

4. If you want the DHCP server to allocate IP addresses dynamically to the server blade, then reserve a block of addresses on the DHCP server for this purpose.

For information about how to do this, refer to the Solaris DHCP Administration Guide.

4.3.3 Configuring the PXE Boot Servers

Linux is installed on the server blade using the PXE boot system. Three server processes are required to perform the installation:

• DHCP
• TFTP
• NFS

This section provides information on how to configure the DHCP and NFS servers, and how to enable the TFTP server, for use with the PXE boot installation.

4.3.3.1 Configuring the DHCP Server

PXE booting is supported by DHCP services, and this means that there are a number of setup steps you need to perform involving the DHCP server. The DHCP server needs to be configured for each individual blade otherwise the network installation will not work.

1. Log into the Network Install Server as root, and start the DHCP Manager by typing:

# DISPLAY=mydisplay:0.0

# export DISPLAY

# /usr/sadm/admin/bin/dhcpmgr &

where mydisplay is the name of the system (for example, a desktop workstation) that you are using to display the DHCP Manager's GUI (Graphical User Interface).

2. Add the global PXE macro to the DHCP server to enable it to support Linux PXE boot clients.

To define the global PXE macro:

a. In the main window of DHCP Manager's GUI, click the Macros tab, and select Create from the Edit menu.

b. In the Name field of the Create Macro window, type the name of the global macro that enables the DHCP server to support PXE booting (PXEClient:Arch:00000:UNDI:002001).

c. In the Option Name field, type BootSrvA. And in the Option Value field type the IP address that was listed for the Boot Server (that is, the Network Install Server). Then click Add.

d. In the Option Name field, type BootFile. And in the Option Value field type the path to the file pxelinux.bin, for example /<Linux_dir>/sun/pxelinux.bin,(where <Linux_dir> is either as-2.1u2,
el-3.O or sles-8sp3, depending on the version of Linux you are installing). Then click Add.

To view the properties of the macro you have created, select it from the list of macros displayed on the left of the Macros tab, then select Properties from the Edit menu (see FIGURE 4-1).

 FIGURE 4-1 The Properties Defined for the Global PXE Macro

3. Click OK to save the settings.

4.3.3.2 Configuring the NFS Server

The NFS server is used by the installation kernel to read all of the packages necessary to the installation process. The NFS server therefore needs to provide access to the directory structure containing the PXE images.

1. Make the tftpboot directory available to all machines running NFS.

Update the /etc/dfs/dfstab file by adding the following line:

share -F nfs -o rw -d "TFTP boot directory" /tftpboot