| Sun Fire X4600 to Sun Fire X4600 M2 Server Migration |     |
|
Sun Fire X4600 to Sun Fire X4600 M2 Server Migration |
This manual describes procedures for upgrading a Sun Fire X4600 server to a Sun Fire X4600 M2 server.
These two systems share a chassis and motherboard. They differ in firmware, which determines the CPU modules and DIMMS that they support:
No one set of system firmware supports both kinds of CPU modules. Two firmware update streams are maintained. Because the two firmware update streams support different hardware, the upgrade procedure is more complicated than for a typical firmware upgrade.
| Note - If you plan to upgrade a Sun Fire X4600 system to use CPU modules with 4 or more cores, you must upgrade the system to be a Sun Fire X4600 M2, as described in this manual. |
The upgrade procedure has four stages:
Populating a Server With CPU Modules
Firmware: Planning and Downloading
|
1. Verify that the system is in working order. The host operating system should be able to boot and have access to all installed CPUs. Resolve any problems before proceeding.
2. Determine the new CPU modules that you will use and obtain them, along with any necessary memory DIMMs.
| Note - You must user DDR2 DIMMs that Sun has certified for use with the new CPU modules; the DDR1 DIMMs in your existing modules cannot be used. |
3. If you plan to use a different number of CPU modules than you currently have, consult the following for:
4. Determine how you will interact with the system while performing the firmware update. See Interacting with the System for specifics.
5. Plan the firmware update and download the necessary files. See Firmware: Planning and Downloading for specifics.
CPU Module Matching Requirements
The following CPU modules can be used together:
Except for the modules listed in this section, all CPU modules in a server must be identical. This means that each module must have the same Sun part number. If two modules are the same model but different part numbers, then their CPUs are different; mixing CPUs in a single server is not supported.
When different CPU modules are in the same system, the CPUs must be identical. This means that even though the part numbers for the CPU modules may be different, the part numbers for the CPUs on the modules must be the same.
|
Caution - Only the CPU modules listed in this section can used together. |
The 8 CPU module slots are designated by letters A through H, as shown in FIGURE 1. Looking from the front of the server, the CPU module slots are lettered from right to left.
When a server contains fewer than 8 CPU modules, only specific slots can be used, as shown in FIGURE 1. This figure also shows how the CPU module slot letters map to the numbers that identify actual CPU modules.
| Note - In an eight-module configuration, the modules are not all in numerical order. Slot B contains module 2, and slot C contains module 1. |
| Note - In a two-module configuration, use of PCI slots 5, 6, and 7 is not supported. |
| Note - In six-module and four-module configurations, unused CPU slots must contain filler modules. In two-module configurations, unused CPU slots must be left empty. |
FIGURE 1 CPU Module Population rules and numbering
You have two options for interacting with the system:
In order to interact directly with the system, you will need the following:
Test this setup before proceeding. If the system is fully booted, you should be able to interact with the system using the monitor and keyboard. If any of the power connectors are live, the service processor should be active, and you will be use the ILOM command line, even if the system is powered down.
| Note - Before you can use the ILOM command line, you must log in. The administrator user name is “root”, and the default password is “changeme”. If the current password is lost, you can reset it to the default; see Resetting SP and BIOS Passwords Using Jumper P20 |
If the service processor is connected to your network using the Net Management port on the back panel, you can interact with the ILOM and system console in several ways:
Test these features before proceeding. You will need the network address of the service processor. If this is not available from your network administrator, obtain the service processor’s IP address by logging into the ILOM command line through the serial management port. (See Interacting Directly.) Once connected, enter this command:
This procedure describes how to reset the administration password (the root password) for the ILOM SP back to the default after it has been set once during initial setup.
| Note - This procedure simultaneously removes any BIOS password that was set. |
1. Shut down the server to standby power mode by using a pencil or other pointed object to press and release the recessed Power button on the front panel. See Powering Off the Server.
2. Disconnect the AC power cords from the server.
3. If the server is in a rack, slide it far enough from the rack so that you can remove the main cover. If you cannot safely view and access the motherboard, remove the server from the rack.
4. Remove the main cover from the server. See To Remove the Main Cover
5. Install the shorting jumper across the P20 header pins.
See FIGURE 2 for the P20 jumper location. The P20 jumper function is to clear the ILOM SP password.
6. Replace the server’s main cover.
7. Reconnect AC power cords to the server.
The server powers up to standby power mode, indicated when the Power/OK LED on the front panel is flashing.
8. Return the server to main power mode by using a pencil or other pointed object to press and release the recessed Power button on the front panel.
9. Log in to the ILOM web GUI using root as the user name and changeme as the password.
For login instructions, refer to the Integrated Lights Out Manager Administration Guide, 819-1160.
10. Change the default password to a password of your choice.
11. Repeat steps 1 through 8 to remove the P20 jumper. (Remove the jumper in step 5 rather than inserting it.)
| Note - If you do not remove the P20 jumper, the ILOM SP and BIOS passwords will be reset every time you power-cycle the server. |
FIGURE 2 Location of Jumpers on the Motherboard
There are two methods for shutting down the server from main power mode to standby power mode.
|
|
Caution - Servers not running ACPI-enabled operating systems will shut down to standby power mode immediately. |
When main power is off, the Power/OK LED on the front panel begins flashing, indicating that the server is in standby power mode.
|
a. Loosen the two captive screws that secure the cover at the rear of the chassis.
b. Using the indent on the cover for leverage, slide the cover toward the rear of the chassis approximately 0.5 inches (12 mm.). See FIGURE 3.
c. Grasp the cover by its edges and lift it straight up from the chassis.
| Note - When you remove the cover, the intrusion switch that is on the motherboard automatically powers down the system to standby mode. |
FIGURE 3 Removing the Main Cover
|
1. Set the cover on the chassis so that the tabs on the cover align with the notches in the chassis.
2. Slide the cover toward the server front about 0.5 inches (12 mm.), ensuring that the front lip on the cover slides into the groove on the chassis.
3. Tighten the two captive screws on the back of the cover by hand.
|
1. Enter the service processor’s network address in a web browser address bar. You will be directed to the web GUI login screen, as shown in FIGURE 4
The remote console is accessed through the web GUI.
2. Choose the “Remote Control” tab, then the “Redirection” tab, and click on the “Launch Redirection” button. You will have to login separate to the JavaRConsole application, as shown in FIGURE 5
3. The ILOM command line is accessible over the network using SSH software. If you use the ssh command line client, you will probably need to specify the ILOM user name on the command line. For example:
FIGURE 5 Login to Redirected System Console
To Upgrade the Firmware Images
To Replace a CPU Module or CPU Filler Board
| Note - Before you proceed with the firmware upgrade see: To Replace the CPU Modules |
Your first planning decision is to determine the firmware version you want on your upgraded system. In most cases, this should be the firmware in the latest software release.
The application used to flash a new ILOM image is the ILOM itself. For this process to work correctly, the ILOM doing the flash must be compatible with the ILOM being flashed. To avoid compatibility issues,
|
1. If your current firmware is not the latest X4600 version available, use your current ILOM to flash the latest X4600 ILOM.
2. With the latest X4600 ILOM installed, use it to flash the latest X4600 M2 ILOM.
3. If your target firmware is not the latest version, downgrade your ILOM to the target.
4. Flash the LSI HBA BIOS from the same software release as the ILOM.
In each of steps 1 through 3, you will also be flashing a new system BIOS, since the two ILOM and BIOS are provided in a single image file.
As you plan the BIOS, you will find a worksheet similar to the one in TABLE 1 useful.
| Note - System BIOS and LSI HBA BIOS versions are not important for every step. During much of the upgrade process, only the service processor will be active; the rest of the system will be inactive. |
The firmware version numbers are documented in the Software Release Notes or Product Notes for the two products.
Begin by determining your system’s current ILOM and system BIOS versions. The ILOM version is available from the web GUI in the System Information/Versions screen. You can also retrieve the version from the ILOM command line using the version command.
| Note - Always include the build number when referring to an ILOM version. Different builds with the same version number are often available. |
Use these two version strings to determine the software release version. In the following examples, we will assume that the initial ILOM is 1.0.2 Build 10708, and the initial BIOS is 0ABHA041. These are both part of Software 1.1 for the Sun Fire X4600.
Now you need the ILOM version for the latest X4600 release. For the purpose of the following examples, we will use Software 1.3a, which contains ILOM 2.0.2.1 build 23233.
Next, you need the version strings for the latest M2 release. For the purpose of the following examples, we will use Software 2.0, which contains ILOM 2.0.2.5 build 29771.
If you plan to use earlier versions of the M2 firmware, you will need the software release, ILOM version, system BIOS version, and LSI HBA BIOS version for your final setup. If you plan to use the latest M2 firmware, you need the latest system BIOS and LSI HBA BIOS version. For the purpose of the following examples, we will assume that no downgrade is needed, so that we only need the versions from Software 2.0. These are fw1.23.90-bios6.18.01.
If you fill out the firmware worksheet using the above examples, it should look something like TABLE 2.
Now that you know which software releases you’ll need, you can download them from this web page:
http://www.sun.com/servers/x64/x4600/downloads.jsp
While working from the example worksheet in TABLE 2, the following files are needed:
If you plan to flash the ILOM and system BIOS images from the ILOM command line, copy the image files to an TFTP server. If you plan to install the ILOM and system BIOS images from the web GUI, they need to be accessible from the same system that’s running the web browser.
If you plan to use a physical disc to install the LSI HBA BIOS, you will need to burn the image you downloaded onto a disc. Otherwise, you can boot the system from the ISO image, using the remote console.
|
Before proceeding with the firmware upgrade, replace the X4600 CPU modules with the X4600 M2 CPU modules. Follow these steps:
1. Perform a graceful shutdown of the host operating system.
2. Disconnect all cables from the back panel
3. Replace CPU modules as described in the following sections:
To Replace a CPU Module or CPU Filler Board
4. Reconnect one of the power cables; leave the other three disconnected.
5. Make the data connections you will need for the upgrade process. Depending on how you plan to interact with the system, you might need:
|
A CPU module includes the CPU (with heat sink) and DIMM slots on a removable board. The CPU modules are not redundant; failure of one CPU module will cause system failure.
| Note - This component is a CRU and can be replaced by anyone. |
Follow these steps to remove and replace a CPU module or CPU filler board.
1. Power off the server as described in Powering Off the Server.
|
|
Caution - The CPU module can be damaged if removed or inserted while standby power is on. Be sure to disable standby power by disconnecting all external power cables before proceeding. |
2. Remove the main cover as described in “To Remove the Main Cover.
3. Identify the CPU module or CPU filler board you are replacing.
4. Remove the CPU module or CPU filler board from the bay (see FIGURE 6):
a. Simultaneously rotate both plastic levers on the top corners of the module to an upright, 90-degree position.
b. Lift the module straight up to remove it from the metal edge-guides in the bay.
FIGURE 6 Removing a CPU Module From a CPU Module Bay
5. If you are replacing a CPU filler board skip to Step 9.
6. Remove the DIMMs from the CPU module.
a. Rotate both DIMM slot ejectors outward as far as they will go. The DIMM is partially ejected from the socket. See FIGURE 7.
b. Carefully lift the DIMM straight up to remove it from the socket.
c. Set the DIMMs on a clean ESD-safe surface until you are ready to install them to the new CPU module.
7. Unpack the new CPU module, and set it flat on its back surface on an ESD-safe mat.
8. Install the DIMMs that you removed to the new CPU module.
| Note - Use the DIMM population rules that are appropriate for the CPU module. See CPU Modules and Memory DIMMs. |
a. Ensure that the DIMM slot ejectors at each end of the slot are fully open (rotated outward) to accept the new DIMM.
b. Align the notch in the bottom edge of the DIMM with the key in the DIMM socket. See FIGURE 7.
c. Press down evenly on both top corners of the DIMM until the ejectors snap over the cutouts in the left and right edges of the DIMM.
9. Install the new CPU module or CPU filler board.
| Note - All CPU modules in the server must have identical part numbers. Mixing CPU speeds or mixing dual-core CPUs with single-core CPUs is not supported. Be sure to populate the CPU module slots according to the rules in Populating a Server With CPU Modules. |
|
|
Caution - The CPU module can be damaged if removed or inserted while standby power is on. Disable standby power by disconnecting all external power cables before proceeding. |
a. Ensure that both plastic levers on the top corners of the module are in the fully open, vertical position.
b. Align the module edges with the metal edge-guides in the CPU module bay.
c. Lower the module until it touches the motherboard connector and the open plastic levers are in contact with the edges of the bay.
d. Rotate both plastic levers downward simultaneously until they are flat against the top corners of the module.
FIGURE 8 Installing a CPU Module
To Update the ILOM and System BIOS
Cleanup Steps, Post ILOM Update
There are two stages to the firmware update process:
1. Using the ILOM, upload and flash one or more ILOM image files. (These image files also contain a matched system BIOS, which is flashed at the same time.)
2. Using a bootable CD or CD image, flash the LSI HBA BIOS. This only needs to be done once, unless an error occurs.
|
Each of the .ima files you downloaded previously (see Firmware: Planning and Downloading) contains both an ILOM image and a system BIOS image. Each, in turn must be uploaded and flashed in the order they are listed on your worksheet. There are two ways to do this: from the ILOM command line, and by way of the web interface.
Specify the .ima in the load command:
load -source tftp://server/filename.ima
Where server is the address of the TFTP server, and filename.ima is the name of the.ima file (possibly including a directory name).
Navigate to the Maintenance/Firmware Upgrade page, and click on “Enter Upgrade Mode”. You will be sent to a web page where you can upload and flash the new images.
The upload and flash processes take about 5 minutes. When they complete, the service processor is reset. If you are working remotely, you will have to re-establish your connection with the service processor:
| Note - If the ILOM update fails and you are unable to correct the problem, you can try the update without safeguards. See To Force the ILOM Update. |
After it is reset, the service processor can take a few minutes to reboot. Once you reconnect, check the version via the web GUI or ILOM command line. If the version (including the build number) matches the version you just uploaded, then the update has succeed. You can then repeat the entire process with the next firmware set on your worksheet.
1. Clear the system CMOS. You can do this by setting a jumper, as described in the Clearing CMOS Jumper P2.
Alternately, you can use ipmitool from a remote system:
ipmitool -U root -H spaddress chassis bootdef \
none clear-cmos=yes
Where spaddress is the service processor’s network address.
2. Connect all four power cables, boot the system, and watch the boot process on the console.
| Note - A CMOS checksum error will be reported. This message is caused by your clearing the CMOS in the previous step. It can be safely ignored. |
| Note - The first time it boots with the new firmware, the system may report only one CPU present. This error will correct itself the next time you boot the system. |
3. When the prompt “Press F2 to run Setup (CTRL+E on Remote Keyboard)” appears, press one of those keys to bring up the BIOS setup screen.
| Note - (If you used ipmitool to clear CMOS, then the BIOS setup screen will appear without your intervention.) |
4. In the BIOS setup screen perform the following steps:
a. Verify that the BIOS version reported matches the last system BIOS version your worksheet.
b. Verify that the date and time are correct. If not, press F2 to correct them.
c. Press F9 to load the optimal default values.
d. Press F10 (if using the remote console, choose Keyboard/F10) to save and exit.
5. Allow the resulting boot to proceed uninterrupted. The correct number of CPU modules should be reported this time.
You are now ready to update the LSI HBA BIOS.
If your attempts to flash the ILOM keep producing an error, you can try “forcing” the update. This means that safeguards in the ILOM update process are disabled. This can only be done from the ILOM command line, using load with the -f option.
load -f -source tftp://server/filename.ima
You can use Jumper P25 to clear the server’s CMOS settings after a system hang. For example, if the server hangs because of incorrect settings and will not boot, use this jumper to invalidate the settings and reboot with defaults.
1. Shut down the server to standby power mode by using a pencil or other pointed object to press and release the recessed Power button on the front panel. See Powering Off the Server
2. Disconnect the AC power cords from the server.
3. If the server is in a rack, slide it far enough from the rack so that you can remove the main cover. If you cannot safely view and access the motherboard, remove the server from the rack.
4. Remove the main cover from the server. See To Remove the Main Cover
5. Install the shorting jumper across the P25 header pins.
See FIGURE 2 for the P25 jumper location. The P25 jumper function is to tell the system to clear the current CMOS settings at system reboot.
6. Wait 10 seconds, then remove the shorting jumper.
This jumper removes battery power from the SouthBridge chipset where the CMOS settings are stored, thereby removing the CMOS settings.
7. Reinstall the main cover to the server.
8. Reconnect AC power cords to the server.
The server powers up to standby power mode, indicated when the Power/OK LED on the front panel is flashing.
In order to update the LSI HBA BIOS, you must boot the system from a CD or CD image containing the update.
1. You can boot from a CD image via the remote console: choose Device/CD-ROM Image, and then select the .iso file you downloaded.
2. Once you’ve inserted the disc in the drive, or mounted the ISO image, reboot.
You will see a menu screen. Choose “1”. The firmware update process begins and takes about 5 minutes.
| Note - The upgrade script runs a number of programs, some of which issue instructions. These can be ignored. Do nothing until the script finishes, and the DOS prompt (A:>) appears. |
Once the script has finished remove the CD from the drive or unselect the CD image from the remote console and reboot. When prompted “Press Ctrl-C to start LSI Configuration Utility” press that key. Verify that the LSI HBA BIOS version information matches that on the last line of your worksheet. Then exit the utility and reboot. The upgrade to an M2 system is complete.
CPU Module Models describes the CPU module models and explains the rules for populating the module with memory DIMMs.
Populating a Server With CPU Modules describes the rules for populating a server with CPU modules and explains how module slot letters map to numeric module IDs.
Locating Faulty CPUs and DIMMs describes the CPU modules’ built-in diagnostic features.
This section describes the various CPU module models. Some models are available in multiple part numbers. All part numbers within a given model share basic features such as the kind of CPU they support and the way they use memory DIMMs. Individual part numbers vary as to the specific CPU they support. For more information refer to the Sun System Handbook:
http://sunsolve.sun.com/handbook_pub/
Each CPU module model description indicates whether it can be used with the X4600 server or the X4600 M2 server. No CPU module model can be used with both systems.
The sections that document individual CPU modules are:
Sun Fire X4600 M2 4-DIMM CPU Module
Sun Fire X4600 M2 8-DIMM and 8-DIMM Split Plane CPU Modules
This module is only compatible with the Sun Fire X4600 server. It supports an AMD Opteron 800 series dual-core CPU and a maximum of 8 GB of DDR1 memory.
Physical layout of the Sun Fire X4600 CPU Module is shown in
| Note - This module does not number its DIMM slots consecutively. Also, DIMM pairs are not placed in adjacent slots. |
FIGURE 9 Physical Layout of Sun Fire X4600 CPU Module
Pressing the Fault Remind button lights LEDs that indicate faulty CPU and DIMM. SeeLocating Faulty CPUs and DIMMs
LEDs are described in TABLE 3
|
Amber. Indicates faulty CPU when Fault Remind button is pressed. |
|
|
Amber. Indicate faulty DIMMs when Fault Remind button is pressed. |
The Sun Fire X4600 CPU Module provides four DIMM slots (TABLE F-2), for a maximum of two pairs. Here are the specifications and population rules used with the DIMM slots:
This module is only compatible with the Sun Fire X4600 M2 server. It supports an AMD Opteron 8000 series dual-core CPU and a maximum of 16 GB of DDR2 memory.
Physical layout of the Sun Fire X4600 M2 4-DIMM CPU module is shown in FIGURE 10
FIGURE 10 Physical Layout of Sun Fire X4600 M2 4-DIMM CPU Module X4600 M2 8-
Pressing the Fault Remind button lights LEDs that indicate faulty CPU and DIMM. See Locating Faulty CPUs and DIMMs
| Note - This module can be used in the same system with Sun Fire X4600 M2 8-DIMM CPU modules, provided all CPUs are identical. For more information on CPU module matching requirements, see CPU Module Matching Requirements |
LEDs are described in TABLE 5.
|
Amber. Indicates faulty CPU when Fault Remind button is pressed |
|
|
Amber. Indicate faulty DIMMs when Fault Remind button is pressed. |
The Sun Fire X4600 M2 4-DIMM CPU module provides four DIMM slots (TABLE 6), for a maximum of two pairs. Here are the specifications and population rules used with the DIMM slots:
These two modules have identical physical layout, and follow the same DIMM usage rules. They both are compatible with the Sun Fire X4600 M2 server only, and support a maximum of 32GB of DDR2 memory. They have important differences:
| Note - For more information on CPU module matching requirements, see CPU Module Matching Requirements |
Physical layout of the Sun Fire X4600 M2 8-DIMM and 8-DIMM split plane CPU modules is shown in FIGURE F-Physical Layout of Sun Fire X4600 M2 8-DIMM and 8-DIMM Split Plane CPU module.
FIGURE 11 Physical Layout of Sun Fire X4600 M2 8-DIMM CPU Module and Sun Fire X4600 M2 8-DIMM Split Plane CPU Modules
Pressing the Fault Remind button lights LEDs that indicate faulty CPU and DIMM. See “To Locate Faulty CPU’s and DIMMs
LEDs are described in TABLE F-3
|
Amber. Indicates faulty CPU when Fault Remind button is pressed |
|
|
Amber. Indicate faulty DIMMs when Fault Remind button is pressed |
The Sun Fire X4600 M2 8-DIMM and 8-DIMM split plane CPU modules provides 8 DIMM slots (TABLE F-6), for a maximum of four pairs. Here are the specifications and population rules used with the DIMM slots:
The following CPU modules can be used together:
Except for the modules listed in this section, all CPU modules in a server must be identical. This means that each module must have the same Sun part number. If two modules are the same model but different part numbers, then their CPUs are different; mixing CPUs in a single server is not supported.
When different CPU modules are in the same system, the CPUs must be identical. This means that even though the part numbers for the CPU modules may be different, the part numbers for the CPUs on the modules must be the same.
|
|
Caution - Only the CPU modules listed in this section can used together. |
The 8 CPU module slots are designated by letters A through H, as shown in FIGURE F-4. Looking from the front of the server, the CPU module slots are lettered from right to left.
FIGURE 12 Sun Fire X4600/X4600 M2 CPU Module Slots
When a server contains fewer than 8 CPU modules, only specific slots can be used, as shown in FIGURE F-5. This figure also shows how the CPU module slot letters map to the numbers that identify actual CPU modules.
| Note - In an eight-module configuration, the modules are not all in numerical order. Slot B contains module 2, and slot C contains module 1. |
| Note - In a two-module configuration, use of PCI slots 5, 6, and 7 is not supported. |
FIGURE 13 CPU Module Population Rules and Numbering
| Note - In six-module and four-module configurations, unused CPU slots must contain filler modules. In two-module configurations, unused CPU slots must be left empty. |
Every CPU module supports the following diagnostic procedure.
1. Make sure that the system is powered down, but that power cables are still connected to the PSU. This ensures that standby power is flowing to the CPU modules
2. Locate the CPU module with a lit amber LED visible at its top edge. The LED indicates that the module has faulty CPU or DIMM.
3. Remove standby power by disconnecting all power cables from the PSU.
|
|
Caution - Removing or inserting a CPU module while standby power is available can damage the module. |
4. Remove the CPU module you previously identified from its slot in the CPU module bay. See “To Replace a CPU Module or CPU Filler Board.
5. Press the module’s fault remind button. If a DIMM is faulty, an amber LED is lit in one of the DIMM’s slot handles. If the CPU is faulty, a CPU fault LED is lit. These LEDs are powered by an on-board capacity and can remain lit about a minute.
To identify the fault remind button and the CPU fault LED, refer to the individual module descriptions in “CPU Module Models” on page F-1.
| Sun Fire X4600 to Sun Fire X4600 M2 Server Migration | 819-5040-11 | |
Copyright © 2009 Sun Microsystems, Inc. All rights reserved.
To Plan and Prepare
