C H A P T E R 1 - Initial Inspection of the Server

C H A P T E R 1

Initial Inspection of the Server

Note - This chapter applies to all Sun Fire X4100/X4100 M2 and X4200/X4200 M2 servers, unless otherwise noted.

Service Visit Troubleshooting Flowchart

Use the following flowchart as a guideline for using the subjects in this book to troubleshoot the server.

FIGURE 1-1 Troubleshooting Flowchart

Graphic showing the X4200 server back panel with the status indicator LEDs called out.

Gathering Service Visit Information

The first step in determining the cause of the problem with the server is to gather whatever information you can from the service call paperwork or the on-site personnel. Use the following general guideline steps when you begin troubleshooting.

1. Collect information about the following items:

Events that occurred prior to the failure

Whether any hardware or software was modified or installed

Whether the server was recently installed or moved

How long the server exhibited symptoms

The duration or frequency of the problem

2. Document the server settings before you make any changes.

If possible, make one change at a time, in order to isolate potential problems. In this way, you can maintain a controlled environment and reduce the scope of troubleshooting.

3. Take note of the results of any change you make. Include any errors or informational messages.

4. Check for potential device conflicts before you add a new device.

5. Check for version dependencies, especially with third-party software.

Serial Number Locations

The system serial number is located on a sticker that is attached to the front bezel (see FIGURE 1-2 or FIGURE 1-3 for the location).

If the bezel is missing, a second serial number label is affixed to the system:

For Sun Fire X4100/X4100 M2 servers, the second sticker is attached to the top of

For Sun Fire X4200/X4200 M2 servers, the second sticker is attached to the side of the chassis. If you are facing the chassis front, the sticker is on the left side near the front.

System Inspection

Improperly set controls and loose or improperly connected cables are common causes of problems with hardware components.

Troubleshooting Power Problems

If the server will power on, skip this section and go to Externally Inspecting the Server.

If the server will not power on, check this list of items:

1. Check that AC power cords are attached firmly to the server's power supplies and to the AC source.

2. Check that both the main cover and rear cover are firmly in place.

There is an intrusion switch on the front I/O board that automatically shuts down the server power to standby mode when the covers are removed.

Externally Inspecting the Server

To perform a visual inspection of the external system:

1. Inspect the external status indicator LEDs, which can indicate component malfunction.

For the LED locations and descriptions of their behavior, see External Status Indicator LEDs.

2. Verify that nothing in the server environment is blocking air flow or making a contact that could short out power.

3. If the problem is not evident, continue with Internally Inspecting the Server.

Internally Inspecting the Server

Perform a visual inspection of the internal system by following these steps. Stop when you identify the problem.

1. Choose a method for shutting down the server from main power mode to standby power mode.

Graceful shutdown: Use a ballpoint pen or other stylus to press and release the Power button on the front panel. This causes Advanced Configuration and Power Interface (ACPI) enabled operating systems to perform an orderly shutdown of the operating system. Servers not running ACPI-enabled operating systems will shut down to standby power mode immediately.

Emergency shutdown: Use a ballpoint pen or other stylus to press and hold the Power button for four seconds to force main power off and enter standby power mode.

When main power is off, the Power/OK LED on the front panel will begin flashing, indicating that the server is in standby power mode.

Caution - When you use the Power button to enter standby power mode, power is still directed to the graphics-redirect and service processor (GRASP) board and power supply fans, indicated when the Power/OK LED is flashing. To completely power off the server, you must disconnect the AC power cords from the back panel of the server.

FIGURE 1-2 Sun Fire X4100/X4100 M2 Server Front Panel

Graphic showing the X4100/X4100 M2 server front panel with the power button and Power/OK LED shown on the upper-left.

FIGURE 1-3 Sun Fire X4200/X4200 M2 Server Front Panel

Graphic showing the X4200/X4200 M2 server front panel with the power button and Power/OK LED shown on the upper-left.

2. Remove the server covers, as required.

For instructions on removing system covers, refer to the Sun Fire X4100/X4100 M2 and Sun Fire X4200/X4200 M2 Servers Service Manual, 819-1157.

3. Inspect the internal status indicator LEDs, which can indicate component malfunction.

For the LED locations and descriptions of their behavior, see Internal Status Indicator LEDs.

Note - You can hold down the Locate button on the server back panel or front panel for 5 seconds to initiate a "push-to-test" mode that illuminates all other LEDs both inside and outside of the chassis for 15 seconds.

4. Verify that there are no loose or improperly seated components.

5. Verify that all cable connectors inside the system are firmly and correctly attached to their appropriate connectors.

6. Verify that any after-factory components are qualified and supported.

For a list of supported PCI cards and DIMMs, refer to the Sun Fire X4100/X4100 M2 and Sun Fire X4200/X4200 M2 Servers Service Manual, 819-1157.

7. Check that the installed DIMMs comply with the supported DIMM population rules and configurations, as described in Troubleshooting DIMM Problems.

8. Replace the server covers.

9. To restore main power mode to the server (all components powered on), use a ballpoint pen or other pointed object to press and release the Power button on the server front panel. See FIGURE 1-2 or FIGURE 1-3.

When main power is applied to the full server, the Power/OK LED next to the Power button lights and remains lit.

10. If the problem with the server is not evident, you can try viewing the power-on self test (POST) messages and BIOS event logs during system startup. Continue with Viewing BIOS Event Logs.

Troubleshooting DIMM Problems

Use this section to troubleshoot problems with memory modules, or DIMMs.

Note - For information on Sun's DIMM replacement policy for x64 servers, contact your Sun Service representative.

How DIMM Errors Are Handled By the System

Uncorrectable DIMM Errors

For all operating systems (OS), the behavior is the same:

When UC error happens, the memory controller causes an immediate reboot of the system.

During reboot, BIOS checks NorthBridge memory controller's "Machine Check" registers and finds out previous reboot was due to Uncorrectable ECC Error (PERR/SERR also), then reports this in POST after the memtest stage:
A Hypertransport Sync Flood occurred on last boot.

Memory reports this event in Service Processor's System Event Log (SEL) as follows:

# ipmitool -H 10.6.77.249 -U root -P changeme -I lanplus sel list

f000 | 02/16/2006 | 03:32:38 | OEM #0x12 |

f100 | OEM record e0 | 00000000040f0c0200200000a2

f200 | OEM record e0 | 01000000040000000000000000

f300 | 02/16/2006 | 03:32:50 | Memory | Uncorrectable ECC | CPU 1 DIMM 0

f400 | 02/16/2006 | 03:32:50 | Memory | Memory Device Disabled | CPU 1

DIMM 0

f500 | 02/16/2006 | 03:32:55 | System Firmware Progress | Motherboard

initialization

f600 | 02/16/2006 | 03:32:55 | System Firmware Progress | Video

initialization

f700 | 02/16/2006 | 03:33:01 | System Firmware Progress | USB resource

configuration

Correctable DIMM Errors

At this time, correctable errors are not logged in the server's system event logs. They are reported or handled in the supported operating systems as follows:

Windows server:

A Machine Check error message bubble pops up on task bar.

User must manually go into Event Viewer to view errors as follows:

Start-->Administration Tools-->Event Viewer

View individual errors (by time) to see details of error

Solaris:

There is no reporting of correctable errors in Solaris x86 at this time.

Linux:

There is no reporting of correctable errors in the Linux distributions that we support at this time.

BIOS DIMM Error Messages

BIOS will display and log three types of error messages:

NODE-n Memory Configuration Mismatch

The following conditions will cause this error message:

DIMMs are not paired (Running in 64-bit mode instead of 128-bit mode)

DIMMs speed not same

DIMMs do not support ECC

DIMMs are not registered

MCT stopped due to errors in DIMM

DIMM module type (buffer) mismatch

DIMM generation (I/II) mismatch

DIMM CL/T mismatch

Banks on two sided DIMM mismatch

DIMM organization mismatch (128-bit)

SPD missing Trc or Trfc info

NODE-n Paired DIMMs Mismatch

The following conditions will cause this error message:

Paired DIMMs are not same, Checksum mismatch

NODE-n DIMMs Manufacturer Mismatch

The following conditions will cause this error message:

DIMMs Manufacturer not supported

Only Samsung, Micron, Infineon and SMART DIMMs are supported

This will be displayed when you add Hitachi DIMMs

DIMM Fault LEDs

The ejectors on the DIMM slots on the motherboard contain DIMM fault LEDs.

Note the following differences between the Sun Fire X4100/X4200 and the X4100 M2/X4200 M2 servers regarding the power requirements for viewing the DIMM fault LEDs:

Sun Fire X4100/X4200 servers only: To see the DIMM fault LEDs, you must put the server in standby power mode, with the AC power cords attached. See Internally Inspecting the Server.

Sun Fire X4100 M2/X4200 M2 servers only: You can view the DIMM fault LEDs without the power cords attached. These LEDs can be lit by a capacitor on the motherboard for up to one minute. To light the DIMM fault LEDs from the capacitor, push the small button on the motherboard labeled "DIMM SW2." See FIGURE 1-5.

Note - The DIMM fault LEDs always indicate a failed DIMM pair, with the LEDs lit on both slots of the pair that contains the failed DIMM. See Isolating and Correcting DIMM ECC Errors for a procedure to determine which DIMM of the pair is faulty.

FIGURE 1-4 shows the numbering of the Sun Fire X4100/X4200 DIMM slots.

FIGURE 1-5 shows the numbering of the Sun Fire X4100 M2/X4200 M2 DIMM slots.

FIGURE 1-4 Sun Fire X4100/X4200 DIMM Slot Locations

Graphic showing the X4100/X4200 server motherboard with the DIMM slot numbering indicated.

FIGURE 1-5 Sun Fire X4100 M2/X4200 M2 DIMM Slot Locations

Graphic showing the X4100 M2/X4200 M2 motherboard with the DIMM slot numbering indicated.

DIMM Population Rules

Note - The Sun Fire X4100/X4200 servers use only DDR1 DIMM. The Sun Fire X4100 M2/X4200 M2 servers use only DDR2 DIMMs.

Sun Fire X4100/X4200 Rules

The DIMM population rules for the Sun Fire X4100/X4200 servers are listed here:

Each CPU can support a maximum of four DDR1 DIMMs.

Each pair of DIMMs must be identical (same manufacturer, size, and speed).

The DIMM slots are paired and the DIMMs must be installed in pairs (0 and 1, 2 and 3). The memory sockets are colored black or white to indicate which slots are paired by matching colors.

CPUs with only a single pair of DIMMs must have those DIMMs installed in that CPU's white DIMM slots (0 and 1).

See TABLE 1-1 for supported DIMM configurations.

TABLE 1-1 Sun Fire X4100/X4200 Supported DIMM Configurations (DDR1 Only)
Slot 3	Slot 1	Slot 2	Slot 0	Total Memory Per CPU
0	512 MB	0	512 MB	1 GB
512 MB	512 MB	512 MB	512 MB	2 GB
512 MB	1 GB	512 MB	1 GB	3 GB
512 MB	2 GB	512 MB	2 GB	5 GB
512 MB	4 GB	512 GB	4 GB	9 GB
0	1 GB	0	1 GB	2 GB
1 GB	512 MB	1 GB	512 MB	3 GB
1 GB	1 GB	1 GB	1 GB	4 GB
1 GB	2 GB	1 GB	2 GB	6 GB
1 GB	4 GB	1 GB	4 GB	10 GB
0	2 GB	0	2 GB	4 GB
2 GB	512 MB	2 GB	512 MB	5 GB
2 GB	1 GB	2 GB	1 GB	6 GB
2 GB	2 GB	2 GB	2 GB	8 GB
2 GB	4 GB	2 GB	4 GB	12 GB
0	4 GB	0	4 GB	8 GB
4 GB	4 GB	4 GB	4 GB	16 GB

Sun Fire X4100 M2/X4200 M2 Rules

The DIMM population rules for the Sun Fire X4100 M2/X4200 M2 servers are listed here:

Each CPU can support a maximum of four DDR2 DIMMs.

Each pair of DIMMs must be identical (same manufacturer, size, and speed).

The DIMM slots are paired and the DIMMs must be installed in pairs (A1 and B1, A0 and B0). The memory sockets are colored black or white to indicate which slots are paired by matching colors.

CPUs with only a single pair of DIMMs must have those DIMMs installed in that CPU's white DIMM slots (A1 and B1).

See TABLE 1-2 for supported DIMM configurations.

TABLE 1-2 Sun Fire X4100/X4200 M2 Supported DIMM Configurations (DDR2 Only)
Slot A1	Slot B1	Slot A0	Slot B0	Total Memory Per CPU
1 GB	1 GB	0	0	2 GB
1 GB	1 GB	1 GB	1 GB	4 GB
2 GB	2 GB	1 GB	1 GB	6 GB
4 GB	4 GB	1 GB	1 GB	10 GB
2 GB	2 GB	0	0	4 GB
2 GB	2 GB	2 GB	2 GB	8 GB
4 GB	4 GB	2 GB	2 GB	12 GB
4 GB	4 GB	0	0	8 GB
4 GB	4 GB	4 GB	4 GB	16 GB

Isolating and Correcting DIMM ECC Errors

If your log files report an ECC error or a problem with a DIMM, complete the steps below until you can isolate the fault.

Note - The slot numbers given in the following example use the slot numbering from Sun Fire X4100/X4200 servers. The pair 0+1 is equivalent to pair A1+B1, and pair 2+3 is equivalent to pair A0+B0, in the Sun Fire X4100 M2/X4200 M2 servers.

In this example, the log file reports an error with the DIMM in CPU0, slot 1. The fault LEDs on CPU0, slots 0+1 are lit.

1. If you have not already done so, shut down your server to standby power mode and remove the main cover.

Refer to the Sun Fire X4100 and Sun Fire X4200 Servers Service Manual, 819-1157.

2. Inspect the installed DIMMs to ensure that they comply with the DIMM Population Rules.

3. Inspect the fault LEDs on the DIMM slot ejectors and the CPU LEDs on the motherboard. See FIGURE 1-4.

If any of these LEDs are lit, they can indicate the component with the fault.

4. Disconnect the AC power cords from the server.

Caution - Before handling components, attach an ESD wrist strap to a chassis ground (any unpainted metal surface). The system's printed circuit boards and hard disk drives contain components that are extremely sensitive to static electricity.

5. Remove the DIMMs.

6. Visually inspect the DIMMs for physical damage, dust, or any other contamination on the connector or circuits.

7. Visually inspect the DIMM slot for physical damage. Look for cracked or broken plastic on the slot.

8. Dust off the DIMMs, clean the contacts, and reseat them.

9. If there is no obvious damage, exchange the individual DIMMs between the two slots of a given pair. Ensure that they are inserted correctly with ejector latches secured.

Using the example, remove the DIMMs from CPU0, slots 0+1 then reinstall the DIMM from slot 1 into slot 0; reinstall the DIMM from slot 0 into slot 1.

10. Reconnect AC power cords to the server.

11. Power on the server and run the diagnostics test again.

12. Review the log file.

If the error now appears in CPU0, slot 0 (opposite to the original error in slot 1), the problem is related to the individual DIMM. In this case, return both DIMMs (the pair) to the Support Center for replacement.

If the error still appears in CPU0, slot 1 (as the original error did), the problem is not related to an individual DIMM. Instead, it might be caused by CPU0 or by the DIMM slot. Continue with the next step.

13. Shut down the server again and disconnect the AC power cords.

14. Remove both DIMMs of the pair and install them into paired slots on the opposite CPU.

Using the example, install the two DIMMs from CPU0, slots 0+1 into CPU1, slots 0+1 or CPU1, slots 2+3.

15. Reconnect AC power cords to the server.

16. Power on the server and run the diagnostics test again.

17. Review the log file.

If the error now appears under the CPU that manages the DIMM slots you just installed, the problem is with the DIMMs. Return both DIMMs (the pair) to the Support Center for replacement.

If the error remains with the original CPU, there is a problem with that CPU.