C H A P T E R 3

Diagnostics Tools

This chapter contains information about diagnostic tools that you can use to determine the status of the Sun Blade X6220 server module and components.

This chapter contains the following topics:

Section 3.1, Service Processor ILOM

Section 3.2, System Status LEDs

Section 3.3, BIOS POST

Section 3.4, Hardware Debug Tool (HDT)

Section 3.5, SunVTS Diagnostic Tests

3.1 Service Processor ILOM

The following component information is available through the service processor (SP) Integrated Lights Out Manager (ILOM).

Viewing replaceable component information

Viewing sensors and LED indicators

Viewing indicators (LEDs) and controlling the locate LED

Managing alerts

Viewing and clearing the system event log

See the Integrated Lights Out Manager (ILOM) Administration Guide for ILOM 1.1.1, 820-0280, for more information.

3.2 System Status LEDs

The Sun Blade X6220 server module has external and internal system status LEDs.

3.2.1 External Status Indicator LEDs

FIGURE 3-1 shows the locations of the external status indicator LEDs.

FIGURE 3-1 External LED Location

Figure showing external LED locations on the server front panel

Refer to TABLE 3-1 for descriptions of the LED behavior.

TABLE 3-1 Front Panel LED Functions
LED Name	Description
Locate button/LED	This LED helps you to identify which system in the rack you are working on in a rack full of servers. Push and release this button to make the Locate LED blink for 30 minutes. Hold down the button for 5 seconds to initiate a "push-to-test" mode that illuminates all other LEDs both inside and outside the chassis for 15 seconds.
Ready-to-Remove LED	The server module is ready to be removed from the chassis. This LED is switched on by the service processor when the server module main power is off.
Service Action Required LED	This LED has three states: Off: Normal operation. Slow Blinking: A new (unacknowledged) event requiring a service action has been detected. On: The event has been acknowledged, but the problem still requires attention.
Power/OK LED	This LED has three states: Off: Server main power and standby power are off. Standby Blinking: Standby power is on, but main power is off. Slow Blinking: POST or diagnostics are running. On: Server is in main power mode with power supplied to all components.
Hard Disk Drive Status LEDs	The hard disk drives have three LEDs. The order listed below is when the server module is installed in the chassis: Right LED (green): Fast blink means normal disk activity, slow blink means RAID activity, and off means power is off or no disk activity. Middle LED (amber): System has detected a hard disk fault. This LED is controlled by the service processor. Top LED (blue): This LED is not used.

3.2.2 Internal Status Indicator LEDs

These servers have internal status indicator LEDs for the DIMM slots and the CPUs.

When the board is removed from the chassis, you can press a fault indicator button to view the location of the DIMM or CPU that has failed.

FIGURE 3-2 Fault Indicator Button

Figure showing fault indicator button and internal LEDs

See TABLE 3-2 for internal LED behavior.

TABLE 3-2 Internal LED Functions
LED Name	Description
DIMM Fault LED (The ejector levers on the DIMM slots are the LEDs.)	This LED has two states: Off: DIMM is operating properly. Lit (amber): The system has detected a fault with the DIMM.
CPU Fault LED (on motherboard)	This LED has two states: Off: CPU is operating properly. Lit (amber): The system has detected a fault with the CPU.

3.3 BIOS POST

The system BIOS provides a rudimentary power-on self-test (POST). The basic devices required for the server to operate are checked, memory is tested, the LSI 1068 disk controller and attached disks are probed and enumerated, and the two dual-gigabit Ethernet controllers are initialized.

The progress of the self-test is indicated by a series of POST codes. Refer to Appendix B for information on BIOS POST codes.

These codes are displayed at the bottom right corner of the system's VGA screen (once the self-test has progressed far enough to initialize the video monitor). However, the codes are displayed as the self-test runs, and they scroll off the screen too quickly to be read. An alternate method of displaying the POST codes is to redirect the output of the console to a serial port (see Section 3.3.2, Redirecting Console Output).

The message BMC Responding is displayed at the end of the POST.

3.3.1 How BIOS POST Memory Testing Works

The BIOS POST memory testing is performed as follows:

1. The first megabyte of DRAM is tested by the BIOS before the BIOS code is shadowed (that is, copied from ROM to DRAM).

2. Once executing out of DRAM, the BIOS performs a simple memory test
(a write/read of every location with the pattern 55aa55aa).

Note - This memory test is performed only if Quick Boot is not enabled from the Boot Settings Configuration screen. Enabling Quick Boot causes the BIOS to skip the memory test. See Section 3.3.3, Changing POST Options for more information.

3. The BIOS polls the memory controllers for both correctable and uncorrectable memory errors and logs those errors into the service processor.

3.3.2 Redirecting Console Output

Use these instructions to access the service processor and redirect the console output so that the BIOS POST codes can be read.

1. Connect a dongle cable to the server module universal connector port (UCP). See FIGURE 1-2.

2. Connect a monitor to the dongle cable video port and a keyboard to a USB port.

3. Power cycle or power on the server.

4. Initialize the BIOS setup utility by pressing the F2 key while the system is performing the power-on self-test (POST).

5. When the BIOS Main Menu screen is displayed, select Advanced.

6. When the Advanced Settings screen is displayed, select IPMI 2.0 Configuration.

7. When the IPMI 2.0 Configuration screen is displayed, select the LAN Configuration menu item.

8. Select the IP Address menu item.

The service processor's IP address is displayed in the following format:
Current IP address in BMC: xxx.xxx.xxx.xxx

9. Start a web browser and type the service processor's IP address in the browser's URL field.

10. When you are prompted, type a user name and password as follows:

User name: rootPassword: changeme

11. When the ILOM Service Processor GUI screen is displayed, click the Remote Control tab.

12. Click the Redirection tab.

13. Set the color depth for the redirection console at either 6 or 8 bits.

14. Click the Start Redirection button.

The remote console window appears and prompts you for your user name and password again.

15. When you are prompted, type a user name and password as follows:

User name: rootPassword: changeme

The current POST screen is displayed.

3.3.3 Changing POST Options

These instructions are optional, but you can use them to change the operations that the server performs during POST.

1. Connect a dongle cable to the server module universal connector port (UCP). See FIGURE 1-2.

2. Connect a monitor to the dongle cable video port and a keyboard to a USB port.

3. Initialize the BIOS setup utility by pressing the F2 key while the system is performing the power-on self-test (POST).

4. When the BIOS Main Menu screen is displayed, select the Boot menu.

5. From the Boot Settings screen, select Boot Settings Configuration.

6. On the Boot Settings Configuration screen, there are several options that you can enable or disable:

Quick Boot: This option is disabled by default. If you enable it, the BIOS skips certain tests while booting, such as the extensive memory test. This decreases the time it takes for the system to boot.

System Configuration Display: This option is disabled by default. If you enable it, the system configuration screen is displayed before booting begins.

Quiet Boot: This option is disabled by default. If you enable it, the Sun Microsystems logo is displayed instead of POST codes.

Language: This option is reserved for future use. Do not change.

Add On ROM Display Mode: This option is set to Force BIOS by default. This option is enabled only if you have also enabled the Quiet Boot option, but it controls whether output from the Option ROM is displayed. The two settings for this option are as follows:

Force BIOS: Remove the Sun logo and display Option ROM output.

Keep Current: Do not remove the Sun logo. The Option ROM output is not displayed.

Boot Num-Lock: This option is set to On by default (keyboard Num-Lock is turned on during boot). If you set this to Off, the keyboard Num-Lock is not turned on during boot.

Wait for F1 if Error: This option is disabled by default. If you enable it, the system pauses if an error is found during POST and resumes only when you press the F1 key.

Interrupt 19 Capture: This option is reserved for future use. Do not change.

Default Boot Order: The letters in the brackets represent the boot devices. To see the letters defined, position your cursor over the field and read the definition on the right side of the screen.

3.4 Hardware Debug Tool (HDT)

The hardware debug tool (HDT) is a diagnostic tool that allows access to all memory spaces and CPU registers of the system.

3.4.1 HDT Functionality

Available functionality includes:

Access to CPU registers and memory spaces

Automated diagnostics

Checking of all HT links for errors at different speeds and widths

Testing of BIOS flash access and BIOS progress

Testing of access to other board components

HDT can be used to:

Check the status and configuration of system

Diagnose the system with the following symptoms:

No video

System hang (any bad HT link will cause this)

BIOS POST failure before video comes up

Note - HDT diagnostics will stop or reset and power cycle the system. Do not use HDT while the operating system is running.

3.4.2 HDT Access

You can access HDT through the server module SP as follows:

Username: sunservice

Password: changeme

3.4.3 HDT Commands

To query HDT, execute:

# hdtl -q

This command does the following:

Dumps the current system status and configuration.

Power cycles the stem and executes the HyperTransport Link test.

Dumps the system status and configuration after another clean start.

Note - On a nonresponsive system, run this command before the system is reset or power cycled.

To log output to the default location (/coredump/debug.log), execute:

# hdt -l -q

To log output to a specified log file, execute:

# hdt -l logfile_name -qa

Where path/logfile_name is the path and file location.

3.5 SunVTS Diagnostic Tests

The Sun Blade X6220 server module is shipped with a Bootable Diagnostics CD
(705-7852) that contains SunVTS trademark software.

SunVTS is the Sun Validation Test Suite, which provides a comprehensive diagnostic tool that tests and validates Sun hardware by verifying the connectivity and functionality of most hardware controllers and devices on Sun platforms. SunVTS software can be tailored with modifiable test instances and processor affinity features.

The current x86 support is for the 32-bit operating system only. Only the following tests are supported on x86 platforms:

CPU Test (cputest)

Disk and Floppy Drives Test (disktest)

Data Translation Look-aside Buffer (dtlbtest)

Floating Point Unit Test (fputest)

Network Hardware Test (nettest)

Ethernet Loopback Test (netlbtest)

Physical Memory Test (pmemtest)

Serial Port Test (serialtest)

System Test (systest)

Universal Serial Bus Test (usbtest)

Virtual Memory Test (vmemtest)

SunVTS software has a sophisticated graphical user interface (GUI) that provides test configuration and status monitoring. You can run the GUI on one system to display the SunVTS testing of another system on the network. SunVTS software also provides a TTY-mode interface for situations in which running a GUI is not possible.

3.5.1 SunVTS Documentation

SunVTS documentation is included on the Bootable Diagnostics CD (705-7852).You can also access SunVTS documentation at this site:

http://docs.sun.com/app/docs/coll/1140.2

3.5.2 Diagnosing Server Problems With the Bootable Diagnostics CD

SunVTS software is preinstalled on these servers. The server is also shipped with the Bootable Diagnostics CD (705-7852). This CD is designed so that the server boots from the CD. This CD boots the Solaris trademark Operating System and starts SunVTS software. Diagnostic tests runs and write output to log files that the service technician can use to determine the problem with the server.

3.5.2.1 Requirements

To use the Bootable Diagnostics CD, you must have either a keyboard, mouse, and monitor attached to the server on which you are performing diagnostics, or a remote console server if you run the diagnostics CD remotely.

3.5.2.2 Using the Bootable Diagnostics CD

To use the Bootable Diagnostics CD to perform diagnostics:

1. Set up the Bootable Diagnostics CD to run on a local or remote server.

If you want to run the CD from a remote server using ILOM, see the Integrated Lights Out Manager (ILOM) Administration Guide for ILOM 1.1.1, 820-0280, for instructions on running the CD remotely.

If you want to run the CD on the local server, continue with the following steps:

a. Attach the dongle to the server UCP port. See FIGURE 1-2.

b. Attach a USB CD or DVD drive to a USB port on the dongle cable.

c. With the server powered on, insert the Bootable Diagnostics CD into the CD/DVD-ROM drive.

d. Reboot the server, but press F2 during the start of the reboot so that you can change the BIOS setting for boot-device priority.

e. When the BIOS Main menu appears, navigate to the BIOS Boot menu.

Instructions for navigating within the BIOS screens are printed on the BIOS screens.

f. On the BIOS Boot menu screen, select Boot Device Priority.

The Boot Device Priority screen appears.

g. Select the CD-ROM drive as the primary boot device.

h. Save and exit the BIOS screens.

i. Reboot the server.

When the server reboots from the CD in the CD- or DVD-ROM drive, the Solaris Operating System boots and SunVTS software starts and opens its first GUI window.

2. In the SunVTS GUI, press Enter or click the Start button when you are prompted to start the tests.

The test suite runs until it encounters an error or the test is completed.

Note - The CD takes approximately nine minutes to boot.

3. When SunVTS software completes the test, review the log files generated during the test.

SunVTS provides access to four separate log files:

SunVTS test error log contains time-stamped SunVTS test error messages. The log file path name is /var/opt/SUNWvts/logs/sunvts.err. This file is not created until a SunVTS test failure occurs.

SunVTS kernel error log contains time-stamped SunVTS kernel and SunVTS probe errors. SunVTS kernel errors are errors that relate to running SunVTS, and not to the testing of devices. The log file path name is /var/opt/SUNWvts/logs/vtsk.err. This file is not created until SunVTS reports a SunVTS kernel error.

SunVTS information log contains informative messages that are generated when you start and stop the SunVTS test sessions. The log file path name is /var/opt/SUNWvts/logs/sunvts.info. This file is not created until a SunVTS test session runs.

Solaris system message log is a log of all the general Solaris events logged by syslogd. The path name of this log file is /var/adm/messages.

a. Click the Log button.

The log file window is displayed.

b. Specify the log file that you want to view by selecting it from the log file window.

The content of the selected log file is displayed in the window.

c. With the three lower buttons you can do the following actions:

Print the log file: A dialog box appears for you to specify your printer options and printer name.

Delete the log file: The file remains displayed, but it will be gone the next time you try to display it.

Close the log file window: The window is dismissed.

Note - If you want to save the log files: You must save them to another networked system or to a removable media device. When you use the Bootable Diagnostics CD, the server boots from the CD. Therefore, the test log files are not on the server's hard disk drive and they are deleted when you power cycle the server.