Sun Netra CP3260 Blade Server Product Notes

The Sun Netra CP3260 Blade Server Product Notes contain important and late-breaking information about the Sun Netratrademark CP3260 blade server. This document includes information about upgrading to ATCA R3U2.

This document contains the following:

The most recent versions of the Sun Netra CP3260 blade server and Sun Netra CT 900 server documentation are available at:

http://docs.sun.com/app/docs/prod/cp3260.brd#hic
http://docs.sun.com/app/docs/prod/n900.srvr#hic


Known Issues

SB Flash Drive Type Unknown

If the SB flash disk appears as “drive type unknown” with format, perform the following:

single-step bullet  Turn off Volume Management before starting sunvts test.

Disktest/CF will be in the test menu under USB-device:

/etc/init.d/volmgt stop

/opt/SUNWvts/bin/sunvts


System Requirements

Software and firmware on the servers must be at least version ATCA R3 or newer.

Use the VTS6.4ps2 SPARC (or newer) Toolkit and Documentation.

Download and install SunVTS. For instructions, refer to the Sun Netra CP3260 User’s Guide (820-0457).


Related Documentation

The following documents ship with the Netra CP3260 blade server:

Refer to the Sun Netra CP3260 Blade Server Getting Started Guide for information on obtaining the rest of the Netra CP3260 blade server documentation or go to the following web sites.

http://docs.sun.com/app/docs/prod/cp3260.brd#hic
http://docs.sun.com/app/docs/prod/n900.srvr#hic


Enhancements

The R3U3 release provides the following enhancements:

Retaining Custom IPMC Configurations Across Upgrades

IPMC configuration variables are now retained across firmware upgrades and downgrades, as long as the version being downgraded to supports this feature.

IPMC has a group of parameters that are stored in SEEPROM (NV Storage). All variables are part of one data structure. Each of these variables performs a function and can be set or read via commands. These variables are stored across IPMC resets. However, previously, these variables were reset to default values when new or upgraded firmware was installed.

This enhancement retains any custom values you configured for the following variables.


TABLE 0-1 IPMC Variables for Which You Can Retain Configurations

Variable

Description

log_level

Controls the log level of the debug messages.

Payload timeout

Controls the timeout for payload response. IPMC, after sending payload alert, expects OS to respond within this time. Used in graceful reboot and graceful shutdown features.

Payload shutdown timeout

Controls the payload shutdown timeout as described in the graceful reboot/shutdown specification. This is the time that an OS would take to shutdown its applications before IPMC shuts it down or resets the payload.

Verbosity

Controls the debug mode verbosity of the messages. Selectively, messages from/to a particular interface can be enabled/disabled. Used for debugging only.

AMC power up timeout

Controls the time it would take for the AMCs/ARTM to come up before the payload is brought to M4 state.

IPMC Control

Reflects IPMC’s control bits. Currently two bits, Bit 0 and Bit 1 control LED2 and LED1 on the boards.


If you want to reset these variables to their default, use the new Reset IPMC configuration variables command.

For detailed command information, refer to the Sun Netra CT 900 Server Administration and Reference Manual (819-1177).

Upgrading Linux (RedHat 5.3) Firmware for BIOS

Support was added for the Linux BIOS Update Utility. This utility is a stand-alone application for Linux that incorporates the BIOS payload and application required to perform a BIOS update. The utility uses the /dev/mem interface to access the BIOS.

For this application to work on a Linux system:

The CP32x0 BIOS consists of three regions:

1. Boot BIOS Region, which contains the lowest level system initialization.

2. NVRAM Region, which stores system specific configuration data.

3. Main BIOS Region, which includes system configuration code.

The utility can be used to update any or all of these regions. Typically, only the main BIOS region needs to be updated, but in some cases it might be necessary to update the Boot BIOS region

For detailed instructions, refer to the README file.


New Features

This release provides the following new features.

Low-Power FB-DIMM

The Sun Netra CP3260 blade server now supports a low power FB-DIMM @1.55 volts, in addition to the currently supported FB-DIMM@1.8 volts.

The FB-DIMM sensor accommodates the new voltage range with the following thresholds:



Note - These critical and non recoverable voltage ranges are threshold ranges.


For information about sensors, refer to the Sun Netra CT 900 Software Developer’s Guide (819-1175).

Multiplexing to Zones 2 or 3 on Sun Netra CP3260 Blade Servers

A new multiplexer (MUX) controller and ShMM configuration is available for use on Sun Netra CP3260 blade server to multiplex 10GbE network interface unit (NIU) ports to Zone 2 (backplane) and/or to Zone 3 (ARTM). You can configure both ports independently.



Note - The host must be configured to match the MUX configuration.


For customers using the blade servers in a Sun Netra CT 900 chassis, a complete end-to-end solution is provided. This new feature is implemented through the ShMM firmware and new IPMI commands on the IPMC. These commands extend MUX configuration access to the management software so that during blade server hot-swaps, the MUX configuration is persistent across blade server activations and deactivations.

Customers who use Sun Netra CP3260 blade server in a third-party chassis, which does not have the Sun Netra CT 900 chassis ShMM management software implemented, can save MUX configurations in a configuration file or in a persistent storage managed by system management software. When a system management software detects blade server activation, it sends the command to set MUX to the programmed state. Because management software sends the command during every blade server activation, the configuration is persistent across blade server deactivation and activation.

Refer to the following documentation:

Firmware Upgrade From Payload for Blade Servers

The R3U2 release introduces the Blade Centric Firmware Update Tool (bcfutool), which provides support for customers who use Sun Netra CP3260 blade servers in a third party ATCA chassis, instead of a Sun Netra CT 900 chassis.

This tool consolidates the functionality from existing utilities and provides an easy-to-use interface for upgrading the IPMC, MMC, and system firmware from the Sun Netra CP3260 blade server payload running Solaris OS.

For more information, refer to the README file included in the R3U2 release package and the instructions in these Product Notes.

System Firmware Progress Sensor (SFPS)

A new systems firmware progress sensor (SFPS) is available on the Sun Netra CP3260 blade server. The purpose of the sensor is to model the firmware running on the payload and provide various states to the external management software (ShMM on Sun Netra CT 900 servers). This occurs via a standard IPMI event mechanism.

The firmware states are Progress, Hang, and Error, with various substates. The sensor generates an IPMI event message for each state. You can verify the messages by using clia sel command on the ShMM, through HPI event and SNMP traps for each state of a sensor event.

Refer to the following documentation:

http://docs.sun.com/app/docs/prod/cp3260.brd

http://docs.sun.com/app/docs/prod/n900.srvr#hic


Solaris Operating System Versions

The Sun Netra CP3260 blade server supports the Solaris 10 (8/07) OS and subsequent compatible versions, with supported Sun Netra patches. The Solaris OS software can be downloaded from the Sun download center at:

http://www.sun.com/download

For information on these versions of the Solaris OS, including installation, see the appropriate Solaris Documentation Collection at the Sun Documentation web site at:

http://docs.sun.com/app/docs/prod/solaris



Note - Users might experience an incomplete download due to the 2.9 GB size of the Solaris 10 netinstall image file. If you are using a Mozilla browser, be sure to disable Mozilla’s download manager, which prevents larger file downloads. To disable it, choose Edit->Preference->Navigator->Downloads. Next, uncheck the default “Open the download manager option.”


After you download the Solaris OS, check the downloaded file to ensure that it downloaded correctly.


# sum Netra_CP3260_S10U4_RR.cpio
1795 5823104 Netra_CP3260_S10U4_RR.cpio


Checking Firmware Versions

Systems With Blades

To display current version information, preview required upgrades, and to perform an upgrade of the Sun Netra CT 900 system and installed blade servers, invoke the autofwupgrade command from the ShMM. The command syntax is as follows.


autofwupgrade [-h] [ [-vp] -f <FTP-server-IP-address>:<release-archive-root-dir>[:<user-name>:<password>] [-s <board-slot-number>] [-t board-type] ]



Tip - We recommend that you use the -p option to preview the required updates before performing the upgrade.



procedure icon  To Display Firmware Versions for Systems

Use the following command at the cli prompt.


cli> showhost <slot#>

or use the autofwupgrade command.

Blades Only

For users who do not have access to the ShMM or who want manual control of the FW upgrade process, use the bcfutool upgrade tool instead of the ShMM autofwupgrade. Refer to the README and these Product Notes for information and instructions.

The Sun Netra CP3260 blade server firmware should be at the versions listed in the README.

The command syntax is as follows.


bcfutool [-h][-g][-c sysfw | ipmc | mmc ] [-m <site_id> ] [-f <file_name>][-y][-v]


procedure icon  To Display Firmware Versions for Blades Only

Use the bcfutool command at the Solaris prompt.


#> bcfutool -g


Upgrading the Software and Firmware

This release contains the most up-to-date features, enhancements, and bug fixes. Using an earlier release could limit your use of features and enhancements, and could affect your blade servers and systems with known issues.



Note - R3U2 includes changes from previous releases.


Upgrading Systems (Sun Netra CT 900 Servers With Sun Netra CP3260 Blade Servers)

A single bundled software and firmware package is available to apply the applicable portions to your chassis and blade servers. To download the release package and upgrade your Sun Netra CP3260 blade server and Sun Netra CT 900 servers, refer to the Sun Netra CP3260 blade server README files and Sun Netra CT 900 Server Product Notes.

Be sure to follow the upgrade sequence documented in the README files.

This tool upgrades the following automatically from the ShMM:

You can use this tool to upgrade firmware from R3 and R3HW1 releases.



Note - To access and use the automated firmware upgrade tool, you must upgrade the ShMM firmware to R3U2 before running the automated upgrade tool.


The following firmware are not upgraded by the autofwupgrade tool and must be upgraded manually. (For upgrade instructions, refer to README files included in the release download package.)

Although this tool does not currently support downgrading automatically to back out of an upgrade, future releases might support downgrading back to R3U2.

For detailed instructions on how to use the firmware upgrade tool and how to roll back an upgrade, refer to the README included with the R3U2 package.

Upgrading Blade Servers Only

A single bundled software and firmware package is available to apply the applicable portions to your blade servers. To download the release package and upgrade your Sun Netra CP3260 blade servers, refer to the Sun Netra CP3260 blade server README files and these Product Notes.

This tool upgrades the following automatically:

A typical firmware update takes about seven minutes, and during this time the service processor will be fully operational because it is running an earlier image.

The following firmware are not upgraded by the bcfutool tool and must be upgraded manually. (For upgrade instructions, refer to README files included in the release download package.)

Although this tool does not currently support downgrading automatically to back out of an upgrade, future releases might support downgrading back to R3U2. For detailed instructions on how to roll back an upgrade, refer to the README included with the R3U2 package.

Be sure to follow the upgrade sequence documented in the README file.



caution icon Caution - Asynchronous events such as power outage, latch opening, or blade extraction during a firmware upgrade might corrupt the blade server firmware such that the blade will not be able to boot, and you will not be able to retry the upgrade. Be careful not to cause any these events during upgrades. For more information, see Troubleshooting Upgrades.



procedure icon  To Download Firmware

1. Download the most recent package listed at the SDLC site:

http://www.sun.com/download/

Find the package for the release you want to download.

2. After the compressed package is downloaded, expand the files by entering the following commands.


# gunzip Rx_Ux_.tar.gz
# tar xvf Rx_Ux_.tar

where Rx_Ux_ is the release’s version and update number.

3. Check the current firmware versions on your blade server, using the bcfutool -g command.

All firmware versions are listed in the results from this command. Individual commands for checking system, IPMC, and MMC firmware are also available.

4. Perform the procedures for upgrading system firmware, IPMC firmware, and MMC firmware, as applicable.


procedure icon  To Upgrade System Firmware

1. Update the system firmware by entering the bcfutool command as follows:


bcfutool -c sysfw -f Sun_System_Firmware-<x_x_x_g>-Netra_CP3260.pkg

where <x_x_x_g> is the newest system firmware version that you are replacing the earlier version with.

2. Enter yes or y to continue.

Entering yes will continue the process of upgrading the firmware.

3. When the upgrade is complete, check the system firmware version.


bcfutool -g -c sysfw
7.1.0.g

If the upgrade was successful, the newest version will be returned after you enter the command.

If you encountered problems during the upgrade, see Troubleshooting Upgrades.


procedure icon  To Upgrade IPMC Firmware

1. Update the IPMC firmware by entering the bcfutool command as follows:


bcfutool -c ipmc -f 3260_ipmc_boot_<x.x.x>.hpm
bcfutool -c ipmc -f 3260_ipmc_main_<x.x.x>.hpm

where <x.x.x> is the newest IPMC boot and main firmware versions that you are replacing the earlier versions with.



Note - Even though IPMC has boot and main images, it is usually not mandatory to update both of them. Because most of features are part of the main image, you typically only apply the main image update. Refer to the README files to check which components need to be updated.


2. Enter yes or y to continue.

Entering yes will continue the process of upgrading the firmware.

When the service processor has been successfully updated, the user will be presented with the Solaris login prompt.

3. When the upgrade is complete, check the IPMC firmware version.


bcfutool -g -c ipmc
Main Flash version: 02.00.08
Boot Flash version: 01.00.05

If the upgrade was successful, the newest versions will be returned after you enter the command.

If you encountered problems during the upgrade, see Troubleshooting Upgrades.


procedure icon  To Upgrade MMC Firmware

1. Update the MMC firmware by entering the bcfutool command as follows:


bcfutool -c mmc -f XCP32X0-HDD-MMC-<x.x.x>.hpm

where <x.x.x> is the newest MMC firmware version that you are replacing the earlier version with.



Note - Even though MMC has boot and main images, it is usually not mandatory to update both of them. Because most of features are part of the main image, you typically only apply the main image update. Refer to the README files to check which components need to be updated.


2. Enter yes or y to continue.

Entering yes will continue the process of upgrading the firmware.

When the service processor has been successfully updated, the user will be presented with the Solaris login prompt.

3. When the upgrade is complete, check the MMC firmware version.


bcfutool -g -c mmc
MMC main code version - 00.01.09
MMC boot code version - 00.01.02

If the upgrade was successful, the newest versions will be returned after you enter the command.

If you encountered problems during the upgrade, see Troubleshooting Upgrades.

Applying Patches

This section provides information on the Sun Netra patches that you must download from the SunSolveSM web site for the Sun Netra CP3260 blade server and instructions on how to apply these patches. If the Solaris OS was pre-installed on your system, these patches might already be installed; verify that they are present.



Note - Refer to the System Administration Guide: Basic Administration documentation for the Solaris OS that you are using for more information on managing and applying patches.


There are two types of patches to download:



Note - Whenever patches are updated, the revision number increases
(117530-02, 117530-03, and so on). To find the latest version of a patch, perform the search without the revision number. For example, if this document asks you to download patch 117530-03, search for “117530” to find the most recent version of the patch.


Before you start downloading, identify or create a directory on your system for receiving the downloaded patches. This directory will be referred to as the patch-download-dir in the following procedures.

There are two procedures for downloading and installing patches for the Sun Netra CP3260 blade server.

Applying Sun Netra Patches to a Boot Device

Following are the instructions for downloading and installing the patches on a boot device.


procedure icon  To Apply Patches to a Boot Device

1. Go to SunSolve web site and accept the onscreen agreement.

2. Locate and download the mandatory Solaris 10 (08/07) OS patches listed in TABLE 1.


TABLE 1 Sun Netra Patches Required for the Solaris 10 (08/07) OS

Patch ID

Patch Location

126143-01

http://www.sun.com/sunsolve/

128593-01

http://www.sun.com/sunsolve/point

118843-01

http://www.sun.com/sunsolve/

125416-05

http://www.sun.com/sunsolve/


3. Use the unzip command on the patches that you downloaded to extract the contents of the downloaded zip files.

Refer to the unzip(1) man page for additional information.

4. Review the patch README files for specific Install Instructions and follow those instructions.

5. To install a patch, become superuser and type the patchadd and the full path to the patch.


# patchadd /patch-download-dir/patch-id

6. Perform a patchadd for each of the patches in the order listed in Step 2.



caution icon Caution - You mustinstall the patches using the patchaddcommand in the order listed. Some of the patches require software that was added through previous patches, so installing the patches out of order could result in error messages and corrupted software.


Refer to the patchadd(1M) man page for instructions on using the command.

Applying Sun Netra Patches to a Solaris 10 OS Boot Server for Diskless Clients

These patches are required for the diskless client boot server installation process documented in the Sun Netra CP3260 Blade Server User’s Guide (820-0457).



Note - Refer to the Solaris 10 OS System Administration Guide: Basic Administration documentation for more information on managing diskless services.


Following are the instructions for downloading and installing the Sun Netra patches that are being applied to a Solaris 10 OS diskless service.


procedure icon  To Apply Patches to Solaris 10 OS Diskless Clients

1. If you have not already done so, go to SunSolve web site and download the following mandatory patches listed in TABLE 2.


TABLE 2 Patches Required for a Solaris 10 Diskless Service

Patch ID

Patch Location

126143-01

http://www.sun.com/sunsolve/

128593-01

http://www.sun.com/sunsolve/point

118843-01

http://www.sun.com/sunsolve/

125416-05

http://www.sun.com/sunsolve/


2. Move the patch zip files into the patch-download-dir directory.

Where patch-download-dir is the path to the directory where the Solaris 10 OS patches are stored. The patch-download-dir directory might already contain patches that were copied during the OS installation.

3. Use the unzip command on all of the patches that you downloaded to extract the contents of the downloaded zip files.

Refer to the unzip(1) man page for instructions on using the command.

4. Change directories to the /usr/sadm/bin directory.


# cd /usr/sadm/bin

5. Install the required patches to the diskless service.

Use the smosservice patch command to install the patches to the installation image.

Install the software patches on the server in the order shown, where root-password is the root password of the server and patch-download-dir is the path to the directory where the Solaris 10 OS patches are stored.



Note - Note that you must add an additional -U option when you install the last patch.



# ./smosservice patch -u root -p root-password -- -a patch-download-dir/126143-01 -m
# ./smosservice patch -u root -p root-password -- -a patch-download-dir/128593-01 -m
# ./smosservice patch -u root -p root-password -- -a patch-download-dir/118843-01 -m
# ./smosservice patch -u root -p root-password -- -a patch-download-dir/125416-05 -m -U

6. Refer to the instructions in the Sun Netra CP3260 Blade Server User’s Guide (820-0457) to complete the installation procedures for diskless clients.

Troubleshooting Upgrades

To prevent issues during bcfutool command upgrades, review the following information.



caution icon Caution - There are vulnerability windows during the firmware upgrade operation that might corrupt the blade firmware, causing the blade not be able to boot. In these cases, these blades can only be recovered by board-external (chassis vendor supplied) tools, if available. If not recoverable, these blades have to be returned to SunService for repair or replacement.


To avoid firmware corruption, prevent the following events from occurring while the firmware is being upgraded:

A recommendation for avoiding failures during ugrade vulnerability periods is to monitor the upgrade manually or by using a script.

If you encounter any issues during an upgrade when using the bcfutool command, see the following tables for information helpful when troubleshooting the issue.


TABLE 3 System Firmware Troubleshooting

Error Message or Description

Probable Cause

Possible Fix

send/receive failed

An incomplete firmware image download caused the upgrade to abort.

Retry bcfutool command.

PCP FW Download: Fail to start Auto FW update

After successful transfer of sysfw image, bcfutool was invoked but the invocation failed.

Retry bcfutool command.

FAIL to power off host - System Firmware is not updated

Service processor tried to power off the Sparc processor and checked for successful poweroff. Poweroff failure caused upgrade to abort.

Retry bcfutool command.

FAIL to update System Firmware
Power on host

Service processor invoked the update sequence to update flash components and encountered an error. The Sparc is powered on again.

Retry bcfutool command.

FAIL to reset SP

An error occurred when the service processor invoked the reset to complete the update.

Retry bcfutool command after power cycling the blade.

Power cycling the blade server can be done either by hotswapping the blade server or from ATCA standard ShMM using deactivate/activate sequence.

Latch opened during upgrade

An open latch event will be detected by IPMC, and after negotiating with ShMM, the blade can be brought down to M0 state, which will cause the system firmware update to fail. Because the new firmware is not fully activated, the blade will come up with original firmware on the next boot up.

However, there is a small vulnerability window (less than two minutes) where the non-redundant firmware portion is updated. If the latch is opened during this window, the non-redundant firmware becomes corrupted.

Close the latch then retry bcfutool command.

If the upgrade fails, then the redundant firmware is probably corrupted.

If the system will not power on ultraSparc T2 and boot Solaris OS, then the firmware is corrupted.

Recover as follows:

If using a Sun Netra CT 900 chassis, use the autofwupgrade utility from the ShMM.

If using a third-party chassis, return the blade to Sun Service for repair or replacement.

Shelf power outage

If Shelf lost power during the system firmware update, the blade is booted with the old image.

However, there is a small vulnerability window (less than two minutes) where the non-redundant firmware portion is updated. If Shelf power is lost during this window, the non-redundant firmware becomes corrupted.

After power is returned to the Shelf, retry bcfutool command.

If the upgrade fails, then the redundant firmware is probably corrupted.

If the system will not power on ultraSparc T2 and boot Solaris OS, then the firmware is corrupted.

Recover as follows:

If using a Sun Netra CT 900 chassis, use the autofwupgrade utility from the ShMM.

If using a third-party chassis, return the blade to Sun Service for repair or replacement.

 

Sudden blade extraction

The blade was extracted suddenly without following standard PICMG hotswap procedures. The service processor will function normally with old image booted up on next insertion.

However, there is a small vulnerability window (less than two minutes) where the non-redundant firmware portion is updated. If the blade is extracted during this window, the non-redundant firmware becomes corrupted.

After blade is reinserted, retry bcfutool command.

If the upgrade fails, then the redundant firmware is probably corrupted.

If the system will not power on ultraSparc T2 and boot Solaris OS, then the firmware is corrupted.

Recover as follows:

If using a Sun Netra CT 900 chassis, use the autofwupgrade utility from the ShMM.

If using a third-party chassis, return the blade to Sun Service for repair or replacement.

 

Flash hardware issues

If any of the flash devices used to store system firmware components are broken (unlikely), the blade will not come up after the update.

Because it is a hardware issue, there is no recovery for this situation. Return the blade to Sun Service for repair or replacement.


 


TABLE 4 IPMC and MMC Firmware Troubleshooting

Error Message or Description

Probable Cause

Possible Fix

IPMC communication Failure, aborting upgrade

If IPMC is reset during firmware update (for example, an internal WDT reset), the upgrade will abort.

Retry bcfutool command.

IPMC communication Failure, aborting upgrade

Because the service processor is a proxy for payload communication, a service processor reset will disrupt the upgrade.

Retry bcfutool command.

User interruption during upgrade

If a user interrupts the upgrade (for example, abruptly terminating bcfutool or rebooting Solaris OS), the upgrade will abort.

There might not be any message displayed.

Retry bcfutool command.

Latch opened during upgrade

If the latch is opened, the event is ignored and the upgrade is completed.

 

Shelf power outage

If Shelf lost power during the firmware update, IPMC firmware is corrupted.

If the upgrade fails, then the firmware is probably corrupted.

If the system will not power on and boot Solaris OS, then recover as follows:

  • If using a Sun Netra CT 900 chassis, use the upgradefw utility from the ShMM.
  • If using a third-party chassis, check with the vendor for a recovery method or tool.

Sudden blade extraction

The blade was extracted suddenly without following standard PICMG hotswap procedures. The IPMC firmware is corrupted and the blade will not power up.

If the upgrade fails, then the firmware is probably corrupted.

If the system will not power on and boot Solaris OS, then recover as follows:

  • If using a Sun Netra CT 900 chassis, use the upgradefw utility from the ShMM.
  • If using a third-party chassis, check with the vendor for a recovery method or tool.

Flash hardware issues

If any of the IPMC/MMC flash devices used to store system firmware components are broken (unlikely), the blade will not come up after the update.

Because it is a hardware issue, there is no recovery for this situation. Return the blade to Sun Service for repair or replacement.



Removing and Installing Blade Servers

To remove and replace a blade server safely, manually shut down Solaris from the Solaris prompt. For detailed instructions, refer to the Solaris Operating System documentation and the Sun Netra CP3260 Blade Server User’s Guide (820-0457).


sync;halt

When the shutdown is completed, the ok prompt is displayed.