Sun Netra CT 900 Server Product Notes

This document contains important and late-breaking information about Oracle’s Sun Netra CT900 server and related components. This document includes instructions for upgrading to ATCA R3U3 from earlier releases.

This document contains the following:

• Known Issues
• Additional Copyright Information
• Related Documentation
• Enhancements
• New Features
• Upgrading the Software and Firmware
• Increasing the FRU Image Size
• Obtaining Rear Transition Module FRU ID
• Maximum Serial Cable Length
• Failure LED on Power Entry Module
• Shielded Cables for Serial Ports
• Serial Number Label Location

Caution - This release is for evaluation purposes only. Compliance testing has not been completed on this release.

---

Known Issues

TABLE 1 shows the known issues with this release.

Changing REDUNDANCY_ENABLED Variable

Do not change the default TRUE setting for the REDUNDANCY_ENABLED variable in the /etc/shelfman.conf file to FALSE. A system with two shelf management cards uses this entry in the /etc/shelfman.conf file to provide redundancy for the shelf management cards.

Changing the default TRUE setting for the REDUNDANCY_ENABLED variable in the /etc/shelfman.conf file to FALSE causes the shelf management card to reboot repeatedly.

---

Additional Copyright Information

This product is covered and controlled by U.S. Export Control laws and may be subject to the export or import laws in other countries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether direct or indirect, are strictly prohibited. Export or reexport to countries subject to U.S. embargo or to entities identified on U.S. export exclusion lists, including, but not limited to, the denied persons and specially designated nationals lists is strictly prohibited.

---

Related Documentation

The following URL provides documentation related to this product:

http://www.oracle.com/pls/topic/lookup?ctx=ct900&id=homepage

---

Enhancements

The R3U3 release and patches provide the following enhancements:

• Automated firmware upgrade support
• Blade server deferred upgrade activation
• Linux (RedHat 5.3) firmware for BIOS
• Retaining custom IPMC configurations during upgrades
• Documentation for configuring SNMP traps and processing notifications
• Shelf Manager upgrades

The following paragraphs briefly describe each enhancement.

Automated Firmware Upgrade Support

The tool is enhanced to include support for new products, to increase reliability, and to reduce upgrade durations.

The following new command line options are provided with the R3U3 release.

The firmware upgrade tool provides a single automated upgrade of Sun Netra CT900 servers and Netra products. This tool reduces the labor and time needed to upgrade your servers, boards/blades, and ARTMs to a new or updated release.

The autofwupgrade command is available on the ShMM and supports automatic firmware upgrades from the R2.0.3 and newer releases to R3U3 for the following:

• System firmware (Sun Netra CP3060 and Sun Netra CP3260)
• IPMC boot firmware (Sun Netra CP3010, Sun Netra CP3020, Sun Netra CP3060, Sun Netra CP3220, Sun Netra CP3250, Sun Netra CP3260, Sun Netra CP3270, and Netra SPARC T3-1BA)
• IPMC firmware (Sun Netra CP3010, Sun Netra CP3020, Sun Netra CP3060, Sun Netra CP3220, Sun Netra CP3250, Sun Netra CP3260, Sun Netra CP3270, and Netra SPARC T3-1BA)
• MMC firmware for XCP32X0-RTM-HDD, XCP32X0-RTM-FC, XCP32X0-RTM-NT (display only)

For detailed instructions on how to use the firmware upgrade tool and how to roll back an upgrade, refer to the README files included with the R3U3 package.

The following firmware are not upgraded by the autofwupgrade tool and must be upgraded manually. (For manual upgrade instructions, refer to README files included in the release download package.)

• NIC and BIOS firmware
• ATCA switch card firmware
• ShMM firmware
• OBP for Sun Netra CP3010 boards, Sun Netra CP3060 blade servers, and Sun Netra CP3260 blade servers
• System firmware for Netra SPARC T3-1BA

For users who do not have access to the ShMM or who want manual control of the firmware upgrade process, use upgrade tools such as upgradefw and others instead of the ShMM autofwupgrade. Refer to the README files for instructions.

Blade Server Deferred Upgrade Activation

The latest release of bcfutool and upgradefw tool introduced support for the deferred firmware activation feature on the Netra SPARC T3-1BA. Refer to the README file in the firmware release bundle for detailed information.

Linux (RedHat 5.3) Firmware for BIOS

Support was added for the Linux BIOS Update Utility. This utility is a stand-alone application for Linux that incorporates the BIOS payload and application required to perform a BIOS update. The utility uses the /dev/mem interface to access the BIOS.

For this application to work on a Linux system:

• The LPC interface driver must be configured within the kernel to allow writes to the BIOS
• The BIOS must be mapped into /dev/mem
• CP32x0 BIOS version 1.03 or newer must be installed to perform a BIOS update

The CP32x0 BIOS consists of three regions:

1. Boot BIOS Region, which contains the lowest level system initialization.

2. NVRAM Region, which stores system specific configuration data.

3. Main BIOS Region, which includes system configuration code.

The utility can be used to update any or all of these regions. Typically, only the main BIOS region needs to be updated, but in some cases it might be necessary to update the Boot BIOS region

For detailed instructions, including how to identify and backup current BIOS version, refer to the README file.

Retaining Custom IPMC Configurations Across Upgrades

IPMC configuration variables are now retained across firmware upgrades and downgrades, as long as the version being downgraded to supports this feature.

IPMC has a group of parameters that are stored in SEEPROM (NV Storage). All variables are part of one data structure. Each of these variables performs a function and can be set or read via commands. These variables are stored across IPMC resets. However, previously, these variables were reset to default values when new or upgraded firmware was installed.

This enhancement retains any custom values you configured for the following variables.

If you want to reset these variables to their default, use the new Reset IPMC Configuration Variables command.

For detailed command information, refer to the Sun Netra CT 900 Server Administration and Reference Manual.

Documentation for Configuring SNMP Traps and Processing Notifications

Asynchronous events of interest in the system are communicated to the SNMP managers through the use of SNMP version 1 and 2 traps.

In addition to the set of notifications supported by the hpiSubagent (based on OpenHPI events and defined in the HPI-B0101-MIB, as of R3), the master agent (snmpd) also generates generic traps such as cold start during the daemon start-up.

Information and instructions for configuring traps and processing notifications were added to the Sun Netra CT900 Server Software Developer’s Guide.

Shelf Manager Upgrades

The following enhancements to the Shelf Manager (ShMM) are available with this release:

• New fan-speed acoustic level commands
• ShMM OpenHPI upgrades
• ShMM firmware version 2.6.4 upgrade

The following sections briefly describe each enhancement.

New Fan-Speed Acoustic Level Commands

Two new commands allow customers who are using the Sun Netra CT900 server chassis and running the default SUNCT900 cooling algorithm to set the default base fan speed to various acoustic levels based on custom configuration requirements.

• setacousticlevel
• getacousticlevel

Previously fan speed 5 was used when inlet temp was below 32C, regardless of acoustic configurations. This base fan speed was static and any attempts to change this speed were ignored by the system. These new commands allow you to set the base fan speed to a value other than 5, based on the acoustic level needs.

Fan speed acoustic levels are configurable using the ShMM clia interface. When you set an acoustic level, the variable will be changed and the default cooling algorithm will set the fan speeds accordingly. This change will take effect dynamically, is persistent, and no ShMM reboot is required.

For detailed information about using these commands, refer to the Sun Netra CT900 Server Administration and Reference Manual.

ShMM OpenHPI Upgrades

With the R3U3 release, the ShMM OpenHPI is migrated to latest OpenHPI version 2.14.0 and OpenHPI Subagent version 2.3.4. This upgrade fixes known issues and provides the following enhancements and additions:

• Support for HPI B.02.01 and B.03.01
• Build and compile OpenHPI on Solaris
• Basic support for MicroTCA shelves, creating both slot resources and FRU resources for MicroTCA entities with appropriate entity paths
• Support for DIMI and FUMI capability

ShMM Firmware Version 2.6.4 Upgrade

The Shelf Manager version is upgraded to 2.6.4. This upgrade fixes known issues in ShMM software and provides the following enhancements and additions:

• Support for RMCP+ protocol
• Full set of firmware firewall commands in accordance with the IPMI 2.0 specification. In addition, several firmware firewall-oriented CLI commands are added (firewall, setcommandpolicy, setfunctionpolicy).
• Supports RMCP and RMCP+ traffic between the Shelf Manager and the System Manager over virtual LANs. Configuration commands defined in the IPMI 2.0 specification for this purpose (Get/Set LAN Configuration Parameters) are supported; new Shelf Manager configuration parameters are defined for configuring default values.

• Support for the IPMI 2.0 extensions to the LAN configuration parameters on the IPMI command level and also in the CLI, WEB, and SNMP interfaces.
• ShMM will respond with IPMI 2.0 for get device id requests; also accepts IPM controllers that report IPMI 2.0 and IPMI 1.5.
• Support for revision 3.0 of the PICMG 3.0 (AdvancedTCA Base) specification.
• New sensors: FT oper.status, Cooling state, Fans state, and Reboot reason. The first three sensors report overall cooling state information to the System Manager (if any).
• Fine tuned kernel network drivers.
• Parallel reading of SDRs from IPM controllers during Shelf Manager initialization has been implemented to reduce initialization time.
• The completion code for the IPMI command “Exit SDR Repository Update Mode” has been changed to “Command Not Supported In Present State” instead of "Invalid Command" for formal compliance with the PICMG 3.0 specification.
• The configuration parameter TACHOMETER_THRESHOLD_UPDATE_DELAY has been added to control the delay between setting the fan level and updating the fan tachometer thresholds in shelves where dynamic fan tachometer thresholds are supported.
• The Shelf Manager now implements special log messages for tracking FRU activation and deactivation commands sent by System Manager applications.
• The configuration variables FAN_LEVEL_STEP_UP and FAN_LEVEL_STEP_DOWN are now used when changing the dynamic minimum fan level in the cooling algorithm (previously step value 1 was always used).
• The following utilities, which are part of the RFS image, have been enhanced:
• New option in the ipmb_traced utility to allow limiting trace file size.
• New mode in the patch_fru_info utility to handle network parameters in IP Connection record.
• New mode in the snmptc utility to dump traps in all supported formats into the local syslog in a user friendly format.

• The SNMP subsystem implements a new SNMP branch sel-info and the events branch has been updated to simplify event handling via the SNMP interface.
• A new configuration variable UNCONDITIONAL_SDR_REREAD_ON_VERSION_CHANGE has been added. If its value is TRUE, the Shelf Manager unconditionally re-reads SDRs from an IPM controller when it receives a Version Change event from that controller, even if the Sensor Population Change Indicator in the “Get Device SDR Info” response does not change. This is done for the benefit of ATCA boards that are not fully IPMI-compliant in this respect. The default value of this variable is FALSE.
• For TELCO alarms triggered via PEF, the CLI command alarm info now displays information about the corresponding PEF filter and the event that triggered the alarm.
• A new configuration variable DHCP_FOR_RMCP_ONLY causes the DHCP client in the Shelf Manager to assign DHCP address only to the RMCP interface.
• A new configuration variable ACTIVATE_LOCAL_WITHOUT_SHELF_FRU allows activation of local Shelf Manager IPMCs (both for the logical and physical Shelf Managers) even if the Shelf FRU Info is not found.
• Re-reading SDRs from external IPM controllers in the case of an SDR population change is now done more gracefully, avoiding a window in time when no available sensors could be reported via the CLI for that IPM controller.
• If Shelf FRU Information is obtained from external IPMCs, and the configuration variables SHELF_FRU_IPMB_SOURCE1 and SHELF_FRU_IPMB_SOURCE2 are set, no other IPM controllers are now considered as potential sources for the Shelf FRU Information.
• A new option -d for the CLI command sensordata shows only discrete sensors having a specific state set in the state mask.
• The default file /var/nvdata/pef file is now created with PEF control disabled to match the /etc/nvdata.orig/pef file that is placed into /var/nvdata at first RFS startup.
• The default value of the configuration variable IGNORE_FAILED_DIRECTED_POWER_DOWN is now TRUE, so that failure to power down a FRU in the critical thermal alert state does not by default cause the whole shelf to be powered down.
• In the zoned cooling management support, processing sensor events and querying sensor values have been optimized.
• The CLI command frudataw now performs synchronous writes to non-local FRU information sources and does not return until the write is complete. Previously, an asynchronous write took place.
• The Shelf Manager termination procedure has been reworked to speed up termination and remove erroneous messages shown during termination.
• The utility patch_fru_info now uses the locking mechanism defined in the PICMG 3.0 specification to access the Shelf FRU Information.
• A new deadlock detection mechanism has been added, in which internal data structures that represent locks are periodically analyzed for circular dependencies among waiting threads.
• The CLI can now parse AMC Clock E-Keying records.
• When started with the watchdog enabled, the Shelf Manager now clears the CPLD flag “Independent IPMB/Watchdog” (which allows the Shelf Manager to operate properly without turning the watchdog on). This flag could have been set by a previous invocation of the Shelf Manager.
• When sending a Platform Event Trap, the Shelf Manager now verifies that the corresponding network adapter is physically connected to the network and returns failure if that is not the case. This is done to improve the reliability of delivery for unacknowledged Platform Event Traps.
• Two new SNMP PET formats, Plain Text and Multi-Variable, have been added, selectable via the new parameter pet_format for the CLI commands getpefconfig and setpefconfig. The snmptc utility has been updated to handle these new formats.
• A second Shelf Manager IP Connection Record in the Shelf FRU Information is now supported as the source for the IP address for the second RMCP-capable network interface on the ShMM.
• Broadcast IP addresses are now properly settable on RMCP LAN interfaces.
• The CLI command minfanlevel has been modified in the case of zoned cooling to allow setting minimum fan levels separately for different zones and to show the minimum fan level separately for each zone.
• Support for settable sensors in the Shelf Manager is improved, including a new CLI command setsensordata.
• The CLI command shelf -v fs has been extended to print a warning if any of the fan trays described in the shelf Address Table are not operational.
• In the CLI commands getfanpolicy and setfanpolicy, the default format is now hexadecimal for the fan tray IPMB address, to match the rest of the CLI commands.
• The Shelf Manager now does a retry for writing data into the AT24LCxx EEPROM, to deal with transient write errors.
• The Shelf Manager now supports using SHMM GPIO E8 for controlling the I2C multiplexor on the ShMM carrier.
• There is now support for swapping network interfaces during Shelf Manager switch-overs in cross connected configurations using the configuration parameter SWAPPED_CROSS_CONNECTS.
• The propagation of session state over the redundancy interface can now be delayed to reduce the volume of data sent over the redundancy interface during intensive RMCP operation.
• The i2ca utility now supports arbitrary I2C write-read transactions.
• The rupgrade_tool launcher now displays a time-stamped message and analyzes the completion code.
• The utility patch_fru_info now implements Internal Use Area access functionality (text and hexlist modes). It sets the Shelf Manager debug mode automatically if the name of the executable contains .debug.
• OEM RMCP commands are now implemented for accessing cached device SDRs on the Shelf Manager (primarily for the benefit of OpenHPI-based System Managers).
• The utilities patch_fru_info and fru_internal now display version information.
• A new Shelf Manager configuration variable ENABLE_DIRECT_SHELF_FRU_WRITE makes it possible to continue using RMCP client applications that do not comply with ECN 3.0-2.0-002 restrictions that all writes to Shelf FRU Info be accomplished with a special locking protocol, to ensure that only one writer is active at a time.
• SNMP support has been added to read and set TELCO alarms (minor, major, and critical).
• The Shelf Manager startup scripts have been updated in the following areas:
• improved remote logging support via the log_remote U-Boot variable;
• adding a logging=upgrade mode to dump syslog messages to the flash partition.

• The CLI command frudataw, when writing data to the master copy of Shelf FRU Information (FRU 254 on BMC), now uses the FRU Inventory Device Lock and Write commands.
• The output of the dumplog utility has been enhanced to display additional information about locks, when invoked in verbose mode.
• The Shelf Manager now does not consider that an IPM controller has been replaced, and preserves the identification of that IPM controller, if only the Firmware Revision and/or Auxiliary Firmware Revision fields are changed in the “Get Device ID” response for the IPM controller. This allows the Shelf Manager to better distinguish firmware upgrades of an IPM controller from the removal and reinsertion of a different IPM controller.
• The major Shelf Manager interface facilities (specifically, the CLI and the RMCP interface) are now monitored for deadlocks by the watchdog timer. A deadlock that makes either of those Shelf Manager interfaces unresponsive now causes the hardware watchdog timer to trigger and reset the Shelf Manager; the backup Shelf Manager takes over. In that case, a dump of the current state of the Shelf Manager’s internal locks is written to the file /var/nvdata/dumplog.bin on the Flash file system. This file can be later analyzed for clues about the cause of the failure with the dumplog utility via the option -f.
• Forced switch-overs from the backup Shelf Manager to handle the case of a frozen active ShMM (CLI command switchover -force) are now supported. No negotiations between the Shelf Managers happen in this case; the backup Shelf Manager immediately takes control in hardware.
• Support for retrieving and setting fan levels in the SNMP interface has been added.
• When responding to the IPMI command “Send Message,” the sequence number in the bridged response is now taken from the encapsulated command. Previously it was taken from the Send Message wrapper command. This change is considered to be more in line with the IPMI specification; the old behavior can be restored by setting to FALSE the configuration parameter INNER_SEQUENCE_NUMBER_IN_SEND_MSG_RESPONSE.
• The window size of RMCP sequence numbers is now configurable via the configuration parameter SESSION_SEQUENCE_WINDOW, with the default being 128, instead of a fixed size of 8. This improves Shelf Manager resiliency against losses of packets in the network.
• Support for on-ShMM event handling via PEF has been added. When an event takes place, a designated local executable or script running on the ShMM is invoked and passed the fields of the event message as parameters.
• The SNMP MIB file PPS_SENTRY-MIB is now stored on the ShMM in the directory /etc/snmp and is available via the Shelf Manager’s Web interface.
• Support has been added for software emulation of the “ShMM site-dependent” cross-connect type on shelves with the “ShMM site-independent” cross-connect type and vice versa. (Please see the ShMM cross-connects specification, PICMG ECN 3.0-2.0-001, for details on these two types of cross-connects.)

---

New Features

The following new features are available in the download package and patches.

• Commands for setting and displaying management port access
• Commands for setting and displaying management port routing

Commands for Setting and Displaying Management Port Access

Two new commands are available to set and get the management port configuration to access either the rear or front panel:

• Set Management Port
• Get Management Port

For command syntax and descriptions, refer to the Sun Netra CT900 Server Administration and Reference Manual.

Commands for Setting and Displaying Management Port Routing

Three new commands are available to set and query the management port routing configuration:

• setmgmtportroute: Sets the management port route, including front or rear access.
• getmgmtportroute: Displays the ShMM management port routing configuration.
• mgmtportstate: Queries the IPMC for management port state or port routing configuration.

For command syntax and descriptions, refer to the Sun Netra CT900 Server Administration and Reference Manual.

---

Upgrading the Software and Firmware

The latest release contains the most up-to-date features, enhancements, and bug fixes. Using an earlier release could limit your use of features and enhancements, and could affect your systems with known issues.

Downloading Packages

This section provides information on software and firmware packages that you must download from the Oracle Support web site.

1. Log in to https://support.oracle.com.

If this is your first time logging in to Oracle Support, register as a new user, following the links and instructions on the site.

2. Select Patches & Updates.

3. In the Patch Search area, click on the Advanced Search link.

4. For Product select Sun Netra CT 900 Server from the pull-down menu.

5. For Release, select the newest release, unless you require an earlier release.

6. For Platform, select the platform that matches your system.

7. Click on the Search button.

8. Review the Patch Search Results to find the package that you want to download.

9. Select the package you want, then download it and the associated README.

10. If the package downloaded is compressed, expand the files.

Upgrading Systems and Components

This section provides instructions on how to apply the contents of the downloaded packages.

To display current version information, preview required upgrades, and to perform an upgrade of the Sun Netra CT900 system and installed boards, invoke the autofwupgrade command from the ShMM.

The command syntax is as follows.

autofwupgrade [-h] [ [-vp] [-q <release>] [-a <release>] -f <FTP-server-IP-address>:<release-archive-root-dir>[:<user-name>:<password>] [-s <board-slot-number>] [-t board-type] ]

1. Upgrade the ShMM.

The Sun Netra CT900 chassis hosts redundant Shelf Managers (ShMM) that manage the FRUs. Among the two ShMMs, one will be active and the other standby. Refer to the README for instructions.

a. Upgrade the standby ShMM.

b. Upgrade the active ShMM.

2. Upgrade the firmware using the autofwupgrade tool.

Using this tool, firmware upgrade is an automatic upgrade where all the blade FRUs in Sun Netra CT900 chassis, except switch boards, are upgraded.

The autofwupgrade tool upgrades IPMC firmware (boot/main), MMC firmware, (ARTMs), and system firmware (Sun Netra CP3060 boards and Sun Netra CP3260 blades). BIOS or OBP upgrade is not supported at this time and has to be upgraded separately.

Follow the instructions provided in the README.

  Refer to instructions in the following sources to complete software upgrade and installation processes on blades and other components:

• README files in the download package
• Sun Netra CT900 Server Installation Guide
• Sun Netra CT900 Server Administration and Reference Manual
• Sun Netra CP3010 Board Product Notes
• Sun Netra CP3020 Blade Server Product Notes
• Sun Netra CP3060 Blade Server Product Notes
• Sun Netra CP3220 Blade Server Product Notes
• Sun Netra CP3250 Blade Server Product Notes
• Sun Netra CP3260 Blade Server Product Notes
• Sun Netra CP3270 Blade Server Product Notes
• Netra SPARC T3-1BA Blade Server Product Notes
• Sun Netra CP32x0 Advanced Rear Transition Module Product Notes
• Sun Netra CP3x40 Switch Product Notes
• Sun Netra CP3240 Advanced Mezzanine Card Product Notes
• Sun Netra CP3000 Advanced Mezzanine Card PCIe Hard Drive and SAS Controller Product Notes

---

Increasing the FRU Image Size

To use the feature (introduced in release 2.0) for customizing FRU data fields, you must perform a one-time configuration to increase ShMM FRU image size to 2000.

After performing the rupgrade, increase the FRU image size as follows:

1. Log in to ShMM and find the FRU size.

# clia frudata 10 0 | grep size
FRU Info size: 435 

2. If the size is less than 2048 bytes, increase the size as follows.

 # clia frudata 10 0 2047 0

This command adds “0” at 2047 offset by padding zeros all the way up to 2047, which makes the size 2048.

3. Repeat the steps to modify the FRU size on another ShMM by replacing “10 0” with “12 0.”

---

Obtaining Rear Transition Module FRU ID

You can access the rear transition module (RTM) fruid using the clia command.

  Log in to ShMM, and enter the following clia command:

# clia fruinfo <addr> <fruid>

Where <addr> is the IPMB address, and <fruid> is the value for the RTM. The clia fru command returns the RTM’s fruid number. The standard value for the RTM FRU ID is 1.

---

Maximum Serial Cable Length

Serial cables must be kept to a maximum of 10 meters (approximately 32 feet).

---

Failure LED on Power Entry Module

The red Failure LED on the power entry modules in the Sun Netra CT900 server does not operate properly at this time. Disregard the instructions in the Sun Netra CT900 Server Service Manual that describes how to use the red Failure LED when troubleshooting the power entry modules in the Sun Netra CT900 server.

---

Shielded Cables for Serial Ports

You must use shielded cables when connecting to either of the serial ports on the shelf alarm panel. Refer to the Sun Netra CT900 Server Overview for the location of the serial ports on the shelf alarm panel.

---

Serial Number Label Location

The serial number label for the Sun Netra CT900 server is located at the rear of the server, directly beneath the rear cable management bracket on the right side of the server (FIGURE 1).

FIGURE 1 Sun Netra CT 900 Server (Rear)

Sun Netra CT 900 Server Product Notes

819-1180-20

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.