Sun Blade 6000 Virtualized Multi-Fabric 10GbE Network Express Module Product Notes |
These product notes contain important and late-breaking information about Oracle’s Sun Blade 6000 Virtualized Multi-Fabric 10GbE Network Express Module (Virtualized NEM).
The following topics are covered in this document:
The following operating systems are supported for the Sun Blade 6000 Virtualized Multi-Fabric 10GbE NEM.
This section covers the following topics:
The following updates are included in software version 2.2:
You can download SW 2.2 as follows:
1. Navigate to the following site:
http://www.oracle.com/goto/blades
2. Click on Drivers and Firmware in the Download box on the right side of the page.
The Sun Blade Systems - Firmware page is displayed.
3. Click on Network Express Modules.
The Sun Blade Systems Netwok Express Modules page is displayed.
4. Navigate to “Sun Blade 6000 Virtualized Multi-Fabric 10GbE Network Express Module Software 2.2”.
5. Click the download button, and log in as instructed, to access the firmware download.
Tools and Drivers CD updates are available at:
http://www.sun.com/servers/blades/downloads.jsp
Refer to the Sun Blade Virtualized Mult-Fabric 10GbE Network Express Module User’s Guide for instructions on instlling the Virtualized NEM drivers.
The following sections contain information about Tools and Drivers CD versions:
The Tools and Drivers CD version 2.1 (SW 2.1) contains the following updates:
The Tools and Drivers CD version 2.0 (SW 2.0) contains the following updates:
ILOM version 2.0.3.12 build17 r44665 is also available with SW 2.0.
For issues fixed in SW 2.0, see Issues Fixed in SW 2.0.
The Tools and Drivers CD version 1.0.1 (SW 1.0.1) contains the following updates:
The following table lists the server blades, Fabric Express Modules (FEMs) and RAID expansion modules (REMs) that are supported with the Virtualized NEM. server blades that do not have FEMs or REMs listed have onboard network controllers or SAS expanders.
The following are known issues for the Sun Blade 6000 Virtualized Multi-Fabric 10GbE Network Express Module (Virtualized NEM).
The following are known hardware issues and networking issues for the Virtualized NEM:
On rare occasions, the Virtualized NEM might shutdown unexpectedly.
To prevent this occurrence, you need to update the Virtualized NEM main 12V power sequencer firmware.
Contact Oracle Service to gain access to the sunservice account for the CMM.
Note - System damage can occur. Perform the firmware update on an offline NEM and do not attempt this procedure without first contacting Oracle Service. |
This section contains the following procedures:
1. Download the following file from the Virtualized NEM SW 2.2 download site: NEMHydraPwrUpd10.zip.
See Software Version 2.2 for information on how to locate the download site.
2. Unzip the NEMHydraPwrUpd10.zip file.
You will need the following files for this procedure:
Note - Make sure that you have execute permissions on these files. If you do not have execute permissions, use the chmod +x command on the files before copying them to the CMM. |
3. Copy the files listed in Step 2 into the CMM’s /coredump directory.
You will need information from Oracle Service to complete this step. If you have not already contacted Oracle Service, do so now.
5. Navigate to the CMM /coredump directory:
6. Update the NEM in slot 0, with the following command:
7. If necessary, update the NEM in slot 1:
Note - Each command may take more than two minutes to complete. |
8. Power cycle the Virtualized NEM.
9. Verify that the correct firmware version is installed with the following command:
The following output displays if the firmware has been correctly updated:
1. Download the following file from the Virtualized NEM SW 2.2 download site: NEMHydraPwrUpd10.zip.
See Software Version 2.2 for information on how to locate the download site.
2. Unzip the NEMHydraPwrUpd10.zip file.
You will need the following files for this procedure:
Note - Make sure that you have execute permissions on these files. If you do not have execute permissions, use the chmod +x command on the files before copying them to the CMM. |
3. Copy the files listed in Step 2 into the CMM’s /coredump directory.
You will need information from Oracle Service to complete this step. If you have not already contacted Oracle Service, do so now.
5. Navigate to the CMM /coredump directory:
6. Restore the NEM in slot 0, with the following command:
7. If necessary, restore the NEM in slot 1:
Note - Each command may take more than two minutes to complete. |
8. Power cycle the Virtualized NEM.
9. Verify that the correct firmware version is installed with the following command:
The following output displays if the firmware has been correctly updated:
If a server module with a SAS-2 RAID Expansion Module (REM) is inserted into a Sun Blade 6000 modular system chassis that has SAS-1 NEMs and a Sun Blade 6000 disk module, the server blade’s SAS-2 REM might hang.
To prevent this problem, you need to upgrade the firmware of your SAS-1 components (SAS-1 NEMs and disk modules) to a firmware version that supports SAS-1/SAS-2 coexistence. This upgrade must be done before you insert the SAS-2 server module into the chassis.
At a minimum, all SAS expanders for SAS-1 NEMs and Sun Blade 6000 disk modules must be upgraded to firmware revision 5.04.03 to allow SAS-1/SAS-2 device coexistence in the Sun Blade 6000 modular system chassis. SAS expander firmware version 5.04.03 is available in the Virtualized NEM software release 2.2.
See the SAS-1/SAS-2 Compatibility Upgrade Guide (821-1800) for details on obtaining the firmware and performing the upgrade.
The Sun Blade T6320 and T6340 server modules require the Sun Blade T63X0 PCIe Pass-Through Fabric Expansion Module (FEM) in order to use the 10 GbE functionality of the Virtualized NEM.
Refer to the Sun Blade T63x0 PCI Pass-Through Fabric Expansion Module User’s Guide for information on the release of this FEM and additional software and firmware requirements for these server modules.
On-board Ethernet devices might see an excessive rate of packet drop using cables 100 meters or longer when link speed is negotiated at 1GB.
When configured with IPMP, hxge interfaces might failover to the alternative interface without any real faults occurring. This is primarily due to IPMP’s path probes failing. Path probes may fail when either the system or the given network interface is over utilized.
Link aggregation is not supported on the virtualized NEM. If multiple server modules are performing independent link aggregations, this will lead to unpredictable results. This is due to limitations in the link aggregation standard from the IEEE.802.1AX, specifically the Link Aggregation Control Protocol (LACP).
When using the Virtualized NEM for PXE booting over the 10 GbE interface, rbr_empty and UDP checksum errors might be displayed during the boot process (See FIGURE 1). This should not affect the PXE installation.
FIGURE 1 Errors Displayed During PXE Boot
When you press the F8 key or view the BIOS Setup Utility boot priority list, only one virtualized NEM device will appear.
To boot from the NEM that is not listed, do the following.
1. Choose the NEM listed in the boot device list.
FIGURE 2 and FIGURE 3 show the boot device list that appears when you either press F8 or use the boot list in the BIOS Setup Utility.
FIGURE 3 BIOS Setup Utility Boot List
2. Select the Virtualized NEM shown in the list.
The system will start to boot from the first Virtualized NEM device with matching device and vendor ids.
3. Press the ESC key to move to the next virtualized NEM device and boot from it.
The following are known ILOM issues for the Virtualized NEM:
If you use the ILOM proxy show /NEM0/state command to view NEM power state, the value property will display Power Off, even though the NEM is powered on.
Ignore the properties value given by the proxy show /NEM0/state command.
Do one of the following to read the correct power state:
a. Log on to the Virtualized NEM SP from the chassis CMM:
b. Use the following command to show the NEM properties:
After using the reset command to reset the Virtualized NEM using the ILOM proxy, you might receive a warning message, as shown in the following example output:
The failure message can be safely ignored.
If you try to set the NEM LED behavior using the ILOM proxy, the command will appear to work in the CLI, but it will not change the LED behavior.
You can use the standard CMM ILOM or NEM SP to set the LED behavior.
Refer to the ILOM Supplement section of the Sun Blade 6000 Virtualized Multi-Fabric 10 GbE Network Express Module User’s Guide.
The following are known operating system issues for the Virtualized NEM:
When the hxge driver is installed on a sever running ESX 4.0, a system panic can occur after changing the maximum transmisstion unit (MTU) value to a jumbo frame size. This issue occurs due to a bug in vmkernel.
This issue is VMware PR 518560.
Change the MTU value to a jumbo frames size when there is no current network activity on the server.
When attempting to load the hxge driver on SLES11, it fails to load with the following error:
FATAL: module ’/lib/modules/2.6.27.19-5-default/kernel/drivers/net/hxge.ko’ is unsupported Use --allow-unsupported or set allow_unsupported_modules to 1 in /etc/modprobe.d/unsupported-modules |
Load the driver with the --allow-unsupported flag as follows:
A blade server running Solaris OS and recovering from RBR empty might fail the test when under heavy stress.
The message "Not digitally signed" will display in Windows Server 2008 device manager for each instance of the Sun Blade 6000 Virtualized Multi-Fabric 10GbE Network Express Module. This is a known issue in Windows Server 2008.
You can ignore this message. The Sun Blade 6000 Virtualized Multi-Fabric 10Gbe Network Express Module network drivers are signed by Microsoft.
When using the NTttcp application in Windows Server 2003 or 2008 OS, and the network cable is disconnected and reconnected within a short period of time, the NTttcp might stop transmitting traffic and exit.
Restart the NTttcp network application.
When using the Solaris OS, customers might experience changes to their network device naming instances (hxge0 and hxge1) on a server module when moving from one slot to an alternative slot.
Use the following workaround to identify the NEM network device:
1. Note the MAC address label assignment on the Sun Blade 6000 Virtualized 10Gb NEM located on the inner left hand side of the NEM module.
2. Compare this MAC Address with the network device instances on the server module using ifconfig -a, and note down the first 5 octets that match.
For example, if the NEM module MAC address label has 00:14:4F:00:00:00, the server module slot zero will be assigned MAC addresses 00:14:4F:00:00:01 through 00:14:4F:00:00:04.
This is the normal behavior in the way Solaris performs its enumeration of network device instances.
After Jumbo Frames is enabled or disabled for the Virtualized NEM (Sun Blade 6000 10Gbe Networking Controller) on a Sun Blade X6440 server module running Windows, the network connection for Virtualized NEM could become disabled.
FIGURE 4 shows a yellow error icon next to the disabled Sun Blade 6000 10GbE Networking Controller in the Device Manager display.
FIGURE 4 Disabled Sun Blade 6000 10GbE Networking Controller
Reboot the server module to recover the network connection. After the reboot, the network connection will be up and running with the desired Jumbo Frame setting.
FIGURE 5 shows the yellow error icon removed from the Sun Blade 6000 10GbE Networking Controller, indicating that the network connection is enabled in the Device Manager display.
FIGURE 5 Enabled Sun Blade 6000 10GbE Networking Controller
When running ethtool on a RHEL 5.2 64-bit system to determine the speed of the Virtualized NEM, ethtool will report an unknown speed. This is because RHEL 5.2 64-bit has an older version of ethtool.
When unloading the RHEL 4.7 version of the Virtualized NEM Linux driver, the following type of warning message will appear:
This is a benign message and can be ignored.
The hxge driver is not bundled with any of the server module operating systems and cannot be installed over a 10GbE interface.
Note - Support for operating system installation over the 10 Gigabit Ethernet interface is planned for future OS releases. |
Create a customized boot archive with hxge driver support.
The following document shows how to customize the boot archive for the Solaris operating system.
http://www.sun.com/blueprints/0806/819-7546.html
For Linux or Windows systems, refer to the documentation from your OS distribution to create a customized boot archive with the hxge driver.
The following issues have been fixed. Unless otherwise noted, the software updates mentioned are for the Virtualized NEM.
After disabling and re-enabling one or both of the 10GbE network devices, one device will sometimes disappear from device manager. The device might also flag problem code 10 (cannot start).
Also, if driver is re-loaded manually, then disabled and re-enable d, the same failure might occur.
The software updates included in SW 2.0 fix the following issues:
When you update to ILOM version 2.0.3.12 build17 r44665 (SW 2.0), the Fcode firmware will be updated to version 1.4, which fixes CR 6861451.
This issue is fixed in Sun Blade X6220 server SW 3.0 (BIOS 114).
When using a Sun Blade X6220 server module to perform a network boot with Sun Blade 6000 Virtualized 10GbE Multi-Fabric NEM, the network boot will not work if you use the F12 key.
Select F8 during BIOS Post and select PXE:SUN 10GNemV 1.8 GPL etherboot.
This issue is fixed with the Windows Server 2008 driver update in SW 1.0.1.
This issue is fixed with the Windows Server 2003 driver update in SW 1.1.
On a Windows Server 2003 OS server module with Sun Blade 6000 Virtualized 10GbE NEM in non-failover mode, if the network link is lost (for example, the network cable is unplugged) there might not be an indication in the Windows “Control Panel->Network Connections" window.
Observe the link LED light above the SFP+ ports on Oracle’s Sun Blade 6000 Virtualized 10 GbE NEM.
If the LED is not illuminated, then the network link is down.
Copyright |
Copyright © 2009, 2010, Oracle and/or its affiliates. All rights reserved.
If this document has been previously published, replace “XXXX” with the first year of publication. Otherwise, use only 2010 as the copyright year. Remove this paragraph and text.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
If this is software or related software documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:
U.S. GOVERNMENT RIGHTS. Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA 94065.
This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications which may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open Company, Ltd.
This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.
Copyright © 2009, 2010, Oracle et/ou ses affiliés. Tous droits réservés.
Ce logiciel et la documentation qui l’accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d’utilisation et de divulgation. Sauf disposition de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, breveter, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé que ce soit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d’interopérabilité avec des logiciels tiers ou tel que prescrit par la loi.
Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu’elles soient exemptes d’erreurs et vous invite, le cas échéant, à lui en faire part par écrit.
Si ce logiciel, ou la documentation qui l’accompagne, est concédé sous licence au Gouvernement des Etats-Unis, ou à toute entité qui délivre la licence de ce logiciel ou l’utilise pour le compte du Gouvernement des Etats-Unis, la notice suivante s’applique :
U.S. GOVERNMENT RIGHTS. Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA 94065.
Ce logiciel ou matériel a été développé pour un usage général dans le cadre d’applications de gestion des informations. Ce logiciel ou matériel n’est pas conçu ni n’est destiné à être utilisé dans des applications à risque, notamment dans des applications pouvant causer des dommages corporels. Si vous utilisez ce logiciel ou matériel dans le cadre d’applications dangereuses, il est de votre responsabilité de prendre toutes les mesures de secours, de sauvegarde, de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité. Oracle Corporation et ses affiliés déclinent toute responsabilité quant aux dommages causés par l’utilisation de ce logiciel ou matériel pour ce type d’applications.
Oracle et Java sont des marques déposées d’Oracle Corporation et/ou de ses affiliés.Tout autre nom mentionné peut correspondre à des marques appartenant à d’autres propriétaires qu’Oracle.
AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques déposées d’Advanced Micro Devices. Intel et Intel Xeon sont des marques ou des marques déposées d’Intel Corporation. Toutes les marques SPARC sont utilisées sous licence et sont des marques ou des marques déposées de SPARC International, Inc. UNIX est une marque déposée concédée sous licence par X/Open Company, Ltd.
Ce logiciel ou matériel et la documentation qui l’accompagne peuvent fournir des informations ou des liens donnant accès à des contenus, des produits et des services émanant de tiers. Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l’accès à des contenus, produits ou services tiers, ou à leur utilisation.