JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun Blade 6000 Virtualized Multi-Fabric 10GE M2 Network Express Module Documentation     Sun Blade 6000 Virtualized Multi-Fabric 10GbE M2 Network Express Module Documentation Library
search filter icon
search icon

Document Information

Using This Documentation

Related Books

About This Documentation (PDF and HTML)

We Welcome Your Comments

Change History

User's Guide

Overview of Sun Blade 6000 Virtualized Multi-Fabric 10GbE M2 NEM User's Guide

Features of the Sun Blade 6000 Virtualized Multi-Fabric 10GbE M2 NEM

Terminology

Features Overview

Components Overview

NEM Ports and LEDs

Performing Hot Plug Insertion and Removal

PCIe Hot Plug

SAS Interface Hot Plug

Installing or Replacing the Virtualized M2 NEM

Installing a NEM

Verifying NEM Installation

Removing a NEM

Replacing a NEM

Installing and Removing SFP+ Optical Transceiver Modules

Cabling the SPF+ Connectors

Booting Over the Virtualized M2 NEM 10-Gigabit Ethernet Port

Booting over the Network With an x86 Blade Server

Booting over the Network With a SPARC Blade Server

Installing and Configuring the hxge Driver on a Solaris SPARC or x86 Platform

How to Configure the Network Host Files

Configuring the hxge Device Driver Parameters

Configuring the Jumbo Frames Feature

Installing and Configuring the hxge Driver on a Linux Platform

Installing and Removing the Driver on a Linux Platform

How to Download and Install the Driver on a Linux Platform

How to Remove the Driver From a Linux Platform

Configuring the Network Interface

About Temporary hxge Network Interface Configuration

About Permanent hxge Network Interface Configuration

How to Bring the Interface Online Manually

How to Configure the Network Interface File Automatically for Red Hat Linux

How to Configure the Network Interface Automatically for SUSE Linux

Checking and Testing the hxge Device

Changing the hxge Driver Configuration

How to Temporarily Configure the hxge Driver Parameters

How to Permanently Configure the hxge Driver Parameters

hxge Driver Configuration Parameters

Troubleshooting the Driver

Debug Messaging Parameter Overview

How to Set the Debug Drive Parameter Dynamically

How to Configure the Syslog Parameter

Configuring Jumbo Frames

How to Temporarily Configure Jumbo Frames Support

How to Permanently Enable Jumbo Frame Support

Installing and Configuring Drivers on a Windows Platform

Installing Drivers on a Windows Platform

Enabling Jumbo Frames

Installing and Configuring Drivers on a VMware ESX Server Platform

Installing the ESX Server Drivers on an Existing ESX Server

Installing the ESX Server Drivers With a New ESX Installation

Configuring the Virtual NEM M2 Network Adapters

Configuring Jumbo Frames

ILOM Supplement

Accessing ILOM Documentation and Updates

Connecting to ILOM

Updating the NEM Firmware

Sun Blade Zone Manager

NEM Sensors

Enabling Private and Failover Mode

Using Hot Plug Commands

Fixing Problems with Oracle ILOM Using the Preboot Menu

Product Notes

Overview of the Sun Blade 6000 Virtualized Multi-Fabric 10GbE M2 NEM Product Notes

Product Information

Supported Software and Firmware

Supported Hardware

Hardware and Networking Issues

Packet Drop at 1Gb with 100m Cable (6879222)

Special Instructions for Network PXE Booting for the Sun Blade X3-2B Server (7183913)

ILOM Issues

ILOM Failover and Private Mode Cannot Be Enabled Simultaneously (6911661)

Cannot Set NEM SP IP Address When NEM Is Powered Off (6938660)

ipmitool Command Fails (6940791)

FMA Not Supported (6938667)

Prepare-to-Remove and Return-to-Service Does Not Work on NEM ILOM (6948103)

Green Rear SAS Link LEDs All Turn On When Setting Prepare-to-Remove (6950981)

NEM SAS Firmware Updates Must be Performed from the CMM and Not the NEM SP (7020854)

Operating System Issues

Oracle Linux: Server Module Might Hang When NEM Replaced Repeatedly (6935520)

Oracle Solaris: Hotplugd(1M) Daemon Must Be Started Before Initiating Hot Plug Actions In Oracle Solaris 10 OS 09/10 (7005233)

Oracle Solaris: Virtualized M2 NEM hxge 10 GbE Driver Can Hang Under High Stress (6932684)

Linux: Configuring PCIe Hot Plug in Linux (7075694)

Windows: Incorrect Driver Version Displayed When Driver File Properties is Viewed (7069358)

Index

How to Download and Install the Driver on a Linux Platform

  1. Use the ifconfig command to obtain a list of the current Ethernet network interfaces.
    host #> ifconfig -a | grep eth
    eth0      Link encap:Ethernet  HWaddr 00:14:4F:CA:15:68  
    eth1      Link encap:Ethernet  HWaddr 00:14:4F:CA:15:69  

    After you have installed the Virtualized Multi-Fabric 10GbE M2 NEM hardware and installed and loaded the driver, a new eth device appears. This will be the eth device for the NEM.

  2. Access the hxge drivers through one of the following methods:
    • Locate and copy the hxge device driver .zip file from the Tools and Drivers CD for the server module operating system on which you want to install the driver.

      For Oracle Enterprise Linux (OEL) use the corresponding Red Hat Enterprise Linux (RHEL) driver.

      The drivers are located in one of the following directories:

      /linux/drivers/rhel5.4

      /linux/drivers/sles10/sp3

      /linux/drivers/sles11


      Note - Linux source files are also available at /linux/drivers/src


    • Download latest drivers packages from the Drivers and Firmware link on the following web site: http://oracle.com/goto/10gbenemm2.
  3. Once the Linux drivers file is downloaded, uncompress and unpack the drivers. In this example, the download directory is tge10:
    host #>mkdir tge10
    host #>cp xxx/SB6000_Virt_MF_10GbE_NEM_Tools_And_Drivers_Linux_48230ai.bz2  tge10
    host #>cd tge10
    host #>tar -xjf SB6000_Virt_MF_10GbE_NEM_Tools_And_Drivers_Linux_48230ai.bz2
  4. Select and install the appropriate OS driver package.

    In this step, RHEL5.4 Linux driver is selected for OEL version 5.4.

    1. Navigate to the Linux directory that contains the driver package that is appropriate for your OS.
      host #> cd linux/drivers/rhel5.4
      host #> ls -l
      total 897
      -rw-r--r-- 1 root root  895938 Mar  8  2010 hxge-1.2.7_rhel54-1.x86_64.rpm
      -rw-r--r-- 1 root root    1485 Mar  8  2010 readme.html
       

      The file hxge-1.2.7_rhel54-1.x86_64.rpm is the package which is to be installed.

      If you are not sure what release you are running, use the lsb_release command to show information about your host operating system.

      host #> lsb_release -a
      LSB Version:    :core-3.1-amd64:core-3.1-ia32:core-3.1-noarch:graphics-3.1-amd64:
      graphics-3.1-ia32:graphics-3.1-noarch
      Distributor ID: EnterpriseEnterpriseServer
      Description:    Enterprise Linux Enterprise Linux Server release 5.4 (Carthage)
      Release:        5.4
      Codename:       Carthage
      
      host #> rpm -q hxge
      package hxge is not installed
    2. Identify the OS-specific subdirectory, and verify that no hxge driver is currently installed.

      Note - If an hxge driver is already installed, uninstall the driver, in order to avoid complications. See How to Remove the Driver From a Linux Platform for instructions on removing the driver. The update command (rpm -u) is not supported for updating the hxge driver.


    3. Install the appropriate package (.rpm) file.
      host #> rpm -ivh hxge-1.2.7_rhel54-1.x86_64.rpm  
      Preparing...                ########################################### [100%]
           1:hxge                   ########################################### [100%]
      post Install Done

      Once you have installed the hxge driver, you can immediately load the driver. If the NEM is physically and electrically installed, the driver automatically attaches to it and makes it available to the system. Alternatively, on the next system reset and reboot, the hxge driver automatically loads if there are any NEM devices present and detected.

  5. Load the driver.
    1. Verify that the NEM is available to the system (that is, it is actively on the PCIe I/O bus).

      The command sample below has been edited for brevity. Sections marked [....] denote output that has been removed.

      host #> lspci 
      [...]
          00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
          00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
          00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
          13:00.0 Ethernet controller: Sun Microsystems Computer Corp. Unknown device aaaa (rev 01)
          19:00.0 Ethernet controller: Sun Microsystems Computer Corp. Unknown device aaaa (rev 01)
          1f:00.0 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
          1f:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
          [...]

      Device code 0xAAAA (Unknown device aaaa (rev 01) output) is the Virtualized M2 NEM Device; presence of this line indicates the NEM is visible and available to the system.

    2. Manually load the hxge driver.
      host #> modprobe hxge
    3. Verify the driver is loaded.
      host #> lsmod | grep hxge
      hxge                  175440  0  
      host#> modinfo hxge
      filename:       /lib/modules/2.6.18-164.el5/kernel/drivers/net/hxge.ko
      version:        1.2.7
      license:        GPL
      description:    Sun Microsystems(R) 10 Gigabit Network Driver
      author:         Sun Microsystems, james.puthukattukaran@sun.com
      srcversion:     270F053A5DE6A454D1D224D
      alias:          pci:v0000108Ed0000AAAAsv*sd*bc*sc*i*
      depends:        
      vermagic:       2.6.18-164.el5 SMP mod_unload gcc-4.1
      parm:           enable_jumbo:enable jumbo packets (int)
      [etc.]
      • If you get the output above, the driver is loaded into memory and actively running.
      • If the modprobe command fails, you will receive the following output:
        host #> modprobe hxge 
        FATAL: Module hxge not found. 

        This indicates that you probably installed the wrong driver version. Uninstall the hxge driver and install the correct package for your Linux release.

        If you are running a custom or patched kernel, you might have to build a custom driver to match your custom kernel.

  6. Identify the NEM eth device.

    Once the NEM has been correctly installed, and the hxge software driver has been successfully installed and loaded, the new NEM eth device will be visible.

    1. Execute the following command to view the available eth devices.
      host #> ifconfig -a | grep eth 
      eth0      Link encap:Ethernet  HWaddr 00:14:4F:CA:15:68  
      eth1      Link encap:Ethernet  HWaddr 00:14:4F:CA:15:69  
      eth2      Link encap:Ethernet  HWaddr 00:14:4F:29:00:1D  

      In this example, eth0 and eth1 were previously present; eth2 , is for a new NEM Ethernet network interface device. You can identify each eth device (NEM0 or NEM1) by matching the Ethernet MAC address with the one you recorded and saved when you physically installed the NEM into the chassis in Installing a NEM.

    2. Make sure that the eth2 driver is the correct Ethernet driver for the Virtualized M2 NEM.
      host #> ethtool -i eth2 
      driver: hxge
      version: 1.2.7
      firmware-version: N/A
      bus-info: 0000:19:00.0
    3. For more detail on eth2, use the ifconfig command.
      host #> ifconfig eth2 
        
      BROADCAST MULTICAST  MTU:1500  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
      Memory:f9000000-fa000000 

      The eth2 device is active and available to the system, but has not yet been configured (assigned an IP address). See the next section for details on configuring the NEM for the Linux OS.