---

1.1 Product Features

The Sun Blade 6048 InfiniBand QDR Switched Network Express Module (FIGURE 1-1) has the following features:

• 30 InfiniBand ports for external connectivity

• 2 I4 on-board switches, both providing 2 Infiniband ports for each slot in a Sun Blade 6048 modular system shelf

• 9 intraswitch links between two switches within IB-QNEM

• 2 passthrough Gigabit Ethernet ports for each slot of IB-QNEM’s respective a Sun Blade 6048 modular system shelf

• Each IB-QNEM InfiniBand port can provide full-duplex data transfers of up to 40 Gbps (QDR)

• Each InfiniBand port is backward compatible with 20-Gbps (DDR) and 10-Gbps (SDR) devices

• The IB-QNEM logically installs inthe NEM-1 slot of a Sun Blade 6048 modular system shelf, but occupies both NEM slots due to physical form factor

• Up to 12 blade servers can connect to a single IB-QNEM. Each blade server is connected to the IB-QNEM through two 4x QDR IB interfaces and two 1 GbE interface

TABLE 1-1 lists additional features of the Sun IB-QNEM.

TABLE 1-1   Sun IB-QNEM Features

Feature	Description
Hot-plug operations supported	Enables system administrators to easily add or remove IB-QNEMs, as needed, without powering down the system
IB transfer rate (maximum)	40 Gbps (QDR) per 4x IB port for the Sun Blade X6275 server module and 20 Gbps (DDR) per 4x IB port for the Sun Blade X6270 server module. There are two 4x IB ports per server module. 1,536 Gbps aggregate throughput
InfiniBand Trade Association (IBTA) interoperability	Version 1.2
IB uplink interface	10 CXP connectors, three 4x QDR/DDR/SDR IB ports per connector
Blade server interface	24 4x IB links
Ethernet interface	24 1-Gbps passthrough ports Note - Two 1-Gbps ports are available on the Sun Blade X6275 server module and one 1-Gbps port is available on the Sun Blade X6270 server module.
IB switch chip	2 Mellanox Infiniscale IV (I4) 36-port IB switches
IB QDR cables	See IB-QNEM Cables
LED indicators	One green LED per 4x port showing IB physical link state Locate button and LED (white), OK LED (green), Service Action Required LED (amber), Ready-to-Remove LED (blue)
Form factor	Sun proprietary form factor
RoHS	5 of 6 compliant
Power consumption	240 watts maximum
Protection	Protected from electrostatic discharge (ESD) and handling damage

---

1.2 Sun Blade 6048 Platform Support

TABLE 1-2 lists supported platforms for the IB-QNEM in the Sun Blade 6048 Series Modular System.

TABLE 1-2   Hardware Requirements for the IB-QNEM

Requirements	Hardware
Sun Blade Server Module	Sun Blade 6275 Server Module Sun Blade 6270 Server Module with Sun InfiniBand Dual Port 4x DDR PCIe Fabric Expansion Modules

For information about the supported operating systems for a server module, go to: http://www.sun.com/servers/blades/os.jsp

---

1.3 IB-QNEM Indicators, Buttons, and Ports

The IB-QNEM provides a standard set of status indicators, buttons, and ports on the back panel. FIGURE 1-2 shows the back panel.

See TABLE 1-3 for a description of the components.

TABLE 1-3   Sun Blade 6048 IB-QNEM Back Panel Components and Functions 

Number	Component Name	Color	Description
1	Locate button/ indicator	White	The Locate indicator is a white button with an associated indicator that blinks (4 Hz) when initiated (remotely) from the ILOM web interface. The Locate indicator enables a system operator to easily locate an IB-QNEM in a system within a large data center. Once activated, this indicator times out after 30 minutes. When pressed, the Locate button lights the Locate indicator. The Locate indicator provides these indications: Fast blink – Identifies a specific IB-QNEM in the chassis. Lights when the LED is initiated from the web interface remotely or from a press of the Locate button locally. Off – IB-QNEM locator function has not been selected.
2	OK indicator	Green	Displays the different states of module initialization. Provides the following indications: Steady on – Lights steadily when IB-QNEM is operating normally. Slow blink – Blinks slowly when IB-QNEM is transitioning from one state to the next. Off – IB-QNEM has no power or one of the other LEDs is lit.
3	Service Action Required indicator	Amber	Provides the following indications: Steady on – Lights when there is a fault associated with the IB-QNEM. Off – The IB-QNEM has no fault condition.
4	Ready-to-Remove indicator	Blue	This indicator is not used on the IB-QNEM.
5	Ethernet ports		RJ-45 Ethernet passthrough ports that connect to the server blades through the midplane. Note - Both P0 and P1 Ethernet ports are available for the Sun Blade X6275 server module and only the P1 Ethernet port is available for the Sun Blade X6270 server module.
6	Physical Link indicator (InfiniBand)	Green	Illuminates when the port is electrically active, that is, when a driver is attached and a physical link to a remote switch (or, possibly an HCA) has been established. Each InfiniBand port has one LED indicator.
7	CXP connectors		CXP connectors combine three 4x IB links per connector.
8	Ejector levers		To remove the IB-QNEM, open the ejector levers.

---

1.4 IB-QNEM Port Mapping

The IB-QNEM has two 36-port InfiniBand switches, I4A and I4B. Each port is capable of negotiating with a peer node up to 4x QDR speed. Each switch I4 device has 15 ports connected to 5 CXP external IB connectors. I4A has 12 ports connected to the two 4x ports from iTRAC 0 through 5 and I4B has 12 ports connected from iTRAC 6 through 11. The remaining nine ports are intralink connections between the two I4 devices.

---

1.5 Specifications

TABLE 1-4 and TABLE 1-5 list the specifications for the IB-QNEM in the Sun Blade 6048 Series Modular System.

TABLE 1-4   Operating Environment

Voltage	12V
Maximum power:	240 watts
Temperature	15° to 32° C (ASHRAE Environmental Class1)
Altitude:	3,000m (derate 1 degree C/300m above 900m)
Humidity:	20 - 80%

TABLE 1-5   Physical Characteristics

Height	155mm (6.1 in.)
Width	509mm (20 in.)
Depth	108.6mm (4.28 in.)
Weight	5.73 kg (12.7 lbs)

---

1.6 Regulatory Compliance

The regulatory compliance certifications of the Sun Blade 6048 Modular System cover the Sun Blade 6048 InfiniBand QDR Switched Network Express Module as an installed component.

To comply with legal EMI emissions regulations, the Sun Blade 6048 Modular Server must be provided with either the standard door or cooling rear door assembly when used with the Sun Blade 6048 InfiniBand QDR Switched Network Express Module. The rear door should remain closed during normal operation and opened only during product servicing or when power is removed from the product.

---

1.7 Commonly Used Terms

TABLE 1-6 identifies some terms commonly used in this guide.

TABLE 1-6   Common Terms

Term	Definition
Chassis	Sun Blade 6048 modular system hardware. For additional information about the Sun Blade 6048 Modular System, go to: http://docs.sun.com/app/docs/coll/blade6048
CMM	Chassis Monitoring Module (CMM) hardware
IB	InfiniBand
I4	Mellanox Infiniscale IV (I4) 36-port IB switch. There are two I4s switches on the IB-QMEN, I4A and I4B.
ILOM	Sun Integrated Lights Out Manager (ILOM) is the embedded management software on the server module SP and CCM SP that enables you to manage your system. For additional information about ILOM, refer to the Sun Integrated Lights Out Manager 2.0 User’s Guide.
IB-QNEM	Sun Blade 6048 InfiniBand QDR Switched Network Express Module
Server module	Sun Blade X6xxx server module hardware
SP	Embedded Service Processor (SP) on server module and Chassis Monitoring Module (CMM)
SDR	Single data rate
DDR	Double data rate
QDR	Quad data rate