Overview

C H A P T E R 1

Overview

This document provides detailed procedures for installing the hardware and software for the Dual Port 4x DDR IB PCIe Host Channel Adapter card. The cards described in this guide have the following main features:

IBTA (InfiniBand Trade Association) v1.2 compliant

Two 4x InfiniBand copper ports for connecting InfiniBand traffic (4x IB connectors)

PCI Express expansion board with an x8 edge connector compatible to the PCI Express 1.0a specification

‘Media detect circuit’ supporting external InfiniBand fiber solutions

European Union Restriction of Hazardous Substances (RoHS) compliant

The boards differ in

4x IB port speed support: 10 Gb/s (SDR-Single Data Rate) or 20 Gb/s (DDR-Double Data Rate)

Bracket height: short or tall

Note - The PCI Express (or PCIe) interface is sometimes called PCI-E.

IB-HCA Card

FIGURE 1-1 Dual Port 4x DDR IB PCIe Host Channel Adapter With Tall Bracket

Figure shows the ExpressModule with tall bracket

I/O Interfaces

Each IB-HCA card includes the following interfaces:

Two 4X InfiniBand copper connectors

PCI Express x8 edge connector

I/O panel LEDs

InfiniBand Interface

The Dual Port 4x DDR IB PCIe Host Channel Adapter device is compliant with the InfiniBand Architecture Specification, Release 1.2. It has two compliant 4X InfiniBand ports, A and B. The IB-HCA card provides access to these ports by means of two 4X InfiniBand connectors for external InfiniBand copper cables also compliant with the InfiniBand Architecture Specification, Release 1.2. Connector 1 connects to port A of the device, while connector 2 connects to port B.

Furthermore, the IB-HCA card is embedded with a ‘media detect circuit’ that supports external InfiniBand fiber solutions. These are connected to the InfiniBand port connectors using active media converters, such as Emcore’s QTR3400 Smart Module or Fujitsu’s FPD-010R008-0E o-microGiGaCN.

PCI Express Interface

The Dual Port 4x DDR IB PCIe Host Channel Adapter device has eight Tx/Rx pairs of SerDes providing for a PCI Express x8 interface, version 1.0a compatible. The device can be either a master initiating the PCI Express bus operations or a slave responding to PCI bus operations. The PCI Express bus can connect to either a host CPU in an IB-HCA application or to an I/O device (such as Gigabit Ethernet) when used as a Target Channel Adapter.

LED Assignment

The board has four LEDs located on the I/O panel - 2 LEDs per 4X port. See TABLE 1-1.

TABLE 1-1 LEDs
Port Number	LED Name
Port 1	Physical Link - Green
	Data Activity - Amber
Port 2	Physical Link - Green
	Data Activity - Amber

The physical link (green) illuminates once VAPI (InfiniBand Verbs API) is started and a physical connection is made between two nodes. The data activity link (amber) illuminates once the InfiniBand network is discovered over the physical link. The activity link is a steady yellow when it is discovered but no data is being passed. The activity link blinks when data is being passed. If the LEDs are not active, either the physical or the logical (or both) connections have not been established. See FIGURE 1-2.

Table describes data ports and LEDs of the ExpressModule, Port 1 physical link - green LED, data activity - amber LED. Same for port 2 .

Short brackets have the same port and LED footprints as tall brackets. See Appendix B.

FIGURE 1-2 I/O Panel With Dual Ports and LEDs (Tall Bracket)

Figure shows data ports and LEDs of the ExpressModule

Figure Legend Ports and LEDs of the ExpressModule
1	InfiniBand Port 1
2	Physical Link (Green LED)
3	Data Activity Link (Amber LED)
4	InfiniBand Port 2
5	Data Activity Link (Amber LED)
6	Physical Link (Green LED)

I²C Compatible Interface

A three-pin header, designated with reference name J5 on the IB-HCA card, is provided as the I²C compatible interface.

Power

All adapter cards receive power from the PCI Express Edge connector. All other required power voltages are generated by on-board switch mode regulators. For power consumption see Appendix A.

Memory

The IB-HCA card support multiple memory devices through the PCI Express, Flash, and I²C-compatible interfaces.

System Memory

The IB-HCA card utilizes the PCI Express interface to store and access IB fabric connection information on the system memory.