Skip Navigation Links | |
Exit Print View | |
Sun Network QDR InfiniBand Gateway Switch HTML Document Collection for Firmware Version 2.1 |
USB Management Connector and Pins
Preparing to Install the Gateway
Suggested Tools for Installation
Antistatic Precautions for Installation
Verify Shipping Carton Contents
Assemble the Optical Fiber Data Cables
Install the Gateway in the Rack
Verifying the InfiniBand Fabric
Network Management Troubleshooting Guidelines
Understanding Routing Through the Gateway
Understanding Administrative Commands
Administering the I4 Switch Chip
Controlling the I4 Switch Chip
Administering the Subnet Manager
Controlling the Subnet Manager
Configuring Secure Fabric Management
Administering the InfiniBand Fabric
Monitoring the InfiniBand Fabric
Controlling the InfiniBand Fabric
Administering Gateway Resources
Installing Gateway Supportive Software (Linux)
Acquire the BXOFED Software (Linux)
Install the BXOFED Software (Linux)
Acquire the ConnectX-2 Firmware
Creating VNICs Under Gateway Manual Mode (Linux)
Gateway Manual Mode Overview (Linux)
Determine VNIC Configuration Parameters For Gateway Manual Mode (Linux)
Determine VLAN Associations for Gateway Manual Mode (Linux)
Configure and Create VNICs for Gateway Manual Mode (Linux)
VNIC Configuration on Linux Hosts in Gateway Manual Mode
Creating VNICs Under Host Manual Mode (Linux)
Host Manual Mode Overview (Linux)
Central Configuration File (Linux)
VNIC-Specific Configuration File (Linux)
Determine VNIC Configuration Parameters for Host Manual Mode (Linux)
Create the Central Configuration File (Linux)
Create the VNIC-Specific Configuration Files (Linux)
mlx4_vnic_confd Daemon (Linux)
Configure and Create VNICs for Host Manual Mode (Linux)
Creating Virtual IO Adapters (Oracle Solaris)
Ethernet Over InfiniBand Overview (Oracle Solaris)
Verify That the Ethernet Over InfiniBand Driver Is Installed (Oracle Solaris)
Install the Ethernet Over InfiniBand Driver (Oracle Solaris)
Configure IP Addresses on the Data Link (Oracle Solaris)
Create Oracle Solaris VNICs With VLAN Tagging (Oracle Solaris)
Display Gateway Ethernet Port Information
Display Gateway Port Information
Display the Gateway Instance Number
Display Information About the Gateway
Add or Delete Connectors From a LAG
VNIC Allocation on the Gateway
Controlling Gateway Ports and Parameters
Enable a Gateway Ethernet Port
Disable a Gateway Ethernet Port
Set the Gateway Ethernet Port Parameters
Set the Gateway Instance Number
Understanding Oracle ILOM on the Gateway
Understanding Oracle ILOM Targets
Administering Oracle ILOM (CLI)
Accessing Oracle ILOM From the CLI
Switching Between the Oracle ILOM Shell and the Linux Shell
Monitoring Oracle ILOM Targets (CLI)
Controlling Oracle ILOM Targets (CLI)
Upgrading the Gateway Firmware Through Oracle ILOM (CLI)
Administering Oracle ILOM (Web)
Access Oracle ILOM From the Web Interface
Monitoring Oracle ILOM Targets (Web)
Controlling Oracle ILOM Targets (Web)
Upgrade the Gateway Firmware (Web)
Accessing the Rear Panel Diagram
Accessing Status Pane Information
Monitoring Parameters and Status
Administering Oracle ILOM (SNMP)
Monitoring Oracle ILOM Targets (SNMP)
Controlling Oracle ILOM Targets (SNMP)
Display the Sensor States (IPMI)
Display the Sensor Information (IPMI)
Display the System Event Log (IPMI)
Display FRU ID Information (IPMI)
Display Gateway Status LED States (IPMI)
Disable the Locator LED (IPMI)
Understanding Oracle ILOM Commands
Identify Faults in the Oracle ILOM Event Log
Determining the Alarm State of a Component or System
Understanding Service Procedures
Antistatic Precautions for Service
Determine If a Power Supply Is Faulty
Determine If the Battery Is Faulty
Remove the Gateway From the Rack
Understanding Hardware Commands
Linux Shells for Hardware Commands
Understanding InfiniBand Commands
Linux Shells for InfiniBand Commands
Understanding the SUN-DCS-IB-MIB MIB OIDs
Understanding the SUN-FABRIC-MIB MIB OIDs
Understanding the SUN-HW-TRAP-MIB MIB OIDs
Understanding the SUN-ILOM-CONTROL-MIB MIB OIDs
Understanding the SUN-PLATFORM-MIB MIB OIDs
Note - If the VIOA is to be assigned to a VLAN, you cannot create the VIOA from the gateway. Instead, use the procedure in Create Oracle Solaris VNICs With VLAN Tagging (Oracle Solaris).
For this procedure, the Oracle Solaris 11 host name is solaris01.
# dladm show-ib LINK HCAGUID PORTGUID PORT STATE PKEYS ibp0 21280001A0A590 21280001A0A591 1 up FFFF ibp1 21280001A0A590 21280001A0A592 2 up FFFF #
In this example, the HCA GUID is 21280001A0A590 and the port GUIDs are 21280001A0A591 and 21280001A0A592.
FabMan@gateway_name->ibhosts Ca : 0x0021280001A0A590 ports 2 "solaris01" Ca : 0x00212856cd22c040 ports 2 "SUN IB QDR GW switch mnm34-97 Bridge 1" Ca : 0x0002c903000891aa ports 2 "mnm34-54 HCA-1" Ca : 0x00212800013ece9e ports 2 "mnm34-55 HCA-1" Ca : 0x0003ba000100e370 ports 2 "mnm34-60 HCA-1" . . . FabMan@gateway_name->
The port GUIDs are the HCA GUID +1 (port 1) and HCA GUID +2 (port 2), respectively.
If not, check the cabling between the gateway and the Oracle Solaris 11 host.
FabMan@gateway_name->showvnics ID STATE FLG IOA_GUID NODE IID MAC VLN PKEY GW --- -------- --- ----------------------- ---------- ---- ----------------- --- ---- -------- 4 UP N 00:03:BA:00:01:00:E3:71 mnm34-60 0000 02:02:02:02:02:03 NO ffff 0A-ETH-1 5 UP N 00:03:BA:00:01:00:E3:71 mnm34-60 0002 02:02:02:02:02:04 NO ffff 0A-ETH-1 3 DISABLED N 00:03:BA:00:01:00:E3:71 mnm34-60 0000 02:02:02:02:02:02 NO ffff 0A-ETH-1 6 UP N 00:03:BA:00:01:00:E3:72 mnm34-60 0000 02:02:02:02:02:05 NO ffff 0A-ETH-1 FabMan@gateway_name->
Note - Until the IP interface is created on the data link (using the ipadm command) at the Solaris host, the STATE of the VNIC is WAIT-IOA.
STATE – The state of the VIOA.
IOA_GUID – The port GUID of the VIOA. The GUID might be listed several times.
NODE – The host name of the node hosting the VIOA.
MAC – The MAC address assigned to the VIOA. There might be several MAC addresses.
PKEY – The partition key assigned.
GW – The physical connector cabled to the node or host.
If no port GUID of the Oracle Solaris 11 host is listed, or there are no MAC addresses assigned to the port GUID, go to Step 10.
Otherwise, go to Configure IP Addresses on the Data Link (Oracle Solaris).
Note - Each MAC address must be unique and not 00:00:00:00:00:00.
Note - Only even numbers are supported for the most significant byte of the MAC address (unicast).
FabMan@gateway_name->createvnic connector -guid guid -mac mac_address -pkey default
where:
connector is the name of the connector (0A-ETH-1 to 0A-ETH-4 and 1A-ETH-1 to 1A-ETH-4).
guid is the global unique identifier of the target port on the host associated with the VIOA.
mac_address is the MAC address to be assigned to the VIOA.
For example, to assign the MAC address of a0:a5:91:95:30:9a to the VIOA for port GUID 21280001A0A591 through connector 0A-ETH-2:
FabMan@gateway_name->createvnic 0A-ETH-2 -guid 0021280001A0A591 -mac a0:a5:91:95:30:9a -pkey default VNIC created FabMan@gateway_name->
The VIOA is created and an Ethernet over InfiniBand data link is bound to the VIOA.
FabMan@gateway_name->showvnics ID STATE FLG IOA_GUID NODE IID MAC VLN PKEY GW --- -------- --- ----------------------- ---------- ---- ----------------- --- ---- -------- 7 WAIT-IOA N 00:21:28:00:01:A0:A5:91 solaris01 0000 a0:a5:91:95:30:9a NO ffff 0A-ETH-2 4 UP N 00:03:BA:00:01:00:E3:71 mnm34-60 0000 02:02:02:02:02:03 NO ffff 0A-ETH-1 5 UP N 00:03:BA:00:01:00:E3:71 mnm34-60 0002 02:02:02:02:02:04 NO ffff 0A-ETH-1 3 DISABLED N 00:03:BA:00:01:00:E3:71 mnm34-60 0000 02:02:02:02:02:02 NO ffff 0A-ETH-1 6 UP N 00:03:BA:00:01:00:E3:72 mnm34-60 0000 02:02:02:02:02:05 NO ffff 0A-ETH-1 FabMan@gateway_name->
Note - Until the IP interface is created on the data link (using the ipadm command) at the Solaris host, the STATE of the VNIC is WAIT-IOA.
See Configure IP Addresses on the Data Link (Oracle Solaris).
dladm command man page
Gateway Reference, ibhosts command
Gateway Reference, showvnics command
Gateway Reference, createvnics command
Verify That the Ethernet Over InfiniBand Driver Is Installed (Oracle Solaris)
Install the Ethernet Over InfiniBand Driver (Oracle Solaris)
Create Oracle Solaris VNICs With VLAN Tagging (Oracle Solaris)