JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun Network QDR InfiniBand Gateway Switch HTML Document Collection for Firmware Version 2.1
search filter icon
search icon

Document Information

Using This Documentation

Product Notes

Related Documentation

Feedback

Access to Oracle Support

Installation

Understanding the Gateway

Gateway Overview

Physical Specifications

Environmental Requirements

Acoustic Noise Emissions

Electrical Specifications

NET MGT Connector and Pins

USB Management Connector and Pins

Data QSFP Connector and Pins

Understanding Cabling

Routing Service Cables

Understanding Data Cabling

Preparing to Install the Gateway

Installation Preparation

Suggested Tools for Installation

Antistatic Precautions for Installation

Installation Responsibilities

Installation Sequence

Verify Shipping Carton Contents

Assemble the Optical Fiber Data Cables

Route the Data Cables

Installing the Gateway

Install the Gateway in the Rack

Powering On the Gateway

Connecting Data cables

Verifying the InfiniBand Fabric

Administration

Troubleshooting the Gateway

Gateway Hardware Problems

InfiniBand Fabric Problems

Network Management Troubleshooting Guidelines

Understanding Routing Through the Gateway

Switch GUIDs Overview

Understanding Administrative Commands

Configuration Overview

Oracle ILOM Command Overview

Hardware Command Overview

InfiniBand Command Overview

Administering the Chassis

Monitoring the Chassis

Controlling the Chassis

Administering the I4 Switch Chip

Monitoring the I4 Switch Chip

Controlling the I4 Switch Chip

Administering the Subnet Manager

Monitoring the Subnet Manager

Controlling the Subnet Manager

Configuring Secure Fabric Management

Administering the InfiniBand Fabric

Monitoring the InfiniBand Fabric

Controlling the InfiniBand Fabric

Administering Gateway Resources

Installing Gateway Supportive Software (Linux)

Acquire the BXOFED Software (Linux)

Install the BXOFED Software (Linux)

Acquire the ConnectX-2 Firmware

Creating VNICs Under Gateway Manual Mode (Linux)

Gateway Manual Mode Overview (Linux)

Determine VNIC Configuration Parameters For Gateway Manual Mode (Linux)

Determine VLAN Associations for Gateway Manual Mode (Linux)

Configure and Create VNICs for Gateway Manual Mode (Linux)

VNIC Configuration on Linux Hosts in Gateway Manual Mode

Creating VNICs Under Host Manual Mode (Linux)

Host Manual Mode Overview (Linux)

Central Configuration File (Linux)

VNIC-Specific Configuration File (Linux)

Determine VNIC Configuration Parameters for Host Manual Mode (Linux)

Create the Central Configuration File (Linux)

Create the VNIC-Specific Configuration Files (Linux)

mlx4_vnic_confd Daemon (Linux)

Configure and Create VNICs for Host Manual Mode (Linux)

Creating Virtual IO Adapters (Oracle Solaris)

Ethernet Over InfiniBand Overview (Oracle Solaris)

Verify That the Ethernet Over InfiniBand Driver Is Installed (Oracle Solaris)

Install the Ethernet Over InfiniBand Driver (Oracle Solaris)

Set Up VIOAs (Oracle Solaris)

Configure IP Addresses on the Data Link (Oracle Solaris)

Create Oracle Solaris VNICs With VLAN Tagging (Oracle Solaris)

Monitoring Gateway Resources

Display LAG Information

Display the VLANs

Display the VNICs

Display IO Adapters

Display Gateway Ethernet Port Information

Display Gateway Port Information

Display the Gateway Instance Number

Display Information About the Gateway

Controlling LAGs

Enable LAG Mode

Create LAGs

Create a Global Gateway

Add or Delete Connectors From a LAG

Delete a LAG

Disable LAG Mode

Controlling VLANs and VNICs

VNIC Allocation on the Gateway

Create VLANs

Delete VLANs

Create VNICs

Modify VNICs

Enable VNICs

Disable VNICs

Delete VNICs

Controlling Gateway Ports and Parameters

Enable a Gateway Ethernet Port

Disable a Gateway Ethernet Port

Set the Gateway Ethernet Port Parameters

Set the Gateway Service Level

Set the Gateway Instance Number

Set the Gateway System Name

Remote Management

Understanding Oracle ILOM on the Gateway

Oracle ILOM Overview

Supported Features

Understanding Oracle ILOM Targets

Administering Oracle ILOM (CLI)

CLI Overview

Accessing Oracle ILOM From the CLI

Switching Between the Oracle ILOM Shell and the Linux Shell

Monitoring Oracle ILOM Targets (CLI)

Controlling Oracle ILOM Targets (CLI)

Upgrading the Gateway Firmware Through Oracle ILOM (CLI)

Administering Oracle ILOM (Web)

Web Interface Overview

Access Oracle ILOM From the Web Interface

Monitoring Oracle ILOM Targets (Web)

Controlling Oracle ILOM Targets (Web)

Upgrade the Gateway Firmware (Web)

Using the Fabric Monitor

Access the Fabric Monitor

Fabric Monitor Features

Accessing the Rear Panel Diagram

Accessing Status Pane Information

Control Panel Function

Monitoring Parameters and Status

Administering Oracle ILOM (SNMP)

SNMP Overview

Understanding SNMP Commands

Monitoring Oracle ILOM Targets (SNMP)

Controlling Oracle ILOM Targets (SNMP)

Administering Hardware (IPMI)

ipmitool Overview

Display the Sensor States (IPMI)

Display the Sensor Information (IPMI)

Display the System Event Log (IPMI)

Display FRU ID Information (IPMI)

Display Gateway Status LED States (IPMI)

Enable the Locator LED (IPMI)

Disable the Locator LED (IPMI)

Understanding Oracle ILOM Commands

cd Command

create Command

delete Command

dump Command

exit Command (Oracle ILOM)

help Command (Oracle ILOM)

load Command

reset Command

set Command

show Command

version Command (Oracle ILOM)

Service

Detecting and Managing Faults

Interpreting Status LEDs

Managing Faulty Components

Identify Faults in the Oracle ILOM Event Log

Determining the Alarm State of a Component or System

Evaluating Sensor Alarms

Accessing CLI Prompts

Understanding Service Procedures

Replaceable Components

Suggested Tools for Service

Antistatic Precautions for Service

Servicing Power Supplies

Determine If a Power Supply Is Faulty

Inspecting a Power Supply

Power Off a Power Supply

Remove a Power Supply

Install a Power Supply

Power On a Power Supply

Servicing Fans

Determine If a Fan Is Faulty

Inspecting a Fan

Remove a Fan

Install a Fan

Servicing Data Cables

Inspecting the Data Cables

Remove a Data Cable

Install a Data Cable

Servicing the Battery

Determine If the Battery Is Faulty

Remove the Gateway From the Rack

Replace the Battery

Reference

Understanding Hardware Commands

Linux Shells for Hardware Commands

addlagport Command

allowhostconfig Command

autodisable Command

checkboot Command

checkpower Command

checktopomax Command

checkvoltages Command

connector Command

create_ipoib Command

createfabric Command

createlag Command

createvlan Command

createvnic Command

dcsport Command

delete_ipoib Command

deletelag Command

deletevlan Command

deletevnic Command

dellagport Command

disablecablelog Command

disablegwport Command

disablelagmode Command

disablelinklog Command

disablesm Command

disableswitchport Command

disablevnic Command

disallowhostconfig Command

enablecablelog Command

enablegwport Command

enablelagmode Command

enablelinklog Command

enablesm Command

enableswitchport Command

enablevnic Command

env_test Command

exit Command (Hardware)

fdconfig Command

fwverify Command

generatetopology Command

getfanspeed Command

getmaster Command

getportcounters Command

getportstatus Command

help Command (Hardware)

listlinkup Command

localmkeypersistence Command

matchtopology Command

modifyvnic Command

setcontrolledhandover Command

setdefaultgwdiscpkey Command

setgwethport Command

setgwinstance Command

setgwsl Command

setgwsystemname Command

sethostvniclimit Command

setmsmlocationmonitor Command

setsmmkey Command

setsmpriority Command

setsmrouting Command

setsubnetprefix Command

showdisk Command

showfree Command

showfruinfo Command

showgwconfig Command

showgwports Command

showioadapters Command

showlag Command

showpsufru Command

showsmlog Command

showtemps Command

showtopology Command

showunhealthy Command

showvlan Command

showvnics Command

smconfigtest Command

smnodes Command

smpartition Command

smsubnetprotection Command

version Command (Hardware)

Understanding InfiniBand Commands

Linux Shells for InfiniBand Commands

ibdiagnet Command

ibhosts Command

ibnetdiscover Command

ibnetstatus Command

ibnodes Command

ibportstate Command

ibroute Command

ibrouters Command

ibstat Command

ibswitches Command

ibtracert Command

perfquery Command

saquery Command

smpquery Command

Understanding SNMP MIB OIDs

OID Tables Overview

Understanding the SUN-DCS-IB-MIB MIB OIDs

Understanding the SUN-FABRIC-MIB MIB OIDs

Understanding the SUN-HW-TRAP-MIB MIB OIDs

Understanding the SUN-ILOM-CONTROL-MIB MIB OIDs

Understanding the SUN-PLATFORM-MIB MIB OIDs

Understanding the ENTITY-MIB MIB OIDs

Index

Create Oracle Solaris VNICs With VLAN Tagging (Oracle Solaris)


Note - Follow this procedure to configure Ethernet over InfiniBand with VLAN tagging in Oracle Solaris. This procedure is independent of and unrelated to Host Manual Mode or Gateway Manual Mode. To create VIOAs on the gateway, see Set Up VIOAs (Oracle Solaris).


  1. Create the VIOAs to use as IP interfaces for the Solaris VNICs.

    See Set Up VIOAs (Oracle Solaris).

  2. On the management controller, create a VLAN to P_Key mapping for the VIOAs.
    FabMan@gateway_name->createvlan connector -vlan vlan_ID -pkey p_key

    where:

    • connector is the name of the connector (0A-ETH-1 to 0A-ETH-4, 0A-ETH, 1A-ETH-1 to 1A-ETH-4, and 1A-ETH).

    • vlan_ID is the VLAN identifier (NO or 2 to 4094).


      Note - Do not use VLAN identifiers 0, 1, or 4095. According to the IEEE 802.1Q specification, VLAN ID 0 is used for priority tag, and VLAN ID 1 is usually reserved for a switch or bridge management VLAN.


    • p_key is the partition identifier (1 to 7fff or default).

    For example, to associate connector 1A-ETH-3 with VLAN 3 and default partition key:

    FabMan@gateway_name->createvlan 1A-ETH-3 -vlan 3 -pkey default
    FabMan@gateway_name->
  3. If you are associating VLANs with the Solaris VNICs, perform these steps.
    1. Set the maximum number of VNICs per host.
      FabMan@gateway_name->sethostvniclimit connector -host hostname -port port -limit number -vlanlist vlan_ID1 ... vlan_IDn

      where:

      • connector is the name of the connector (0A-ETH-1 to 0A-ETH-4 and 1A-ETH-1 to 1A-ETH-4).

      • hostname is the host name of the management controller or system name recognized by the BridgeX manager.

      • port is the port of the host.

      • number is the maximum number of VNICs for the IOA.

      • vlan_ID is the VLAN identifier (NO or 2 to 4094).


        Note - Do not use VLAN identifiers 0, 1, or 4095. According to the IEEE 802.1Q specification, VLAN ID 0 is used for priority tag, and VLAN ID 1 is usually reserved for a switch or bridge management VLAN.


        For example, to limit the host cupcake to 4 VNICs:

        FabMan@gateway_name->sethostvniclimit 0A-ETH-1 -host cupcake -port 1 -limit 4 -vlanlist 3 
        FabMan@gateway_name->
    2. Repeat Step a for all hosts in the fabric that need to be regulated.
  4. On the Oracle Solaris 11 host, create a Oracle Solaris VNIC on the data link.
    # dladm create-vnic -l IP_interface -m mac_address -v vlan_ID name

    where:

    • IP_interface is the interface corresponding to the data link and VIOA.

    • mac_address is the MAC address to be assigned to the VIOA.

    • vlan_ID is the VLAN identifier (NO or 2 to 4094).


      Note - Do not use VLAN identifiers 0, 1, or 4095. According to the IEEE 802.1Q specification, VLAN ID 0 is used for priority tag, and VLAN ID 1 is usually reserved for a switch or bridge management VLAN.


    • name is the name of the VNIC. For ease of identification, the VNIC name can be formatted as eX_vnicY, where:

      • X is the instance number of the Ethernet over InfiniBand IP interface.

      • Y is the instance of the VNIC, independent of the interface instance.

    For example:

    # dladm create-vnic -l net6 -m 2:8:20:7c:5d:d3 -v 3 e0_vnic0
    #
  5. Verify the creation of the VNIC.
    # dladm show-vnic
    LINK        OVER    SPEED    MACADDRESS        MACADDRTYPE   VID
    e0_vnic0    net6    10000    2:8:20:7c:5d:d3   random        3
    #
  6. Display a list of all data links.
    # dladm show-link
    LINK                CLASS     MTU    STATE    OVER 
    net6                phys      1500   up       --
    e0_vnic0            vnic      1500   up       net6
     
  7. Configure the IP address of the data link e0_vnic0.

    See Configure IP Addresses on the Data Link (Oracle Solaris).

  8. On the management controller, verify that the VNICs are properly associated and up.
    FabMan@gateway_name->showvnics
    ID  STATE  FLG IOA_GUID          NODE       IID  MAC               VLN PKEY   GW
    --- -----  --- ----------------- ---------- ---- ----------------- --- ------ --------
     0 UP       N 0021280001CF37AC  solaris01  00c0 02:42:42:42:12:03 NO  0xffff 0A-ETH-1
     1 UP       H 0021280001CF37AC  solaris01  80c1 02:08:20:7C:5D:D3 3   0xffff 0A-ETH-1
    FabMan@gateway_name->
Related Information