JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun Network QDR InfiniBand Gateway Switch HTML Document Collection for Firmware Version 2.1
search filter icon
search icon

Document Information

Using This Documentation

Product Notes

Related Documentation

Feedback

Access to Oracle Support

Installation

Understanding the Gateway

Gateway Overview

Physical Specifications

Environmental Requirements

Acoustic Noise Emissions

Electrical Specifications

NET MGT Connector and Pins

USB Management Connector and Pins

Data QSFP Connector and Pins

Understanding Cabling

Routing Service Cables

Understanding Data Cabling

Preparing to Install the Gateway

Installation Preparation

Suggested Tools for Installation

Antistatic Precautions for Installation

Installation Responsibilities

Installation Sequence

Verify Shipping Carton Contents

Assemble the Optical Fiber Data Cables

Route the Data Cables

Installing the Gateway

Install the Gateway in the Rack

Powering On the Gateway

Connecting Data cables

Verifying the InfiniBand Fabric

Administration

Troubleshooting the Gateway

Gateway Hardware Problems

InfiniBand Fabric Problems

Network Management Troubleshooting Guidelines

Understanding Routing Through the Gateway

Switch GUIDs Overview

Understanding Administrative Commands

Configuration Overview

Oracle ILOM Command Overview

Hardware Command Overview

InfiniBand Command Overview

Administering the Chassis

Monitoring the Chassis

Controlling the Chassis

Administering the I4 Switch Chip

Monitoring the I4 Switch Chip

Controlling the I4 Switch Chip

Administering the Subnet Manager

Monitoring the Subnet Manager

Controlling the Subnet Manager

Configuring Secure Fabric Management

Administering the InfiniBand Fabric

Monitoring the InfiniBand Fabric

Controlling the InfiniBand Fabric

Administering Gateway Resources

Installing Gateway Supportive Software (Linux)

Creating VNICs Under Gateway Manual Mode (Linux)

Creating VNICs Under Host Manual Mode (Linux)

Creating Virtual IO Adapters (Oracle Solaris)

Monitoring Gateway Resources

Controlling LAGs

Controlling VLANs and VNICs

Controlling Gateway Ports and Parameters

Remote Management

Understanding Oracle ILOM on the Gateway

Oracle ILOM Overview

Supported Features

Understanding Oracle ILOM Targets

Administering Oracle ILOM (CLI)

CLI Overview

Accessing Oracle ILOM From the CLI

Switching Between the Oracle ILOM Shell and the Linux Shell

Monitoring Oracle ILOM Targets (CLI)

Controlling Oracle ILOM Targets (CLI)

Upgrading the Gateway Firmware Through Oracle ILOM (CLI)

Administering Oracle ILOM (Web)

Web Interface Overview

Access Oracle ILOM From the Web Interface

Monitoring Oracle ILOM Targets (Web)

Controlling Oracle ILOM Targets (Web)

Upgrade the Gateway Firmware (Web)

Using the Fabric Monitor

Access the Fabric Monitor

Fabric Monitor Features

Accessing the Rear Panel Diagram

Accessing Status Pane Information

Control Panel Function

Monitoring Parameters and Status

Administering Oracle ILOM (SNMP)

SNMP Overview

Understanding SNMP Commands

Monitoring Oracle ILOM Targets (SNMP)

Controlling Oracle ILOM Targets (SNMP)

Administering Hardware (IPMI)

ipmitool Overview

Display the Sensor States (IPMI)

Display the Sensor Information (IPMI)

Display the System Event Log (IPMI)

Display FRU ID Information (IPMI)

Display Gateway Status LED States (IPMI)

Enable the Locator LED (IPMI)

Disable the Locator LED (IPMI)

Understanding Oracle ILOM Commands

cd Command

create Command

delete Command

dump Command

exit Command (Oracle ILOM)

help Command (Oracle ILOM)

load Command

reset Command

set Command

show Command

version Command (Oracle ILOM)

Service

Detecting and Managing Faults

Interpreting Status LEDs

Managing Faulty Components

Identify Faults in the Oracle ILOM Event Log

Determining the Alarm State of a Component or System

Evaluating Sensor Alarms

Accessing CLI Prompts

Understanding Service Procedures

Replaceable Components

Suggested Tools for Service

Antistatic Precautions for Service

Servicing Power Supplies

Determine If a Power Supply Is Faulty

Inspecting a Power Supply

Power Off a Power Supply

Remove a Power Supply

Install a Power Supply

Power On a Power Supply

Servicing Fans

Determine If a Fan Is Faulty

Inspecting a Fan

Remove a Fan

Install a Fan

Servicing Data Cables

Inspecting the Data Cables

Remove a Data Cable

Install a Data Cable

Servicing the Battery

Determine If the Battery Is Faulty

Remove the Gateway From the Rack

Replace the Battery

Reference

Understanding Hardware Commands

Linux Shells for Hardware Commands

addlagport Command

allowhostconfig Command

autodisable Command

checkboot Command

checkpower Command

checktopomax Command

checkvoltages Command

connector Command

create_ipoib Command

createfabric Command

createlag Command

createvlan Command

createvnic Command

dcsport Command

delete_ipoib Command

deletelag Command

deletevlan Command

deletevnic Command

dellagport Command

disablecablelog Command

disablegwport Command

disablelagmode Command

disablelinklog Command

disablesm Command

disableswitchport Command

disablevnic Command

disallowhostconfig Command

enablecablelog Command

enablegwport Command

enablelagmode Command

enablelinklog Command

enablesm Command

enableswitchport Command

enablevnic Command

env_test Command

exit Command (Hardware)

fdconfig Command

fwverify Command

generatetopology Command

getfanspeed Command

getmaster Command

getportcounters Command

getportstatus Command

help Command (Hardware)

listlinkup Command

localmkeypersistence Command

matchtopology Command

modifyvnic Command

setcontrolledhandover Command

setdefaultgwdiscpkey Command

setgwethport Command

setgwinstance Command

setgwsl Command

setgwsystemname Command

sethostvniclimit Command

setmsmlocationmonitor Command

setsmmkey Command

setsmpriority Command

setsmrouting Command

setsubnetprefix Command

showdisk Command

showfree Command

showfruinfo Command

showgwconfig Command

showgwports Command

showioadapters Command

showlag Command

showpsufru Command

showsmlog Command

showtemps Command

showtopology Command

showunhealthy Command

showvlan Command

showvnics Command

smconfigtest Command

smnodes Command

smpartition Command

smsubnetprotection Command

version Command (Hardware)

Understanding InfiniBand Commands

Linux Shells for InfiniBand Commands

ibdiagnet Command

ibhosts Command

ibnetdiscover Command

ibnetstatus Command

ibnodes Command

ibportstate Command

ibroute Command

ibrouters Command

ibstat Command

ibswitches Command

ibtracert Command

perfquery Command

saquery Command

smpquery Command

Understanding SNMP MIB OIDs

OID Tables Overview

Understanding the SUN-DCS-IB-MIB MIB OIDs

Understanding the SUN-FABRIC-MIB MIB OIDs

Understanding the SUN-HW-TRAP-MIB MIB OIDs

Understanding the SUN-ILOM-CONTROL-MIB MIB OIDs

Understanding the SUN-PLATFORM-MIB MIB OIDs

Understanding the ENTITY-MIB MIB OIDs

Index

smsubnetprotection Command

Manages the secret M_Key.

Syntax

smsubnetprotection subcommand [-h]

This hardware command has subcommands that determine its functionality. This table describes the subcommands and provides their syntax.

Subcommand Syntax
Description
start [-force][-enable][-addonly|-deleteonly]
Initiates a new configuration based upon a currently used configuration. Use the -force option to ignore the partition daemon check.
list active|modified
Displays a list of active secret M_Keys, the current secret M_Key, and the enabled status, or displays a list of pending M_Keys and the M_Key to be assigned to current status.
listlocalmkey
Displays the current local M_Key for an I4 switch chip without a corresponding Subnet Manager and its status.
setlocalsecretmkey m_key
Sets the secret M_Key locally for an I4 switch chip without a corresponding Subnet Manager.
clearlocalmkey
Clears the local secret M_Key.
add m_key
Adds an M_Key to the configuration.
delete m_key
Deletes an M_Key from the configuration.
undo
Reverts the previous add, delete, or set-current operation.
set-current m_key
Sets the current M_Key.
commit [-force]
Commits the modified configuration to become the active configuration. Use the -force option to ignore the partition daemon check.
abort
Abruptly ends the configuration session. All modified configuration information is lost, and the active configuration remains unchanged.
setreplicationpassword password
Configures the replication (and encryption) password.
enablesecretmkey [-force]
Enables secret M_Key functionality. Use the -force option to ignore the partition daemon check.
disablesecretmkey [-force]
Disables secret M_Key functionality. Use the -force option to ignore the partition daemon check.

where:

Description

This hardware command manages the secret M_Key and its implementation. The secret M_Key is a passphrase used by trusted Subnet Managers to securely perform activities (enabling ports, setting parameters, and so on) on the I4 switch chips of the InfiniBand fabric. The secret M_Key is an encrypted M_Key that is implemented at a lower level in fabric management than the standard M_Key. Use the smsubnetprotection command and its subcommands to create and manage the list of secret M_Keys.

When configuring a list of secret M_Keys, you initiate the configuration session on the master Subnet Manager with the smsubnetprotection start command. During the session, you add or delete secret M_Keys to the configuration, set the current secret M_Key, and list the M_Keys configured.


Note - There is a maximum of 10 secret M_Keys for the configuration.


To end the session, you must use the smsubnetprotection commit command to make the configuration active. Once committed, the configuration is automatically distributed to all Subnet Managers in the InfiniBand fabric.


Note - You cannot both add and delete secret M_Keys within a single configuration session. You must perform these actions in separate configuration sessions.


Should a local secret M_Key be created for an I4 switch chip without a corresponding Subnet Manager, that secret M_Key is only recognized by that I4 switch chip, and is unrecognized by the other I4 switch chips in the InfiniBand fabric.

Because of the complexity of the secret M_Key functionality, this table describes the impact of certain scenarios and actions you can take.

Scenario
Impact and Actions
Setting up secret M_Key in a mixed firmware fabric.
If the master Subnet Manager has firmware 2.1, only other Subnet Managers with firmware 2.1 can administrate the fabric. For Subnet Managers with firmware 2.0 or lower, the fabric “disappears”.

If the master Subnet Manager has firmware 2.0 or lower, you can only set up local secret M_Keys for the I4 switch chips on their respective Subnet Managers with firmware 2.1.

Both situations are unsupported and not recommended.

Downgrading firmware after secret M_Key has been enabled.
If the master Subnet Manager is downgraded to firmware 2.0 or lower and there is a standby Subnet Manager with firmware 2.1, the secret M_Key is maintained through the standby Subnet Manager during the master Subnet Manager's reboot. After the reboot, the situation becomes as described above.

If you downgrade any other Subnet Manager to firmware 2.0 or lower, the situation becomes as described above.

Before you downgrade any firmware, disable secret M_Key.


Note - Standard M_Key is not affected by a downgrade from firmware 2.1 to 2.0.


Upgrading from a lower firmware version.
Do not enable secret M_Key until all Subnet Managers in the fabric are at firmware version 2.1 or higher.
Introducing a new Subnet Manager with firmware 2.1 or higher, yet no secret M_Key policy, into a secret M_Key fabric.
If the new Subnet Manager has a low priority, it is updated with the new M_Keys and functionality automatically.

If the new Subnet Manager has the highest priority, manually apply the secret M_Key policy and it will become the master Subnet Manager.

Alternatively, preconfigure the new master Subnet Manager with a low priority. Allow it to be automatically updated. Then set the priority to master.

Before introducing the new Subnet Manager:

  • Update the smnodes list with the smnodes command.

  • Update the fabric configuration with the fdconfig command.

  • Update the fabric mapping with the createfabric command.

M_Key values are mismatched.
If you add a Subnet Manager with one set of M_Keys to a fabric with a different set of M_Keys, the added Subnet Manager is not recognized.

The situation is the same as above, however manually apply the correct secret M_Key policy using the setlocalsecretmkey subcommand.

Merging two or more subnets into one fabric.
If each subnet is configured with different secret M_Key policies, then the subnets will not “see” each other and will act independently.

If each subnet is configured with identical secret M_Key policies, there might be master Subnet Manager contention.

Before physically merging:

  • Update the smnode files for all smnodes of both subnets with the smnodes command.

  • Configure both subnets with the new partition information with the smpartition command.

  • Configure both subnets with the new fabric configuration with the fdconfig command.

  • Correlate both subnets to the new fabric mapping with the createfabric command. You might need to use the -force option.

  • Configure identical secret M_Keys for both subnets.

  • Set the priority of one master Subnet Manager lower than the other.

This table describes each of the columns of the output of the smsubnetprotection command.

Column Heading
Description
Mkey
M_Keys that are assigned to Subnet Managers, but are not secure.
Untrusted Mkey
M_Keys that are not configured for all Subnet Managers, and hence untrusted.
Smkey
Secret M_Keys that are secure and trusted.
Attribute
The attribute of the M_Key:.
  • C – The current secret M_Key.

  • S – The standby secret M_Key about to become current.

The smsubnetprotection command is available from the /SYS/Fabric_Mgmt Linux shell target of the Oracle ILOM CLI interface.

Options

This table describes the options to the smsubnetprotection command and their purposes.

Option
Purpose
-force
Specifies the action to ignore the partition daemon check.
-enable
Specifies that after committing the configuration session, the secret M_Key functionality is automatically enabled.
-addonly
Specifies that the session is only to add secret M_Keys to the configuration.
-deleteonly
Specifies that the session is only to delete secret M_Keys from the configuration.
Example

This example shows how to display the active secret M_Keys with the smsubnetprotection command.

FabMan@gateway_name->smsubnetprotection list active
# File_format_version_number 1
# Sun DCS IB mkey config file
# This file is generated, do not edit
# secretmkey=enabled
# nodeid=o4nm2-gw-6
# time=15 Sep 03:54:46
# checksum=378d9b09744e1d8b8ba6ae868c99d0c9
#! commit_number : 3
Mkey                    Untrusted Mkey          Smkey                   Attribute 
------------------      ------------------      ------------------      --------- 
0x00abcdefabcdef01      0x1aa45124fee612ae      0x15fc26aea300f831
0x00abcdefabcdef02      0x4ccd8230de6cd348      0x3fc7e6ad701a8a2a
0x00abcdefabcdef03      0x9baa1debcc74de5e      0x1b253003600d137b      C
FabMan@gateway_name->
Related Information