Setting a Spine Switch as the Subnet Manager Master
The InfiniBand switch located in rack unit 1 (U1) is the spine switch. Recovery Appliance has a spine switch only when it is connected to another Recovery Appliance. It is not included as a basic component of the rack.
Perform these steps after the racks are cabled together
The spine switch is the Subnet Manager Master for the InfiniBand subnet. The Subnet Manager Master has priority 8.
To verify the priority setting of the spine switch:
-
Log in to the spine switch as the
root
user. -
Run the
setsmpriority list
command.The command should show that
smpriority
has a value of 8. Ifsmpriority
has a different value, then do the following:-
Use the
disablesm
command to stop the Subnet Manager. -
Use the
setsmpriority 8
command to set the priority to 8. -
Use the
enablesm
command to restart the Subnet Manager.
-
The other two InfiniBand switches are the leaf switches. The leaf switches are located in rack units 20 and 22 (U20 and U22). They are the Standby Subnet Managers with a priority of 5. You can verify the status using the preceding procedure, substituting a value of 5 in the command shown in step 22.b.
To determine the Subnet Manager Master:
-
Log in as the
root
user on any InfiniBand switch. -
Display the location of the Subnet Manager Master.
# getmaster 20100701 11:46:38 OpenSM Master on Switch : 0x0021283a8516a0a0 ports 36 Sun DCS 36 QDR switch ra01sw-ib1.example.com enhanced port 0 lid 1 lmc 0
The preceding output shows the proper configuration. The Subnet Master Manager is running on spine switch ra01sw-ib1.example.com.
If the spine switch is not the Subnet Manager Master, then reset the Subnet Manager Master:
-
Use the
getmaster
command to identify the current location of the Subnet Manager Master. -
Log in as the
root
user on the leaf switch that is the Subnet Manager Master. -
Disable Subnet Manager on the switch. The Subnet Manager Master relocates to another switch.
See Also:
"Disable the Subnet Manager" in Sun Datacenter InfiniBand Switch 36 User's Guide at
http://docs.oracle.com/cd/E19197-01/835-0784-05/z4001de61813698.html#z40003f12047367
-
Use the
getmaster
command to identify the current location of the Subnet Manager Master. If the spine switch is not Subnet Manager Master, then repeat steps 2 and 3 until the spine switch is the Subnet Manager Master. -
Enable Subnet Manager on the leaf switches that were disabled during this procedure.
See Also:
"Enable the Subnet Manager" in Sun Datacenter InfiniBand Switch 36 User's Guide at
http://docs.oracle.com/cd/E19197-01/835-0784-05/z4001de61707660.html#z40003f12047359
Note:
If the InfiniBand network consists of four or more racks cabled together, then only the spine switches run Subnet Manager. Disable the Subnet Manager on the leaf switches.