You can configure or unconfigure an IB device on a system by using the cfgadm command. The command manages dynamic reconfiguration (DR) of the entire IB fabric as seen by a host. In addition, the command enables you to do the following:
Display the IB fabric
Manage communication services
Update P_key table databases
For more information, see the cfgadm_ib(8) man page.
All IB devices such as Port, VPPA, HCA_SVC, IOC, and pseudo devices support cfgadm operations.
To display IB device information, use the following command syntax:
# cfgadm -[a]l
Including the –a option displays additional information.
The cfgadm command displays information about attachment points, which are locations in the system where DR operations can occur. Note that all IB Ap_Ids are shown as connected.
For more information, see Attachment Points. For more information about attachment points supported by the cfgadm command, see the cfgadm_ib(8) man page.
An attachment point consists of a receptacle and an occupant. The following table describes the state of a device depending on the corresponding combined states of the receptacle, occupant, and condition.
|
The following sample output is truncated to show relevant IB information only.
# cfgadm -al Ap_Id Type Receptacle Occupant Condition ib IB-Fabric connected configured ok hca:21280001A0A478 IB-HCA connected configured ok ib::21280001A0A47A,0,ipib IB-PORT connected configured ok ib::21280001A0A479,0,ipib IB-PORT connected configured ok ib::1730000008070,0,hnfs IB-HCA_SVC connected configured ok ib::daplt,0 IB-PSEUDO connected configured ok ib::iser,0 IB-PSEUDO connected configured ok ib::rdsib,0 IB-PSEUDO connected configured ok ib::rdsv3,0 IB-PSEUDO connected configured ok ib::rpcib,0 IB-PSEUDO connected configured ok ib::sdpib,0 IB-PSEUDO connected configured ok ib::sol_umad,0 IB-PSEUDO connected configured ok ib::sol_uverbs,0 IB-PSEUDO connected configured ok
AP_Ids listed in the sample output are as follows:
Identifies an IB port device that is connected to port GUID and is bound to the ipib service.
Identifies a pseudo device.
Identifies an HCA device.
Identifies an IB HCA_SVC device that is bound to the hnfs service.
Identifies a pseudo device.
The following examples show the use of selected cfgadm options that customize the command output.
Example 18 Displaying Information About a Port DeviceThis example shows information about the port device ib::21280001A0A47A,0,ipib.
# cfgadm -al -s "cols=ap_id:info" ib::21280001A0A47A,0,ipib Ap_Id Information ib::21280001A0A47A,0,ipib ipibExample 19 Displaying Port and GUID Information of an HCA Device
This example shows the number of ports and their respective GUIDs for the HCA device hca::1730000008070.
# cfgadm -al -s "cols=ap_id:info" hca::1730000008070 Ap_Id Information hca::1730000008070 VID: 0x15b3, PID: 0x5a44, #ports: 0x2, port1 GUID: 0x1730000008071, port2 GUID: 0x1730000008072Example 20 Displaying Kernel Clients of an HCA Device
This example shows the kernel clients of the HCA hca:173000007F50.
$ cfgadm -x list_clients hca:173000007F50 Ap_Id IB Client Alternate HCA ib::1730000007F51D0 ibgen no ib::1730000007F51D1 ibgen no ib::1730000007F51,0,ipib ibd no ib::ibgen,0 ibgen no - ibdm no - ibmf no - nfs/ib no
In the sample output, if a kernel IB client uses an HCA other than hca:173000007F50, the entry under the column Alternate HCA would indicate yes. If IB Managers and kernel clients that do not use the HCA, their Ap_Ids would not be listed. Moreover, although EoIB uses the Solaris IB framework, the command does not list EoIB as a kernel IB client.
Example 21 Displaying Supported Communication ServicesThis example lists IB communication services currently used by the InfiniBand Transport Framework (IBTF).
# cfgadm -x list_services ib Port communication services: srp VPPA communication services: ibd HCA_SVC communication services: hnfsExample 22 Displaying IB Device Information by Using the prtconf Command
In addition to using the cfgadm command, you can also use the prtconf command to display general information about IB devices. In the following example, pci15b3,673c refers to an IB HCA.
$ prtconf . . . ib, instance #0 rpcib, instance #0 rdsib, instance #0 daplt, instance #0 rdsv3, instance #0 sdpib, instance #0 sol_umad, instance #0 sol_uverbs, instance #0 iser, instance #0 . . . pci15b3,673c, instance #0 ibport, instance #0 ibport, instance #1
You use cfgadm subcommands to configure or unconfigure IB devices. The commands apply to IB port, HCA_SVC, or VPPA devices.
To configure the IB device: cfgadm -c configure device
To unconfigure the IB device: cfgadm -c unconfigure device
In both commands, the device is represented by its Ap_Id.
A best practice is to display information about the system devices before and after configuring or unconfiguring a device. You can then properly identify the bus or device to dynamically reconfigure and verify from the state of the bus or device that the reconfiguration completed successfully.
The following examples show how to use the cfgadm command to perform DR on IB devices.
Example 23 Configuring an IB Port DeviceThe following example configures the IB port device ib::1730000007F51,*0*,ipib and verifies that the operation was successful.
# cfgadm -c configure ib::1730000007F51,*0*,ipib
# cfgadm -a ib::1730000007F51,*0*,ipib
Ap_Id Type Receptacle Occupant Condition
ib::1730000007F51,*0*,ipib IB-Port connected configured okThe port device is configured.
Note that if EoIB datalinks were configured over the IB Port when the port was previously unconfigured, you must re-create the VNICs corresponding to those EoIB datalinks on the EoIB gateway with the new IB HCA port GUIDs. You must re-create the vnics before re-configuring the IB Port. For information about how to create VNICs on the gateway, refer to the Sun Network QDR InfiniBand Gateway Switch Administration Guide.
Example 24 Unconfiguring an IB Port DeviceThis example unconfigures the IB port device ib::1730000007F51,*0*,ipib and verifies that the operation was successful.
# cfgadm -c unconfigure ib::1730000007F51,*0*,ipib
Unconfigure the device: /devices/ib:fabric::1730000007F51,*0*,ipib
This operation will suspend activity on the IB device
Continue (yes/no)? Y
# cfgadm -a ib::1730000007F51,*0*,ipib
Ap_Id Type Receptacle Occupant Condition
ib::1730000007F51,*0*,ipib IB-Port connected unconfigured unknownThe port device is unconfigured.
Example 25 Configuring an IB Pseudo Device
This example configures the pseudo device ib::ibgen,0 and verifies that the operation was successful.
# cfgadm -yc configure ib::ibgen,0
# cfgadm -a ib::ibgen,0
Ap_Id Type Receptacle Occupant Condition
ib::ibgen,0 IB-PSEUDO connected configured okThe device is configured.
Example 26 Unconfiguring an IB Pseudo Device
This example unconfigures the pseudo device ib::ibgen,0 and verifies that the operation was successful.
# cfgadm -c unconfigure ib::ibgen,0
Unconfigure the device: /devices/ib:fabric::ibgen,0
This operation will suspend activity on the IB device
Continue (yes/no)? Y
# cfgadm -a ib::ibgen,0
Ap_Id Type Receptacle Occupant Condition
ib::ibgen,0 IB-PSEUDO connected unconfigured unknownThe device is unconfigured.
This section describes how to modify existing IB configurations by supplying hardware option or hardware function information. The format and content of this information is hardware specific. You use the following cfgadm command options to supply the modification infformation:
cfgadm -o hardware-options – Supplies hardware-specific options. The format and content of the hardware option string is completely hardware specific.
cfgadm -x hardware-functions – Performs hardware-specific functions. Private hardware-specific functions can change the state of a receptacle or occupant. Attachment point conditions can change as the result of errors encountered during private hardware specific functions.
When you enable additional P_keys or disable them, the P_key table information of an HCA's port changes. For consistency, corresponding updates must also occur in the internal P_key databases of the InfiniBand Transport Framework (IBTF) and IBDM.
Example 27 Updating the IB P_key DatabasesThis example shows how to update the P_key databases of IBTF and IBDM.
# cfgadm -x update_pkey_tbls -y ib
For more information, see the ibtl(4D) and ibdm(4D) man pages.
You can add and remove communication services by using the cfgadm command options.
To add communication services:
cfgadm -o comm=vppa,service=svc-name -x add_service ib
To delete communication services:
# cfgadm -o comm=vppa,service=svc-name -x add_service ib
This example adds a VPPA communication service to the IB device and verifies that the operation was successful.
# cfgadm -o comm=vppa,service=newsvc -x add_service ib # cfgadm -x list_services ib Port communication services: srp VPPA communication services: ibd newsvc HCA_SVC communication services: nfs_serviceExample 29 Removing an Existing VPPA Communication Service
This example removes the newsvc communication service that was added in Example 28, Adding a VPPA Communication Service. and verifies that the operation was successful.
# cfgadm -o comm=vppa,service=comm1 -x delete_service ib # cfgadm -x list_services ib Port communication services: srp VPPA communication services: ibd HCA_SVC communication services: hnfs