IP over InfiniBand (IPoIB) devices enable transporting IP packets over IB connections. You configure IPoIB VNICs by specifying the partition key. Although you can migrate IPoIB VNICs from one underlying datalink to another underlying datalink, IPoIB partition links do not support migration. For more information, see About InfiniBand Devices in Managing Devices in Oracle Solaris 11.4.
Before You Begin
Ensure that your role has the appropriate rights profile to perform this procedure. See Using Rights Profiles to Perform Network Configuration.
$ dladm show-ib link
The output shows the datalinks and information about their partition keys.
$ dladm create-vnic [-f] -l link -P pkey -p [prop=value] VNIC
Optional. Forces the creation of the IPoIB VNIC even if the partition key on the port or the multicast group is absent, or the port is down.
The partition key that needs to be used. This option is mandatory for IPoIB VNICs and not applicable for other types of datalinks. When you specify the partition key, it is always considered a hexadecimal, regardless of whether it has the prefix 0x.
Used to specify the value of the linkmode property of the IPoIB VNIC, which enables you to set the link transport service type on an IB partition datalink. You can set the following values for the linkmode property:
cm - connected mode. This mode uses a default maximum transmission unit (MTU) of 65520 bytes and supports a maximum MTU of 65535 bytes. If connected mode is not available for a remote node, unreliable datagram mode is automatically used. cm is the default value.
ud - unreliable datagram mode. This mode uses a default MTU of 2044 bytes and supports a maximum MTU of 4092 bytes.
$ dladm show-vnic
$ ipadm create-ip interface $ ipadm create-addr -a address interface [address-object]
The name of the IPoIB VNIC that you created.
Specifies the IP address.
A name that identifies the IP address in association with the IP interface. If address-object is not specified, the OS automatically assigns a name by using the format IP-name/protocol.
The following example shows how to create an IPoIB VNIC over the datalink net4 by using the partition key 0xffff.
$ dladm show-ib net4 LINK HCAGUID PORTGUID PORT STATE GWNAME GWPORT PKEYS net4 21280001A0A58C 21280001A0A58D 1 up -- -- FFFF $ dladm create-vnic -l net4 -P 0xffff ipoib_vnic0 $ dladm show-vnic LINK OVER SPEED MACADDRESS MACADDRTYPE IDS eth_vnic0 net0 1000 2:8:20:ef:d2:77 random VID:0 ipoib_vnic0 net4 32000 80:0:0:4a:fe:.. fixed PKEY:0xFFFF $ ipadm create-ip ipoib_vnic0 $ ipadm create-addr -a 192.0.2.10 ipoib_vnic0/v4
To display information about selected properties only, use the –o link,[properties[ option. Separate multiple properties with a comma.
IPoIB VNICs can be down in any of following scenarios:
HCA port is down
Partition key is not present
IPoIB VNIC fails to join the broadcast multicast group
You can use the dladm show-ib command to check the HCA port status and the configured partition key. To find out if the IPoIB VNIC has failed to join the broadcast multicast group, you can use the dladm show-linkprop command to display the broadcast-group datalink property of the IPoIB VNIC. The valid values of this read-only property are unknown, joined, assigned, and unsuccessful, and the default value is unknown.
Example 6 Identifying the Link Status of an IPoIB VNICThis example shows how to find the link state of an IPoIB VNIC.
$ dladm show-ib net1
LINK HCAGUID PORTGUID PORT STATE GWNAME GWPORT PKEYS
net4 21290001A0A58C 21280001A0A59D 1 up -- -- FFFF
$ dladm create-vnic -l net4 -P 0xffff ip_vnic0
$ dladm show-vnic
LINK OVER SPEED MACADDRESS MACADDRTYPE IDS
eth_vnic0 net0 1000 2:8:20:ef:d2:77 random VID:0
ip_vnic0 net1 32000 80:0:0:4a:fe:.. fixed PKEY:0xFFFF
$ dladm show-linkprop -p broadcast-group vnic0
LINK PROPERTY PERM VALUE EFFECTIVE DEFAULT POSSIBLE
ip_vnic0 broadcast-group r- joined joined unknown unknown
absent,
joined,
unsuccessful
The broadcast-group property can have the following values:
Initial state of the datalink after it is created and before an IP address is configured over it.
Broadcast group is not created on the subnet manager (SM).
Broadcast group is configured and IPoIB has successfully joined the broadcast group.
Broadcast group is configured but IPoIB has failed to join the broadcast group. This can occur if one or more parameters such as MTU, state, and key are different from broadcast group created on the subnet manager. See SM log to learn the exact reason for the failure of the link to join the broadcast group.
The value of the broadcast-group property is displayed as joined when the VNIC is created.
The broadcast-group property displays the value unsuccessful when the broadcast group is configured but IPoIB failed to join the broadcast group.
For information about IPoIB devices, see Administering IPoIB Devices in Managing Devices in Oracle Solaris 11.4.
You migrate and delete IPoIB VNICs just as you would with Ethernet VNICs. For more information, see Migrating VNICs and Deleting VNICs.
The following example shows how to migrate ipoib_vnic0 to the datalink net5.
$ dladm modify-vnic -l net5 ipoib_vnic0
The following example shows how to delete ipoib_vnic0.
$ dladm delete-vnic ipoib_vnic0