InfiniBand (IB) is an I/O technology based on switched fabrics. It provides a high bandwidth and low latency interconnect for attaching I/O devices to hosts and for host-to-host communication. IB devices are managed by the Solaris IB nexus driver.
Oracle Solaris supports the following devices:
IP over IB (IPoIB) devices – Enables transporting the IP packets over IB connections. This feature is implemented by the ibp driver. See the ibp(7D) man page for details.
Ethernet over IB (EoIB) - Enables transporting Ethernet frames over IB fabric.
Socket Direct Protocol (SDP) – Supports sockets over IB
Reliable Datagram Service version 3 (RDSv3) and Reliable Datagram Service (RDS)
NFS over Remote Direct Memory Access (NFSoRDMA) - Provides NFS services over IB using RDMA
iSCSI Extension for RDMA (iSER) – Provides RDMA data transfer capability to the iSCSI protocol
User Direct Access Programming Language (uDAPL)
Open Fabric User Verb (OFUV)
The IB nexus driver queries the Solaris IB Device Manager (IBDM) for communication services to enumerate the IB Port, HCA_SVC, and IB VPPA devices, where VPPA refers to a virtual physical point of attachment.
The IB partition link represents a new part class of data link and is managed by using dladm subcommands. The partition links are used for data transfers. You can create an IB partition link on top of an IB physical link, one per each partition key (P_Key) on the port.
Port devices bind a communication service to a specific port of a Host Channel Adapter (HCA). A port is represented by a port number. HCA_SVC devices bind a communication service to a specific HCA. VPPA devices bind a communication service to a combination of a port and a partition key (P_key). Note that Port devices and HCA_SVC devices always use a partition key whose value is zero. Port, HCA_SVC, and VPPA devices are children of the HCA and are enumerated through the ib.conf file. For more information, see the ib(7D) man page.
Input Output Controller (IOC) devices are children of the IB nexus driver and are part of an I/O unit. Pseudo devices are also children of the IB nexus driver and refer to all other devices that provide their own configuration files to enumerate. For more information, see the ib(4) man page.
The following list shows possible IB device tree path names.
IOC device – /ib/ioc@1730000007F510C,173000007F50
IB pseudo device – /ib/driver@unit-address
IB VPPA device – /pci@1f,2000/pci@1/pci15b3,5a44@0/ibport@,port#,P_key,service
IB HCA_SVC device –
IB Port device – /pci@1f,2000/pci@1/pci15b3,5a44@0/ibport@<port#>,0,service
HCA – /pci@1f,2000/pci@1/pci15b3,5a44@0
The following list describes some of the IB components that are part of the path names.
A communication service. For example, ipib is the communication service used by the ibd kernel client driver.
The partition link key value being used.
The port number.
The IB kernel client driver's property by this name as specified in its driver.conf file. For more information, see driver.conf(4).
EoIB uses the services provided by the Oracle Solaris IB framework. However, EoIB is not a child of the IB nexus driver. Oracle Solaris EoIB is a single-instance pseudo device driver with a device tree path of /pseudo/eib@0. An EoIB datalink is a special eoib class of virtual links created and managed by dladm subcommands. You can create an EoIB datalink over an existing IB physical link, one for each EoIB Gateway discovered on the IB fabric from that IB physical link. To display a list of discovered EoIB Gateways, use the dladm subcommands.
For information about using IB diagnostic commands and utilities, see Monitoring and Troubleshooting IB Devices.