Using the uDAPL Application Interface With InfiniBand Devices

User Direct Access Programming Library (uDAPL) is a standard API that promotes data center application data messaging performance, scalability, and reliability over Remote Direct Memory Access (RDMA) capable interconnects such as InfiniBand. The uDAPL interface is defined by the DAT collaborative. For more information about the DAT collaborative, go to the following site:

http://www.datcollaborative.org

The Solaris release provides the following uDAPL features:

• A standard DAT registry library, libdat. For more information, see libdat(3LIB).

• A standard service provider registration file, dat.conf. For more information, see dat.conf(4).

• Support for multiple service providers so that each provider specifies their own uDAPL library path, version number, and so on, in their own service_provider.conf file. For more information, see, service_provider.conf(4).

• An administrative tool, the datadm command, to configure dat.conf. For more information, see datadm(1M).

• A new resource control property, project.max-device-locked-memory, to regulate the amount of locked down physical memory.

• A naming scheme that uses either IPv4 or IPv6 addresses that leverage the IP infrastructure, such as ARP in IPv4 and neighbor discovery in IPv6, for address resolution. The Solaris uDAPL Interface Adapter directly maps to an IPoIB device instance.

• Support for the standard Address Translation Scheme that is used by the DAT collaborative community.

• A uDAPL service provider library to support the Mellanox Tavor Host Channel Adapter with automatic registration to the dat.conf registration file.

• Supports both SPARC platform and x86 platforms.

How to Enable uDAPL

1. Become superuser.
2. Confirm that the following packages are installed. Or, install them, if needed.

   • SUNWib – Sun InfiniBand Framework

   • SUNWtavor – Sun Tavor HCA Driver

   • SUNWipoib – Sun IP over InfiniBand

   • SUNWudaplr – Direct Access Transport (DAT) registry package (root)

   • SUNWudaplu – Direct Access Transport (DAT) registry packages (usr)

   • SUNWudapltr – Service Provider for Tavor packages (root)

   • SUNWudapltu – Service Provider for Tavor packages (usr)

3. Select one of the following to plumb the IPoIB interfaces.

   • Manually plumb the interfaces with the ifconfig and datadm commands.

     For example:

     # ifconfig ibd1 plumb
     # ifconfig ibd1 192.168.0.1/24 up
     # datadm -a /usr/share/dat/SUNWudaplt.conf

   • Automatically plumb the interfaces by doing the following:

     • Create the following file with the appropriate IP address.

       /etc/hostname.ibd1

     • Reboot the system.

Updating the DAT Static Registry

You can use the datadm command to maintain the DAT static registry, the dat.conf file. For more information about this file, see dat.conf(4).

The datadm command can also be used to register or unregister a service provider to the dat.conf file. For more information, see datadm(1M).

When IPoIB interface adapters are added or removed, run the datadm command to update the dat.conf file to reflect the current state of the system. A new set of interface adapters for all the service providers that are currently installed will be regenerated.

How to Update the DAT Static Registry

1. Become superuser.
2. Update the DAT static registry after you add or remove IPoIP interface adapters from the system.

   # datadm -u

3. Display the updated DAT static registry.

   # datadm

How to Register a Service Provider in the DAT Static Registry

1. Become superuser.
2. Update the DAT static registry after you add Sun's service provider for the Mellanox Tavor Host Channel Adapter.

   # datadm -a /usr/share/dat/SUNWudaplt.conf

3. Display the updated DAT static registry.

   # datadm -v

How to Unregister a Service Provider from the DAT Static Registry

1. Become superuser.
2. Update the DAT static registry after you remove Sun's service provider for the Mellanox Tavor Host Channel Adapter from the system.

   # datadm -r /usr/share/dat/SUNWudaplt.conf

3. Display the updated DAT static registry.

   # datadm -v