1. Managing Network File Systems in Oracle Solaris 11.4
  2. Network File System Features
  3. How the NFS Service Works
  4. NFS Over RDMA

NFS Over RDMA

Starting in the Oracle Solaris 11.1 release, the default transport for NFS is the Remote Direct Memory Access (RDMA) protocol. This protocol provides memory-to-memory transfer of data over high-speed networks. Specifically, RDMA provides remote data transfer directly to and from memory without CPU intervention. RDMA also provides direct data placement, which eliminates data copies and therefore further eliminates CPU intervention. Thus, RDMA frees up not only the host CPU, but also reduces contention for the host memory and I/O buses. To provide this capability, RDMA combines the interconnect I/O technology of InfiniBand, which you can use on both SPARC and x86 platforms, with the Oracle Solaris operating system. The following figure shows the relationship of RDMA to other protocols, such as UDP and TCP.

Relationship of RDMA to Other Protocols


Graphic shows the relationship of RDMA to UDP and TCP.

Because RDMA is the default transport protocol for NFS, no special share or mount options are required to use RDMA on a client or server. The existing automounter maps, vfstab and file system shares, work with the RDMA transport. NFS mounts over the RDMA transport occur transparently when InfiniBand connectivity exists between the client and the server. InfiniBand connectivity feature works on both SPARC and x86 platforms. If the RDMA transport is not available on both the client and the server, the TCP transport is the initial fallback, followed by UDP if TCP is unavailable. However, if you use the proto=rdma mount option, NFS mounts are forced to use RDMA only.

To specify the use of only TCP and UDP, you can use the proto=tcp/udp mount option. This option disables RDMA on an NFS client. For more information about NFS mount options, see the mount_nfs(8) and mount(8) man pages.

Note:

RDMA for InfiniBand uses the IP addressing format and the IP lookup infrastructure to specify peers. However, because RDMA is a separate protocol stack, it does not fully implement all IP semantics. For example, RDMA does not use IP addressing to communicate with peers. Therefore, RDMA might bypass configurations for various security policies that are based on IP addresses. However, the NFS and RPC administrative policies, such as mount restrictions and secure RPC, are not bypassed.