Go to main content

man pages section 7: Standards, Environments, Macros, Character Sets, and Miscellany

Exit Print View

Updated: Wednesday, February 9, 2022
 
 

rdma_cm (7)

Name

rdma_cm - RDMA communication manager.

Synopsis

#include <rdma/rdma_cma.h>

Description

Librdmacm Programmer's Manual                                       RDMA_CM(7)



NAME
       rdma_cm - RDMA communication manager.

SYNOPSIS
       #include <rdma/rdma_cma.h>

DESCRIPTION
       Used to establish communication over RDMA transports.


ATTRIBUTES
       See attributes(7) for descriptions of the following attributes:


       +---------------+-----------------------+
       |ATTRIBUTE TYPE |   ATTRIBUTE VALUE     |
       +---------------+-----------------------+
       |Availability   | network/open-fabrics  |
       +---------------+-----------------------+
       |Stability      | Pass-through volatile |
       +---------------+-----------------------+

NOTES
       The  RDMA  CM  is  a communication manager used to setup reliable, con-
       nected and unreliable datagram data transfers.   It  provides  an  RDMA
       transport neutral interface for establishing connections.  The API con-
       cepts are based on sockets, but  adapted  for  queue  pair  (QP)  based
       semantics:  communication must be over a specific RDMA device, and data
       transfers are message based.

       The RDMA CM can control both the QP and communication management  (con-
       nection setup / teardown) portions of an RDMA API, or only the communi-
       cation management piece.  It works in conjunction with  the  verbs  API
       defined by the libibverbs library.  The libibverbs library provides the
       underlying interfaces needed to send and receive data.

       The RDMA CM can operate asynchronously.  The mode of operation is  con-
       trolled by the user through the use of the rdma_cm event channel param-
       eter in specific calls.  If an event channel is  provided,  an  rdma_cm
       identifier will report its event data (results of connecting, for exam-
       ple), on that channel.  If a channel is not provided, then all  rdma_cm
       operations  for  the  selected rdma_cm identifier will block until they
       complete.

       Source code for open source software components in Oracle  Solaris  can
       be found at https://www.oracle.com/downloads/opensource/solaris-source-
       code-downloads.html.

       This    software    was    built    from    source     available     at
       https://github.com/oracle/solaris-userland.    The  original  community
       source   was   downloaded   from    ['https://www.openfabrics.org/down-
       loads/ibutils/ibutils-1.5.7-0.2.gbd7e502.tar.gz', 'https://www.openfab-
       rics.org/downloads/libibverbs/libibverbs-1.1.8.tar.gz',
       'https://www.openfabrics.org/downloads/libmlx4/libmlx4-1.0.6.tar.gz',
       'https://www.openfabrics.org/downloads/libsdp/lib-
       sdp-1.1.108-0.15.gd7fdb72.tar.gz',   'https://www.openfabrics.org/down-
       loads/management/infiniband-diags-1.6.5.tar.gz',  'https://www.openfab-
       rics.org/downloads/management/libibmad-1.3.12.tar.gz',
       'https://www.openfabrics.org/downloads/management/libibu-
       mad-1.3.10.2.tar.gz',    'https://www.openfabrics.org/downloads/manage-
       ment/opensm-3.3.19.tar.gz',          'https://www.openfabrics.org/down-
       loads/perftest/perftest-1.3.0-0.42.gf350d3d.tar.gz', 'https://www.open-
       fabrics.org/downloads/qperf/qperf-0.4.9.tar.gz',  'https://www.openfab-
       rics.org/downloads/rdmacm/librdmacm-1.0.21.tar.gz',  'https://www.open-
       fabrics.org/downloads/rds-tools/rds-tools-2.0.4.tar.gz'].

       Further information about this software can be found on the open source
       community website at http://www.openfabrics.org/.

RDMA VERBS
       The  rdma_cm  supports  the  full  range of verbs available through the
       libibverbs library and interfaces.  However, it also  provides  wrapper
       functions for some of the more commonly used verbs funcationality.  The
       full set of abstracted verb calls are:

CLIENT OPERATION
       This section provides a general overview of the basic operation for the
       active,  or  client, side of communication.  This flow assume asynchro-
       nous operation with low level call details shown.  Users may also refer
       to  the  example  applications  for code samples.  A general connection
       flow would be:

       rdma_getaddrinfo
              retrieve address information of the destination

       rdma_create_event_channel
              create channel to receive events

       rdma_create_id
              allocate an rdma_cm_id, this is conceptually similar to a socket

       rdma_resolve_addr
              obtain a local RDMA device to reach the remote address

       rdma_get_cm_event
              wait for RDMA_CM_EVENT_ADDR_RESOLVED event

       rdma_ack_cm_event
              ack event

       rdma_create_qp
              allocate a QP for the communication

       rdma_resolve_route
              determine the route to the remote address

       rdma_get_cm_event
              wait for RDMA_CM_EVENT_ROUTE_RESOLVED event

       rdma_ack_cm_event
              ack event

       rdma_connect
              connect to the remote server

       rdma_get_cm_event
              wait for RDMA_CM_EVENT_ESTABLISHED event

       rdma_ack_cm_event
              ack event

       Perform data transfers over connection

       rdma_disconnect
              tear-down connection

       rdma_get_cm_event
              wait for RDMA_CM_EVENT_DISCONNECTED event

       rdma_ack_cm_event
              ack event

       rdma_destroy_qp
              destroy the QP

       rdma_destroy_id
              release the rdma_cm_id

       rdma_destroy_event_channel
              release the event channel

       An almost identical process is used to setup unreliable  datagram  (UD)
       communication  between  nodes.   No actual connection is formed between
       QPs however, so disconnection is not needed.

       Although this example  shows  the  client  initiating  the  disconnect,
       either side of a connection may initiate the disconnect.

SERVER OPERATION
       This section provides a general overview of the basic operation for the
       passive, or server, side of communication.  A general  connection  flow
       would be:

       rdma_create_event_channel
              create channel to receive events

       rdma_create_id
              allocate an rdma_cm_id, this is conceptually similar to a socket

       rdma_bind_addr
              set the local port number to listen on

       rdma_listen
              begin listening for connection requests

       rdma_get_cm_event
              wait   for   RDMA_CM_EVENT_CONNECT_REQUEST   event  with  a  new
              rdma_cm_id

       rdma_create_qp
              allocate a QP for the communication on the new rdma_cm_id

       rdma_accept
              accept the connection request

       rdma_ack_cm_event
              ack event

       rdma_get_cm_event
              wait for RDMA_CM_EVENT_ESTABLISHED event

       rdma_ack_cm_event
              ack event

       Perform data transfers over connection

       rdma_get_cm_event
              wait for RDMA_CM_EVENT_DISCONNECTED event

       rdma_ack_cm_event
              ack event

       rdma_disconnect
              tear-down connection

       rdma_destroy_qp
              destroy the QP

       rdma_destroy_id
              release the connected rdma_cm_id

       rdma_destroy_id
              release the listening rdma_cm_id

       rdma_destroy_event_channel
              release the event channel

RETURN CODES
       =  0   success

       = -1   error - see errno for more details

       Most librdmacm functions return 0 to indicate success, and a -1  return
       value  to  indicate  failure.  If a function operates asynchronously, a
       return value of 0 means that the operation  was  successfully  started.
       The  operation  could  still  complete in error; users should check the
       status of the related event.  If the return value  is  -1,  then  errno
       will  contain additional information regarding the reason for the fail-
       ure.

       Prior versions of the library would return -errno and not set errno for
       some  cases  related  to  ENOMEM, ENODEV, ENODATA, EINVAL, and EADDRNO-
       TAVAIL codes. Applications that want to check these codes and have com-
       patability  with  prior library versions must manually set errno to the
       negative of the return code if it is < -1.

SEE ALSO
       rdma_accept(3),  rdma_ack_cm_event(3),   rdma_bind_addr(3),   rdma_con-
       nect(3),   rdma_create_event_channel(3),  rdma_create_id(3),  rdma_cre-
       ate_qp(3),      rdma_destroy_event_channel(3),      rdma_destroy_id(3),
       rdma_destroy_qp(3),        rdma_disconnect(3),       rdma_event_str(3),
       rdma_free_devices(3),    rdma_getaddrinfo(3),     rdma_get_cm_event(3),
       rdma_get_devices(3),    rdma_get_dst_port(3),   rdma_get_local_addr(3),
       rdma_get_peer_addr(3),  rdma_get_src_port(3),   rdma_join_multicast(3),
       rdma_leave_multicast(3),         rdma_listen(3),        rdma_notify(3),
       rdma_reject(3),      rdma_resolve_addr(3),       rdma_resolve_route(3),
       rdma_set_option(3)  mckey(1), rdma_client(1), rdma_server(1), rping(1),
       ucmatose(1), udaddy(1)




librdmacm                         2010-07-19                        RDMA_CM(7)