qperf
(1)
Name
qperf - Measure RDMA and IP performance
Synopsis
qperf
qperf SERVERNODE [OPTIONS] TESTS
Description
User Commands QPERF(1)
NAME
qperf - Measure RDMA and IP performance
SYNOPSIS
qperf
qperf SERVERNODE [OPTIONS] TESTS
DESCRIPTION
qperf measures bandwidth and latency between two nodes. It
can work over TCP/IP as well as the RDMA transports. On one
of the nodes, qperf is typically run with no arguments des-
ignating it the server node. One may then run qperf on a
client node to obtain measurements such as bandwidth,
latency and cpu utilization.
In its most basic form, qperf is run on one node in server
mode by invoking it with no arguments. On the other node,
it is run with two arguments: the name of the server node
followed by the name of the test. A list of tests can be
found in the section, TESTS. A variety of options may also
be specified.
One can get more detailed information on qperf by using the
--help option. Below are examples of using the --help
option:
qperf --help examples Some examples of using qperf
qperf --help opts Summary of options
qperf --help options Description of options
qperf --help tests Short summary and descrip-
tion of tests
qperf --help TESTNAME More information on test
TESTNAME
EXAMPLES
In these examples, we first run qperf on a node called
myserver in server mode by invoking it with no arguments.
In all the subsequent examples, we run qperf on another node
and connect to the server which we assume has a hostname of
myserver.
To run a TCP bandwidth and latency test:
qperf myserver tcp_bw tcp_lat
To run a SDP bandwidth test for 10 seconds:
qperf myserver -t 10 sdp_bw
To run a UDP latency test and then cause the server to ter-
minate:
qperf myserver udp_lat quit
qperf Last change: August 2009 1
User Commands QPERF(1)
To measure the RDMA UD latency and bandwidth:
qperf myserver ud_lat ud_bw
To measure RDMA UC bi-directional bandwidth:
qperf myserver rc_bi_bw
To get a range of TCP latencies with a message size from 1
to 64K
qperf myserver -oo msg_size:1:64K:*2 -vu tcp_lat
OPTIONS
-ar, --access_recv OnOff
If OnOff is non-zero, data is accessed once received.
Otherwise, data is ignored. By default, OnOff is 0.
This can help to mimic some applications.
-ar1 Cause received data to be accessed.
-ap, --alt_port Port
Set alternate path port. This enables automatic path
failover.
-lap, --loc_alt_port Port
Set local alternate path port. This enables automatic
path failover.
-rap, --rem_alt_port Port
Set remote alternate path port. This enables automatic
path failover.
-ca, --cpu_affinity PN
Set cpu affinity to PN. CPUs are numbered sequentially
from 0. If PN is "any", any cpu is allowed otherwise
the cpu is limited to the one specified.
-lca, --loc_cpu_affinity PN
Set local processor affinity to PN.
-rca, --rem_cpu_affinity PN
Set remote processor affinity to PN.
-f, --flip OnOff
If non-zero, cause sender and receiver to play opposite
roles.
-f1 Cause sender and receiver to play opposite roles.
-h, --help Topic
Print out information about Topic. To see the list of
topics, type qperf --help
-H, --host Host
qperf Last change: August 2009 2
User Commands QPERF(1)
Run test between the current node and the qperf running
on node Host. This can also be specified as the first
non-option argument.
-i, --id Device:Port
Use RDMA Device and Port.
-li, --loc_id Device:Port
Use local RDMA Device and Port.
-ri, --rem_id Device:Port
Use remote RDMA Device and Port.
-lp, --listen_port Port
Set the port we listen on to ListenPort. This must be
set to the same port on both the server and client
machines. The default value is 19765.
-oo, --loop Var:Init:Last:Incr
Run a test multiple times sequencing through a series
of values. Var is the loop variable; Init is the ini-
tial value; Last is the value it must not exceed and
Incr is the increment. It is useful to set the --ver-
bose_used (-vu) option in conjunction with this option.
-m, --msg_size Size
Set the message size to Size. The default value varies
by test. It is assumed that the value is specified in
bytes however, a trailing kib or K, mib or M, or gib or
G indicates that the size is being specified in
kibibytes, mebibytes or gibibytes respectively while a
trailing kb or k, mb or m, or gb or g indicates kilo-
bytes, megabytes or gigabytes respectively.
-mt, --mtu_size Size
Set the MTU size. Only relevant to the RDMA UC/RC
tests. Units are specified in the same manner as the
--msg_size option.
-n, --no_msgs N
Set test duration by number of messages sent instead of
time.
-cp, --cq_poll OnOff
Turn polling mode on or off. This is only relevant to
the RDMA tests and determines whether they poll or wait
on the completion queues. If OnOff is 0, they wait;
otherwise they poll.
-lcp, --loc_cq_poll OnOff
Locally turn polling mode on or off.
qperf Last change: August 2009 3
User Commands QPERF(1)
-rcp, --rem_cq_poll OnOff
Remotely turn polling mode on or off.
-cp1 Turn polling mode on.
-lcp1
Turn local polling mode on.
-rcp1
Turn remote polling mode on.
-ip, --ip_port Port
Use Port to run the socket tests. This is different
from --listen_port which is used for synchronization.
This is only relevant for the socket tests and refers
to the TCP/UDP/SDP/RDS/SCTP port that the test is run
on.
-e, --precision Digits
Set the number of significant digits that are used to
report results.
-nr, --rd_atomic Max
Set the number of in-flight operations that can be han-
dled for a RDMA read or atomic operation to Max. This
is only relevant to the RDMA Read and Atomic tests.
-lnr, --loc_rd_atomic Max
Set local read/atomic count.
-rnr, --rem_rd_atomic Max
Set remote read/atomic count.
-sl, --service_level SL
Set RDMA service level to SL. This is only used by the
RDMA tests. The service level must be between 0 and
15. The default service level is 0.
-lsl, --loc_service_level SL
Set local service level.
-rsl, --rem_service_level SL
Set remote service level.
-sb, --sock_buf_size Size
Set the socket buffer size. This is only relevant to
the socket tests.
-lsb, --loc_sock_buf_size Size
Set local socket buffer size.
-rsb, --rem_sock_buf_size Size
qperf Last change: August 2009 4
User Commands QPERF(1)
Set remote socket buffer size.
-sp, --src_path_bits N
Set source path bits. If the LMC is not zero, this will
cause the connection to use a LID with the low order
LMC bits set to N.
-lsp, --loc_src_path_bits N
Set local source path bits.
-rsp, --rem_src_path_bits N
Set remote source path bits.
-sr, --static_rate Rate
Force InfiniBand static rate. Rate can be one of: 2.5,
5, 10, 20, 30, 40, 60, 80, 120, 1xSDR (2.5 Gbps), 1xDDR
(5 Gbps), 1xQDR (10 Gbps), 4xSDR (2.5 Gbps), 4xDDR (5
Gbps), 4xQDR (10 Gbps), 8xSDR (2.5 Gbps), 8xDDR (5
Gbps), 8xQDR (10 Gbps).
-lsr, --loc_static_rate
Force local InfiniBand static rate
-rsr, --rem_static_rate
Force remote InfiniBand static rate
-t, --time Time
Set test duration to Time. Specified in seconds how-
ever a trailing m, h or d indicates that the time is
specified in minutes, hours or days respectively.
-to, --timeout Time
Set timeout to Time. This is the timeout used for var-
ious things such as exchanging messages. The default
is 5 seconds.
-lto, --loc_timeout Time
Set local timeout to Time. This may be used on the
server to set the timeout when initially exchanging
data with each client. However, as soon as we receive
the client's parameters, the client's remote timeout
will override this parameter.
-rto, --rem_timeout Time
Set remote timeout to Time.
-un, --unify_nodes
Unify the nodes. Describe them in terms of local and
remote rather than send and receive.
-uu, --unify_units
Unify the units that results are shown in. Uses the
qperf Last change: August 2009 5
User Commands QPERF(1)
lowest common denominator. Helpful for scripts.
-ub, --use_bits_per_sec
Use bits/sec rather than bytes/sec when displaying net-
working speed.
-cm, --use_cm OnOff
Use the RDMA Connection Manager (CM) if OnOff is non-
zero. It is necessary to use the CM for iWARP devices.
The default is to establish the connection without
using the CM. This only works for the tests that use
the RC transport.
-cm1 Use RDMA Connection Manager.
-v, --verbose
Provide more detailed output. Turns on -vc, -vs, -vt
and -vu.
-vc, --verbose_conf
Provide information on configuration.
-vs, --verbose_stat
Provide information on statistics.
-vt, --verbose_time
Provide information on timing.
-vu, --verbose_used
Provide information on parameters used.
-vv, --verbose_more
Provide even more detailed output. Turns on -vvc,
-vvs, -vvt and -vvu.
-vvc, --verbose_more_conf
Provide more information on configuration.
-vvs, --verbose_more_stat
Provide more information on statistics.
-vvt, --verbose_more_time
Provide more information on timing.
-vvu, --verbose_more_used
Provide more information on parameters used.
-V, --version
The current version of qperf is printed.
-ws, --wait_server Time
If the server is not ready, continue to try connecting
qperf Last change: August 2009 6
User Commands QPERF(1)
for Time seconds before giving up. The default is 5
seconds.
TESTS
conf Show configuration
quit Cause the server to quit
rds_bw
RDS streaming one way bandwidth
rds_lat
RDS one way latency
sctp_bw
SCTP streaming one way bandwidth
sctp_lat
SCTP one way latency
sdp_bw
SDP streaming one way bandwidth
sdp_lat
SDP one way latency
tcp_bw
TCP streaming one way bandwidth
tcp_lat
TCP one way latency
udp_bw
UDP streaming one way bandwidth
udp_lat
UDP one way latency
rc_bi_bw
RC streaming two way bandwidth
rc_bw
RC streaming one way bandwidth
rc_lat
RC one way latency
uc_bi_bw
UC streaming two way bandwidth
uc_bw
UC streaming one way bandwidth
qperf Last change: August 2009 7
User Commands QPERF(1)
uc_lat
UC one way latency
ud_bi_bw
UD streaming two way bandwidth
ud_bw
UD streaming one way bandwidth
ud_lat
UD one way latency
xrc_bi_bw
XRC streaming two way bandwidth
xrc_bw
XRC streaming one way bandwidth
xrc_lat
XRC one way latency
rc_rdma_read_bw
RC RDMA read streaming one way bandwidth
rc_rdma_read_lat
RC RDMA read one way latency
rc_rdma_write_bw
RC RDMA write streaming one way bandwidth
rc_rdma_write_lat
RC RDMA write one way latency
rc_rdma_write_poll_lat
RC RDMA write one way polling latency
uc_rdma_write_bw
UC RDMA write streaming one way bandwidth
uc_rdma_write_lat
UC RDMA write one way latency
uc_rdma_write_poll_lat
UC RDMA write one way polling latency
rc_compare_swap_mr
RC compare and swap messaging rate
rc_fetch_add_mr
RC fetch and add messaging rate
ver_rc_compare_swap
qperf Last change: August 2009 8
User Commands QPERF(1)
Verify RC compare and swap
ver_rc_fetch_add
Verify RC fetch and add
AUTHOR
Written by Johann George.
BUGS
None of the RDMA tests are available if qperf is compiled
without the RDMA libraries. The -f option is not yet imple-
mented in many of the tests.
ATTRIBUTES
See attributes(5) for descriptions of the following
attributes:
+---------------+----------------------+
|ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+---------------+----------------------+
|Availability | network/open-fabrics |
+---------------+----------------------+
|Stability | Volatile |
+---------------+----------------------+
NOTES
This software was built from source available at
https://java.net/projects/solaris-userland. The original
community source was downloaded from ['http://www.openfab-
rics.org/downloads/ibutils/ibutils-1.5.7.tar.gz',
'http://www.openfabrics.org/downloads/libibverbs/libib-
verbs-1.1.4-1.22.g7257cd3.tar.gz', 'http://www.openfab-
rics.org/down-
loads/libmlx4/libmlx4-1.0.1-1.18.gb810a27.tar.gz',
'http://www.openfabrics.org/downloads/libsdp/lib-
sdp-1.1.108-0.15.gd7fdb72.tar.gz', 'http://www.openfab-
rics.org/downloads/management/infiniband-
diags-1.5.8.tar.gz', 'http://www.openfabrics.org/down-
loads/management/libibmad-1.3.7.tar.gz', 'http://www.open-
fabrics.org/downloads/management/libibumad-1.3.7.tar.gz',
'http://www.openfabrics.org/downloads/manage-
ment/opensm-3.3.9.tar.gz', 'http://www.openfabrics.org/down-
loads/perftest/perftest-1.3.0-0.42.gf350d3d.tar.gz',
'http://www.openfabrics.org/down-
loads/qperf/qperf-0.4.6-0.1.gb81434e.tar.gz',
'http://www.openfabrics.org/downloads/rdmacm/librd-
macm-1.0.14.1.tar.gz', 'http://www.openfabrics.org/down-
loads/rds-tools/rds-tools-2.0.4.tar.gz']
Further information about this software can be found on the
open source community website at
qperf Last change: August 2009 9
User Commands QPERF(1)
http://www.openfabrics.org/.
qperf Last change: August 2009 10