rds-stress - send messages between processes over RDS sockets
rds-stress [-p port_number] -r [receive_address] [-s send_address] [-Q tos] [-a ack_bytes] [-q request_bytes] [-D rdma_bytes] [-d queue_depth] [-t nr_tasks] [-T time] [-c] [-R] [-V] [-v]
RDS-STRESS(1) General Commands Manual RDS-STRESS(1)
NAME
rds-stress - send messages between processes over RDS sockets
SYNOPSIS
rds-stress [-p port_number] -r [receive_address] [-s send_address]
[-Q tos] [-a ack_bytes] [-q request_bytes] [-D rdma_bytes]
[-d queue_depth] [-t nr_tasks] [-T time] [-c] [-R] [-V] [-v]
DESCRIPTION
rds-stress sends messages between groups tasks, usually running on
seperate machines.
First a passive receiving instance is started.
$ rds-stress
Then an active sending instance is started, giving it the address and
port at which it will find a listening passive receiver. In addition,
it is given configuration options which both instances will use.
$ rds-stress -s recvhost -p 4000 -t 1 -d 1
The active sender will parse the options, connect to the passive
receiver, and send the options over this connection. From this point
on both instances exhibit the exact same behaviour.
They will create a number of child tasks as specified by the -t option.
Once the children are created the parent sleeps for a second at a time,
printing a summary of statistics at each interval.
Each child will open an RDS socket, each binding to a port number in
order after the port number given on the command line. The first child
would bind to port 4001 in our example. Each child sets the send and
receive buffers to exactly fit the number of messages, requests and
acks, that will be in flight as determind by the command line argu-
ments.
The children then enter their loop. They will keep a number of sent
messages outstanding as specified by the -d option. When they reach
this limit they will wait to receive acks which will allow them to send
again. As they receive messages from their peers they immediately send
acks.
Every second, the parent process will display statistics of the ongoing
stress test. The output is described in section OUTPUT below.
If the -T option is given, the test will terminate after the specified
time, and a summary is printed.
Each child maintains outstanding messages to all other children of the
other instance. They do not send to their siblings.
OPTIONS
The following options are available for use on the command line:
-p port_number
Each parent binds a TCP socket to this port number and their
respective address. They will trade the negotiated options over
this socket. Each child will bind an RDS socket to the range of
ports immediately following this port number, for as many chil-
dren as there are.
-s send_address
A connection attempt is made to this address. Once its complete
and the options are sent over it then children will be created
and work will proceed.
-r receive_address
This specifies the address that messages will be sent from. If
-s is not specified then rds-stress waits for a connection on
this address before proceeding.
If this option is not given, rds-stress will choose an appropri-
ate address. The passive process will accept connections on all
local interfaces, and obtain the address once the control con-
nection is established. The active process will choose a local
address based on the interface through which it connects to the
destination address.
-Q tos Uses the RDS connection between IP addresses with the specified
tos value. By default, the base (tos = 0) RDS connection is
used. Valid values are 0-255.
-a ack_bytes
This specifies the size of the ack messages, in bytes. There is
a minimum size which depends on the format of the ack messages,
which may change over time. See section "Message Sizes" below.
-q request_bytes
This specifies the size of the request messages, in bytes. It
also has a minimum size which may change over time. See section
"Message Sizes" below.
-D rdma_bytes
RDSv3 is capable of transmitting part of a message via RDMA
directly from application buffer to application buffer. This
option enables RDMA support in rds-stress: request packets
include parameters for an RDMA READ or WRITE operation, which
the receiving process executes at the time the ACK packet is
sent. See section "Message Sizes" below.
-d queue_depth
Each child will try to maintain this many sent messages out-
standing to each of its peers on the remote address.
-t nr_tasks
Each parent will create this many children tasks.
-T seconds
Specify the duration of the test run. After the specified number
of seconds, all processes on both ends of the connection will
terminate, and the active instance will print a summary. By
default, rds-stress will keep on sending and receiving messages.
-z This flag can be used in conjunction with -T. It suppresses the
ongoing display of statistics, and prints a summary only.
-c This causes rds-stress to create child tasks which just consume
CPU cycles. One task is created for each CPU in the system.
First each child observes the maximum rate at which it can con-
sume cycles. This means that this option should only be given
on an idle system. rds-stress can then calculate the CPU use of
the system by observing the lesser rate at which the children
consume cycles. This option is *not* shared between the active
and passive instances. It must be specified on each rds-stress
command line.
-R This tells the rds-stress parent process to run with SCHED_RR
priority, giving it precedence over the child processes. This is
useful when running with lots of tasks, where there is a risk of
the child processes starving the parent, and skewing the
results.
-v With this option enabled, packets are filled with a pattern that
is verified by the receiver. This check can help detect data
corruption occuring under high load.
Message Sizes
Options which set a message size (such as -a) specify a number
of bytes by default. By appending K, M, or G, you can specify
the size in kilobytes, megabytes or gigabytes, respectively. For
instance, the following will run rds-stress with a message and
ACK size of 1024 bytes, and an RDMA message size of 1048576
bytes:
rds-stress ... -q 1K -a 1K -D 1M
OUTPUT
Each parent outputs columns of statistics at a regular interval:
tsks The number of child tasks which are running.
tx/s The number of sendmsg() calls that all children are executing,
per second.
rx/s The number of recvmsg() calls that all children are executing,
per second.
tx+rx K/s
The total number of bytes that are flowing through sendmsg()
and recvmsg() for all children. This includes both request and
ack messages.
mbi K/s The total number of bytes that are being received via RDMA
READs and WRITEs for all children.
mbo K/s The total number of bytes that are being transmited via RDMA
READs and WRITEs for all children.
tx us/c The average number of microseconds spent in sendmsg() calls.
rtt us The average round trip time for a request and ack message pair.
This measures the total time between when a task sends a
request and when it finally receives the ack for that message.
Because it includes the time it takes for the receiver to wake
up, receive the message, and send an ack, it can grow to be
quite large under load.
cpu % This is the percentage of available CPU resources on this
machine that are being consumed since rds-stress started run-
ning. It will show -1.00 if -c is not given. It is calculated
based on the amount of CPU resources that CPU soaking tasks are
able to consume. This lets it measure CPU use by the system,
say in interrupt handlers, that task-based CPU accounting does
not include. For this to work rds-stress must be started with
-c on an idle system.
ATTRIBUTES
See attributes(7) for descriptions of the following attributes:
+---------------+-----------------------+
|ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+---------------+-----------------------+
|Availability | network/open-fabrics |
+---------------+-----------------------+
|Stability | Pass-through volatile |
+---------------+-----------------------+
NOTES
Source code for open source software components in Oracle Solaris can
be found at https://www.oracle.com/downloads/opensource/solaris-source-
code-downloads.html.
This software was built from source available at
https://github.com/oracle/solaris-userland. The original community
source was downloaded from ['https://www.openfabrics.org/down-
loads/ibutils/ibutils-1.5.7-0.2.gbd7e502.tar.gz', 'https://www.openfab-
rics.org/downloads/libibverbs/libibverbs-1.1.8.tar.gz',
'https://www.openfabrics.org/downloads/libmlx4/libmlx4-1.0.6.tar.gz',
'https://www.openfabrics.org/downloads/libsdp/lib-
sdp-1.1.108-0.15.gd7fdb72.tar.gz', 'https://www.openfabrics.org/down-
loads/management/infiniband-diags-1.6.5.tar.gz', 'https://www.openfab-
rics.org/downloads/management/libibmad-1.3.12.tar.gz',
'https://www.openfabrics.org/downloads/management/libibu-
mad-1.3.10.2.tar.gz', 'https://www.openfabrics.org/downloads/manage-
ment/opensm-3.3.19.tar.gz', 'https://www.openfabrics.org/down-
loads/perftest/perftest-1.3.0-0.42.gf350d3d.tar.gz', 'https://www.open-
fabrics.org/downloads/qperf/qperf-0.4.9.tar.gz', 'https://www.openfab-
rics.org/downloads/rdmacm/librdmacm-1.0.21.tar.gz', 'https://www.open-
fabrics.org/downloads/rds-tools/rds-tools-2.0.4.tar.gz'].
Further information about this software can be found on the open source
community website at http://www.openfabrics.org/.
May 15, 2007 RDS-STRESS(1)