This chapter provides instructions for using the Coherence datagram test utility to check network performance between two or more computers. Any production deployment should be preceded by a successful run of the datagram test.
The following sections are included in this chapter:
The Coherence datagram test utility is used to test and tune network performance between two or more computers. There are two types of tests: a point-to-point test that tests the performance of a pair of servers to ensure they are properly configured, and a distributed datagram test to ensure the network itself is functioning properly. Both tests need to be run successfully.
The datagram test operates in one of three modes: either as a packet publisher, a packet listener, or both. When the utility is run, a publisher transmits UDP packets to the listener who then measures the throughput, success rate, and other statistics. Tune an environment based on the results of these tests to achieve maximum performance. See Chapter 5, "Performance Tuning" for more information.
The datagram test utility is run from the command line using either the com.tangosol.net.DatagramTest class or by running the datagram-test script that is provided in the COHERENCE_HOME/bin directory. A script is provided for both Windows and UNIX-based platforms.
The following example demonstrates using the DatagramTest class:
java -server com.tangosol.net.DatagramTest <command value> <command value> ...
The following example demonstrates using the script:
datagram-test <command value> <command value> ...
Table 3-1 describes the available command line options for the datagram test utility.
Table 3-1 Command Line Options for the Datagram Test Utility
| Command | Required/ Optional |
Applicability | Description | Default |
|---|---|---|---|---|
|
-local |
Optional |
Both |
The local address to bind to, specified as |
localhost:9999 |
|
-packetSize |
Optional |
Both |
The size of packet to work with, specified in bytes. |
1468 |
|
-payload |
Optional |
Both |
The amount of data to include in each packet. Use |
0 |
|
-processBytes |
Optional |
Both |
The number of bytes (in multiples of 4) of each packet to process. |
4 |
|
-rxBufferSize |
Optional |
Listener |
The size of the receive buffer, specified in packets. |
1428 |
|
-rxTimeoutMs |
Optional |
Listener |
The duration of inactivity before a connection is closed. |
1000 |
|
-txBufferSize |
Optional |
Publisher |
The size of the transmit buffer, specified in packets. |
16 |
|
-txRate |
Optional |
Publisher |
The rate at which to transmit data, specified in megabytes. |
unlimited |
|
-txIterations |
Optional |
Publisher |
Specifies the number of packets to publish before exiting. |
unlimited |
|
-txDurationMs |
Optional |
Publisher |
Specifies how long to publish before exiting. |
unlimited |
|
-reportInterval |
Optional |
Both |
The interval at which to output a report, specified in packets. |
100000 |
|
-tickInterval |
Optional |
Both |
The interval at which to output tick marks. |
1000 |
|
-log |
Optional |
Listener |
The name of a file to save a tabular report of measured performance. |
none |
|
-logInterval |
Optional |
Listener |
The interval at which to output a measurement to the log. |
100000 |
|
-polite |
Optional |
Publisher |
Switch indicating if the publisher should wait for the listener to be contacted before publishing. |
off |
|
-provider |
Optional |
Both |
The socket provider to use ( |
system |
|
arguments |
Optional |
Publisher |
Space separated list of addresses to publish to, specified as |
none |
This section includes instructions for running a point-to-point datagram test and a distributed datagram test. Both tests must be run successfully and show no significant performance issues or packet loss. For details about interpreting test statistics, see "Understanding Report Statistics".
The example in this section demonstrates how to test network performance between two servers— Server A with IP address 195.0.0.1 and Server B with IP address 195.0.0.2. One server acts as a packet publisher and the other as a packet listener. The publisher transmits packets as fast as possible and the listener measures and reports performance statistics.
First, start the listener on Server A. For example:
datagram-test.sh
After pressing ENTER, the utility displays that it is ready to receive packets. Example 3-1 illustrates sample output.
Example 3-1 Output from Starting a Listener
starting listener: at /195.0.0.1:9999
packet size: 1468 bytes
buffer size: 1428 packets
report on: 100000 packets, 139 MBs
process: 4 bytes/packet
log: null
log on: 139 MBs
The test, by default, tries to allocate a network receive buffer large enough to hold 1428 packets, or about 2 MB. The utility reports an error and exits if it cannot allocate this buffer. Either decrease the requested buffer size using the -rxBufferSize parameter, or increase the operating system's network buffer settings. Increase the operating system buffers for the best performance. See Chapter 6, "Production Checklist" for details on tuning an operating system for Coherence.
Start the publisher on Server B and direct it to publish to Server A. For example:
datagram-test.sh servera
After pressing ENTER, the test instance on Server B starts both a listener and a publisher. However, the listener is not used in this configuration. Example 3-2 demonstrates the sample output that displays in the Server B command window.
Example 3-2 Datagram Test—Starting a Listener and a Publisher on a Server
starting listener: at /195.0.0.2:9999
packet size: 1468 bytes
buffer size: 1428 packets
report on: 100000 packets, 139 MBs
process: 4 bytes/packet
log: null
log on: 139 MBs
starting publisher: at /195.0.0.2:9999 sending to servera/195.0.0.1:9999
packet size: 1468 bytes
buffer size: 16 packets
report on: 100000 packets, 139 MBs
process: 4 bytes/packet
peers: 1
rate: no limit
no packet burst limit
oooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooO
The series of o and O marks appear as data is (O)utput on the network. Each o represents 1000 packets, with O indicators at every 10,000 packets.
On Server A, a corresponding set of i and I marks, representing network (I)nput. This indicates that the two test instances are communicating.
The point-to-point test can also be run in bidirectional mode where servers act as publishers and listeners. Use the same test instances that were used in the point-to-point test and supply the instance on Server A with the address for Server B. For example on Server A run:
datagram-test.sh -polite serverb
The -polite parameter instructs this test instance to not start publishing until it starts to receive data. Run the same command as before on Server B.
datagram-test.sh servera
A distributed test is used to test performance with more than two computers. For example, setup two publishers to target a single listener. This style of testing is far more realistic then simple one-to-one testing and may identify network bottlenecks that may not otherwise be apparent.
The following example runs the datagram test among 4 computers:
On Server A:
datagramtest.sh -txRate 100 -polite serverb serverc serverd
On Server B:
datagramtest.sh -txRate 100 -polite servera serverc serverd
On Server C:
datagramtest.sh -txRate 100 -polite servera serverb serverd
On Server D:
datagramtest.sh -txRate 100 servera serverb serverc
This test sequence causes all nodes to send a total of 100MB per second to all other nodes (that is, 33MB/node/second). On a fully switched 1GbE network this should be achievable without packet loss.
To simplify the execution of the test, all nodes can be started with an identical target list, they obviously transmit to themselves as well, but this loopback data can easily be factored out. It is important to start all but the last node using the -polite switch, as this causes all other nodes to delay testing until the final node is started.
Each side of the test (publisher and listener) periodically report performance statistics. The publisher simply reports the rate at which it is publishing data on the network. For example:
Tx summary 1 peers:
life: 97 MB/sec, 69642 packets/sec
now: 98 MB/sec, 69735 packets/sec
The report includes both the current transmit rate (since last report) and the lifetime transmit rate.
Table 3-2 describes the statistics that can be reported by the listener.
| Element | Description |
|---|---|
|
Elapsed |
The time interval that the report covers. |
|
Packet size |
The received packet size. |
|
Throughput |
The rate at which packets are being received. |
|
Received |
The number of packets received. |
|
Missing |
The number of packets which were detected as lost. |
|
Success rate |
The percentage of received packets out of the total packets sent. |
|
Out of order |
The number of packets which arrived out of order. |
|
Average offset |
An indicator of how out of order packets are. |
As with the publisher, both current and lifetime statistics are reported. The following example demonstrates a typical listener report:
Lifetime:
Rx from publisher: /195.0.0.2:9999
elapsed: 8770ms
packet size: 1468
throughput: 96 MB/sec
68415 packets/sec
received: 600000 of 611400
missing: 11400
success rate: 0.9813543
out of order: 2
avg offset: 1
Now:
Rx from publisher: /195.0.0.2:9999
elapsed: 1431ms
packet size: 1468
throughput: 98 MB/sec
69881 packets/sec
received: 100000 of 100000
missing: 0
success rate: 1.0
out of order: 0
avg offset: 0
The primary items of interest are the throughput and success rate. The goal is to find the highest throughput while maintaining a success rate as close to 1.0 as possible. A rate of around 10 MB/second should be achieved on a 100 Mb network setup. A rate of around 100 MB/second should be achieved on a 1 Gb network setup. Achieving these rates require some throttle tuning. If the network cannot achieve these rates or if the rates are considerably less, then it is very possible that there are network configuration issues. For details on tuning network performance, see "Network Tuning".
The publishing side of the test may be throttled to a specific data rate expressed in megabytes per second by including the -txRate M parameter, when M represents the maximum MB/second the test should put on the network.