The required settings (shown in order) are as follows.
<host> <port> <log> <num connections> <num iterations>
Their usage is as follows.
|
Setting |
Description |
|---|---|
|
|
Target host for requests. |
|
|
Port on which the target host is listening for requests. |
|
|
Log file of the query portion of the MDEX Engine URLs and optional associated information (that is, the portion that resides in the last three columns of the MDEX Engine request log). This log file is used for HTTP request generation. URLs
and associated information from the
Each line of the
If a dash (-) is found in an optional column, the column is ignored. |
|
|
Maximum number of outstanding requests
to allow before waiting for replies. In other words, the number of simultaneous
HTTP connection streams to keep open at all times. This number emulates
multiple clients for the target server. For example, using
|
|
|
Number of times to replay the URL query log. All outstanding requests are processed before a new iteration is started. |
You can run Eneperf locally or from a remote machine.
Running Eneperf locally. Eneperf is lightweight and has a very slight impact on performance. It can usually be run on the same machine as the Dgraph being tested with no impact on results.
To run Eneperf on the same machine as the Dgraph, you point it to
localhostand<port>. This configuration is useful for isolating MDEX Engine performance from any potential networking issues.Running Eneperf on a remote host. Eneperf can also be run from a remote host. Using Eneperf to test the same MDEX Engine from the local machine and from across the network can expose networking problems if the throughputs are significantly different.
Note
Eneperf can be run on a machine with a different architecture than one you are testing.
MDEX Engine request logs can be used as Eneperf input with some modifications.
URLs in the log should not include any machine connection parameters such as protocol, host, or port. These are added automatically. For example, a log entry of the following form is valid:
/graph?node=0
But a log entry of the following form is not valid:
http://myhost:5555/graph?node=0
You can achieve higher concurrent load by using a single large request log file (which might simply be repeated concatenations of a smaller log file) than by using multiple iterations of a small log file. The log file should preferably be at least 100 lines, even if it consists of the same query repeated over and over. Because Eneperf drains all connections between each iteration, running a one-line log file through Eneperf 100 times results in skewed throughput statistics.
If you are planning to measure performance of partial updates with Eneperf, (as opposed to measuring performance of regular queries), create a separate updates log based on your existing request log.
That is, suppose your MDEX Engine request log contains both regular
queries and updates operations. Then your updates log should contain only
config?op=update operations. You can create this updates
log manually, by extracting these operations from a regular log. You can then
run Eneperf against the updates log and the regular log, to measure the
performance of your updates, by using the
--updates-log and the
--log settings together.
Note
This is only one way to measure performance of updates and should only be used in cases when you care about the time between the updates. (If you do not care about the timing between updates, you can use the regular log for your testing.)
Eneperf load is driven by the
num connections setting, which indicates the number of
simultaneous connections Eneperf tries to maintain at a time.
For
example, if
num connections is set to 4, it sends four
requests to the MDEX Engine. When one returns, another is sent out to replace
it.
To adequately measure performance of the MDEX Engine, you need to identify the number of connections for Eneperf that saturates the MDEX Engine thread pool.
The number of connections needed to saturate the MDEX Engine depends on the MDEX Engine threading configuration and the server characteristics, and generally correlates with the number of the MDEX Engine threads in use, (assuming the MDEX Engine is configured with enough threads). However, an MDEX Engine with four threads might be saturated by only three connections if the queries are complex and all CPUs are being fully utilized.
To identify an appropriate setting for
num connections, Oracle recommends running tests with
the following settings:
For debugging, run a test with
num connectionsset to one. This test sends only one request to the MDEX Engine at a time. Each query is processed alone; no other query computations are contending for the machine’s resources. This test generates an MDEX Engine request log showing the canonical time for each query. You can examine the request log to identify slow queries without the concern that they happened to be slow because other queries were processed simultaneously. Note that using a log file with just one entry limits num connections to one.For stress testing, run a test with
num connectionsset to the number of threads for the MDEX Engine. In this test, no requests are waiting in the queue. This lets you obtain an estimate of the maximum expected MDEX Engine performance. Because no queuing occurs, this test offers a conservative bias for throughput.In addition, you can run a test with
num connectionsset to the “number of threads + one”. In this test case, a minimal waiting in the queue for the MDEX Engine request may occur. This also lets you obtain an estimate of the maximum expected MDEX Engine performance. Because queuing does not occur, this test offers an aggressive bias for throughput.Do not use a small log with a large number of
num connections. Also, do not run a small log many times to simulate a large log.
Commonly, you will wish to perform the load testing of the MDEX Engine to a level below saturation. Use the following examples to help you select an appropriate number of connections for Eneperf that will saturate MDEX Engine performance to the desired levels.
Typically, front-end applications have different requirements for response times and peak loads. Such as:
An application that is used steadily across the year. For applications of this type, MDEX Engine performance must support average query response time under average loads. Occasional slowdowns under peak load are acceptable. Therefore, you need to measure average response time under average load.
An application that is used during the peak seasons. For applications of this type, MDEX Engine performance must support peak response time under peak loads. It is acceptable for this application to have extra performance capacity during non-peak seasons.
To identify the projected throughput for the MDEX Engine, use the following formulas.
These formulas represent a highly simplified approach to calculating throughput. Although you can use more thorough methods, these formulas provide reasonable estimates, and can be used for initial assessment:
concurrent users / (expected page latency + think time) = page views/sec page views / second x MDEX queries/page = ops/second for the MDEX Engine
Where:
The number of concurrent users is the estimated number of users currently logged in to the application
The number of simultaneous requests is the number of users currently making a request to the application. Typically, it is 20-30% of the number of concurrent users.
Peak load is the expected maximum number of simultaneous requests, such as during a specific time period
Think time is the time between requests issued by a single user. It is used to calculate simultaneous requests based on the estimated number of concurrent users.
For example, 100 concurrent users with a 5 second think time and a 1 second expected page latency will yield 17 pages/sec. 17 pages/second with 2 MDEX Engine queries per page will yield 34 ops/sec for the expected performance of the MDEX Engine. This means that to support 100 concurrent users in this application, the MDEX Engine must perform at 34 ops/sec.
In another example, if your implementation includes a load balancer serving four application servers, and two MDEX Engines with another load balancer, the following calculations provide you with the estimated performance for each of the MDEX Engines:
600 concurrent users are distributed across 4 application servers. This means 150 users per server.
150 users divided by 5 (4 sec think time and 1 sec expected page latency) yields 30 simultaneous page views per server.
30 page views with 2 MDEX Engine queries per page yield 60 MDEX Engine queries per server.
60 queries per server multiplied by 4 application servers yield 240 queries total.
240 queries are sent to the load balancer that distributes them across two MDEX Engines. Each MDEX Engine serves 120 queries.
This means that to support 100 concurrent users in this application, each MDEX Engine must perform at 120 ops/sec.
To summarize, you can use these recommendations to identify the number of connections (equal to the number of simultaneous requests in these examples) that you need to provide to Eneperf to achieve the desired MDEX Engine performance.
