Diagnostics Guide

Tuning For Low Latencies

Long latencies can make some applications behave poorly even though the overall throughput is good. For example, a transaction based system may seem to perform well as to the number of transactions executing during a specified amount of time, but still show some uneven behavior with transactions timing out now and then even on low loads. Latencies in the application or the environment in which the application is run may cause this uneven or poor performance. Latencies can be due to anything from contention in the Java code to slow network connections to a database server. Latencies may also be caused by the JVM, for example during garbage collection, depending on how the JVM is tuned. This section describes how to tune the Oracle JRockit JVM for low latencies, covering the following subjects:

Measuring Latencies

Most application developers have a way of measuring their application’s performance. You can for example run a set of simulated use cases and measure the time it took to execute them, the number of a specific kind of transactions executed per minute, the average transaction time or how many percent of the transaction times are above or below a specific threshold. When you tune for low latencies you will be most interested in measuring the amount of transaction times that are above a certain threshold. For best tuning results you should have a varied set of benchmarks that are as realistic as possible and run for a longer period of time. Twenty minutes is often a minimum, and sometimes the full effect of the tuning can be seen only after several hours.

When you have identified a situation where the long latencies occur, you can start monitoring the JRockit JVM using some of the following methods:

Now you have the tools to see the results of your tuning.

Tune the Garbage Collection

The first step for tuning the JRockit JVM for low latencies is to select a garbage collection mode that gives you short garbage collection pauses. The best bet is one of the following two dynamic garbage collection modes or one static garbage collection strategy, described further in this section:

For more information about different garbage collector options, see Selecting and Tuning a Garbage Collector.

Dynamic Garbage Collection Mode Optimized for Deterministic Pauses

Applications that require minimal latency, such as those used in the telecom and finance industries, cannot abide by the unpredictable pause times caused common garbage collection strategies. To avoid these overly-long pauses, the JRockit JVM provides “deterministic” garbage collection, a dynamic garbage collection mode that keeps the garbage collection pauses short and deterministic.

Set the deterministic garbage collector at the commandline as follows:

java -XgcPrio:deterministic -Xms:1g -Xmx:1g myApplication

The garbage collector will aim on keeping the garbage collection pauses below the given pause target. How well it will succeed depends on the application and the hardware. For example, a pause target on 30 ms has been verified on an application with 1 GB heap and an average of 30% live data or less at collection time, running on the following hardware:

Running on slower hardware, with a different heap size and/or with more live data might break the deterministic behavior or cause performance degradation over time, while faster hardware or less live data might allow you to set a lower pause target.

The pause target for deterministic mode is by default 30 ms, and can be changed with the command line option -XpauseTarget:<time>. For example:

java -XgcPrio:deterministic -Xms:1g -Xmx:1g -XpauseTarget:40ms MyApplication

This starts up the JVM with the garbage collection optimized for short and deterministic pauses and a pause target of 40ms.

For more information, see the documentation on -XpauseTarget.

The deterministic garbage collector optimizes the compaction for the given pause target and does not use a nursery. Further tuning of compaction and nursery size should thus be unnecessary when the deterministic garbage collector is used.

Dynamic Garbage Collection Mode Optimized for Short Pauses

The dynamic garbage collection mode optimized for short pauses is useful for applications that don’t require quite as short and deterministic pauses as the deterministic garbage collector guarantees. This garbage collection mode selects a garbage collection strategy to keep the garbage collection pauses below a given pause target (500 ms by default). Compaction will also be adjusted automatically to keep down the pause times caused by compaction.

Set the pausetime priority as follows:

java -XgcPrio:pausetime myApplication

If you use the pausetime priority but find that the default (500 ms) is too long, you can specify a target pause time by using the -XpauseTarget option, for example:

java -XgcPrio:pausetime -XpauseTarget=200ms myApplication

Be aware that there is a certain trade off between short pauses and application throughput. Shorter garbage collection pauses require more overhead in bookkeeping and may cause more fragmentation, which lowers the performance. If your application can tolerate pause times longer than 500 ms you can increase the pause target to increase the application’s performance.

The target value is used as a pause time goal and by the dynamic garbage collector to more precisely configure itself to keep pauses near the target value. Using this option allows you to specify the pause target to be between 200 ms and 5 seconds. If you don’t specify a pause target, the default remains 500 ms.

The garbage collection mode for short pauses optimizes the compaction for the given pause target, so further tuning of the compaction should not be necessary. The nursery size is adjusted automatically, but for an even performance you may need to tune the nursery size manually. In R27.3 and later releases, the nursery size is static for this garbage collection mode and will have to be tuned manually.

Static Generational Concurrent Garbage Collection

If you want to use a static garbage collector and still experience minimal pause times, use a concurrent garbage collector. Generally, using a generational garbage collector is preferable to using a single-spaced garbage collector since a generational garbage collector gives you better application throughput.

To use a generational concurrent garbage collector, enter the following at the command line:

java -Xgc:gencon myApplication

To use a single-spaced concurrent garbage collector, enter the following at the command line:

java -Xgc:singlecon myApplication

When you use a static garbage collector you may have to tune the nursery size and the compaction manually.

Tune the Heap Size

You can resize the heap by using the -Xms (initial and minimum heap size) and -Xmx (maximum heap size) command line options when you launch the JRockit JVM. Usually you can set the initial and the maximum heap size to the same value. Increasing the heap size reduces the frequency of garbage collections. A larger heap may also take slightly longer to garbage collect, but this effect is usually not considerable until the heap reaches sizes of several gigabytes.

The best approach to tuning the heap size is simply to benchmark the application with many different heap sizes. Monitor the garbage collection pauses as described in Measuring Latencies while you do this to determine the largest possible heap size for your application.

The only exception is the deterministic garbage collector. The deterministic garbage collector is verified using a heap of about 1 GB, and will work best with heaps of about this size.

To set the heap size, use the -Xms and the -Xmx options, for example

java -Xms:1g -Xmx:1g myApp

For more information, see Optimizing Memory Allocation Performance.

Manually Tune the Nursery Size

If you are running -XgcPrio:pausetime or -Xgc:gencon you might want to tune the nursery size manually.

The size of the nursery changes dynamically in runtime when you use -XgcPrio:pausetime, but setting it manually gives a more even behavior (note that when you use the dynamic garbage collector, the nursery might also be turned off completely when single-spaced garbage collection is used).

The default nursery size for -Xgc:gencon is static, and may thus not be optimal for all applications. You might benefit from manually setting a custom nursery size. The nursery should be as large as possible, but the nursery size must be decreased if the pause time created by a young collection (nursery garbage collection) is too long. Tune the nursery size by benchmarking your application with several different nursery sizes while monitoring the garbage collection pauses as described in Measuring Latencies.

To set the size of the nursery, use the -Xns option; for example:

java -XgcPrio:pausetime -Xns:64m myApp

Manually Tune Compaction

If you are using a static garbage collector, tuning compaction manually might help improve latencies. Compaction is performed during a garbage collection pause, and thus the compaction time affects the garage collection pause times. By default, the static garbage collectors use a compaction scheme that aims at keeping the compaction times fairly even, but does not put an upper bound on the compaction time.

You can limit the compaction manually by setting a static compaction area size (-XXcompactRatio) or by limiting the number of references that can be updated due to compaction (-XXcopactSetLimit). Neither action will not guarantee an upper bound on the compaction time, but will reduce the risk for long compaction times.

Be aware that if you set the compaction ratio to low, the heap slowly becomes more and more fragmented until it is impossible to find free space that is big enough for object allocation. The heap becomes full of dark matter (basically severe fragmentation). When this happens, a full compaction (a compaction of the complete heap) will be done, which can result in a pause times of up to half a minute. Dark matter is reported for the heap in a JRA recording.

For complete information on limiting compaction, please refer to Adjusting Compaction.

Tune When to Trigger a Garbage Collection

The -XXgcTrigger option determines how much free memory should remain on the heap when a concurrent garbage collection starts. If the heap becomes full during the concurrent garbage collection, the Java application can’t allocate more memory until garbage collection frees some memory, which might cause the application to pause. While the trigger value will tune itself in runtime to prevent the heap from becoming too full, this automatic tuning might take too long. Instead, you can use -XXgcTrigger option to set from the start a garbage collection trigger value more appropriate to your application.

If the heap becomes full during the concurrent mark phase, the sweep phase will revert to parallel sweep (unless -XXnoParSweep has been specified). If this happens frequently and the garbage collection trigger doesn't increase automatically to prevent this, use -XXgcTrigger to manually increase the garbage collection trigger; for example:

java -XXgcTrigger myApp

The current value of the garbage collection trigger appears in the -Xverbose:memdbg outputs whenever the trigger changes.