|
|
Long latencies may for example manifest as single transactions that time out in a transaction based application while the overall performance is good. The problem usually lies in uneven performance and non-deterministic latencies.
This chapter includes these topics:
Long latencies often indicate that your application is not tuned for short and deterministic pause times. Before engaging in time-consuming troubleshooting and mitigation tasks, you should retune the Oracle JRockit JVM to optimize for short pause times and restart the application. For detailed instructions on how to tune the JVM for short pause times, please refer to Tuning For Low Latencies.
There is a certain trade-off between low latencies and high overall application throughput. High latencies that cause transactions to time-out are often caused by garbage collection pauses. To reduce the individual garbage collection pauses the garbage collector runs in a mostly concurrent mode, where the garbage collection is, for the most part, done while the Java threads are still running. This causes some extra work for the garbage collector, which has to keep track of changes during the concurrent phases of the garbage collection. The garbage collections will also be less efficient, since objects that are allocated during the concurrent garbage collection will not be garbage collected until the next garbage collection. This can force the JVM to collect garbage more often. Also, the heap might become more fragmented when compaction is limited to reduce the pause times caused by compaction. All this reduces garbage collection pauses, but it will also have a negative impact on overall throughput.
You can increase the overall throughput while keeping the latencies low by allowing longer garbage collection pauses or by manually tuning the garbage collection using the tips in Tuning For Better Application Throughput.
This section lists some latency troubleshooting hints that apply for mostly concurrent garbage collection, for example -XgcPrio:deterministic, -XgcPrio:pausetime, -Xgc:gencon and -Xgc:singlecon.
The garbage collection trigger (gctrigger) value determines how much free heap space should be available when a concurrent garbage collection starts in order to allow the Java threads to continue allocating objects during the entire garbage collection. The gctrigger value changes in runtime in order to avoid situations where the heap becomes full during the concurrent garbage collection. Monitor the gctrigger value in -Xverbose:memdbg outputs or in JRA recordings. If the gctrigger value keeps increasing, the load on the application is to high for the concurrent garbage collector. Decrease the load on the application.
Monitor the garbage collection reasons for the old collections with -Xverbose:memdbg or JRA. The normal garbage collection reason for a mostly concurrent old collection is “heap too full”. If the old collections are frequently triggered due to failed object allocation, the GC trigger is too low. Increase the GC trigger value using the command line option -XXgcTrigger, or decrease the load on the application.
Monitor the pause times for young collections in -Xverbose:gcpause outputs or JRA recordings. If the young collection pause times are too long, decrease the nursery size using the -Xns command line option or run a single generational garbage collector.
Monitor the garbage collection pause times in a JRA recording. Check the pause parts for pause times that are too long. If the pause parts for Compaction are too long, decrease the pause target. If the pause parts in Mark:Final, especially the ones concerning RefrenceQueues, are too long you may have a problem with many java.lang.ref.Reference objects in your application. The best way to handle this would be to re-design the Java application using fewer reference objects. You could also try decreasing the heap size, which will cause reference objects to be handled more often and reduce the amount of reference objects to handle at each old collection.
If none of the tuning solutions suggested in Tuning For Low Latencies or in Troubleshooting Tips resolve the problem, you will need to open a case with Oracle Support. You can find instructions on how to report a problem to Oracle, including the sort of information to include, in Submitting Problems to Oracle Support.
|