15 Codecache Tuning

Introduction

The Java Virtual Machine (JVM) generates native code and stores it in a memory area called the codecache. The JVM generates native code for a variety of reasons, including for the dynamically generated interpreter loop, Java Native Interface (JNI) stubs, and for Java methods that are compiled into native code by the just-in-time (JIT) compiler. The JIT is by far the biggest user of the codecache. This appendix describes techniques for reducing the JIT compiler's codecache usage while still maintaining good performance.

This chapter describes three ways to reduce the JIT's use of the codecache:

• Constrain the amount of codecache available to the JIT.

• Tune the JIT to compile fewer methods.

• Tune the JIT to generate less code per method.

java Launcher Codecache Option Summary

The JVM options passed by the java launcher listed in the tables in this section can be used to reduce the amount of codecache used by the JIT. The table descriptions are summaries. Most of the options are described in more detail in the sections that follow.

Measuring Codecache Usage

To measure the success of a codecache usage reduction effort, you must measure the codecache usage and the effect on performance. This section explains how to measure the codecache usage. It is up to you to decide the best way to measure performance for your application.

Start with a baseline (the amount of codecache used when no codecache reduction techniques are applied), and then monitor the effect of your codecache reduction techniques on both codecache size and performance relative to the baseline.

Keep in mind that the codecache starts relatively small and then grows as needed as new methods are compiled. Sometimes compiled methods are freed from the codecache, especially when the maximum size of the codecache is constrained. The memory used by free methods can be reused for newly compiled methods, allowing additional methods to be compiled without growing the codecache further.

You can get information on codecache usage by specifying –XX:+PrintCodeCache on the java launcher command line. When your application exits, you will see output similar to the following:

CodeCache: size=32768Kb used=542Kb max_used=542Kb free=32226Kb
 bounds [0xb414a000, 0xb41d2000, 0xb614a000] 
 total_blobs=131 nmethods=5 adapters=63 
 compilation: enabled

The most useful part of the output for codecache reduction efforts is the first line. The following describes each of the values printed:

• size: The maximum size of the codecache. It should be equivalent to what was specified by –XX:ReservedCodeCacheSize. Note that this is not the actual amount of physical memory (RAM) used by the codecache. This is just the amount of virtual address space set aside for it.

• used: The amount of codecache memory actually in use. This is usually the amount of RAM the codecache occupies. However, due to fragmentation and the intermixing of free and allocated blocks of memory within the codecache, it is possible that the codecache occupies more RAM than is indicated by this value, because blocks that were used then freed are likely still in RAM.

• max_used: This is the high water mark for codecache usage; the maximum size that the codecache has grown to and used. This generally is considered to be the amount of RAM occupied by the codecache, and will include any free memory in the codecache that was at some point in use. For this reason, it is the number you will likely want to use when determining how much codecache your application is using.

• free: This is size minus used.

The –XX:+PrintCodeCacheOnCompilation option also produces the same output as the first line above produced by –XX:+PrintCodeCache, but does so each time a method is compiled. It can be useful for measuring applications that do not terminate. It can also be useful if you are interested in the codecache usage at a certain point in the application's execution, such as after application startup has completed.

Because max_used generally represents the amount of RAM used by the codecache, this is the value you will want note when you take your baseline measurement. The sections that follow describe how you can reduce max_used.

Constraining the Codecache Size

Constraining the codecache size means the codecache is limited to a size that is less than what would an unconstrained codecache would use. The ReservedCodeCacheSize option determines the maximum size of the codecache. It defaults to a minimum of 32MB for the client JVM and 48MB for the server VM. For most Java applications, this size is so large that the application will never fill the entire codecache. Thus the codecache is viewed as being unconstrained, meaning the JIT will continue to compile any code that it thinks should be compiled.

max_used = 3M
CodeCacheMinimumFreeSpace = 500k

-XX:ReservedCodeCacheSize=2100k
-XX:CodeCacheMinimumFreeSpace=100k

Reducing Compilations

Reducing the number of compiled methods, or the rate at which they are compiled, is another effective way of reducing the amount of codecache that is used. Two main command line options that affect how aggressively methods are compiled: CompileThreshold and OnStackReplacePercentage. CompileThreshold relates to the number of method invocations needed before the method is compiled. OnStackReplacePercentage relates to the number of backwards branches taken in a method before it gets compiled, and is specified as a percentage of CompileThreshold. When a method's combined number of backwards branches and invocations reaches or exceeds CompileThreshold * OnStackReplacePercentage / 100, the method is compiled. Note that there is also an option called BackEdgeThreshold, but it currently does nothing. Use OnStackReplacePercentage instead.

Larger values for these options decreases compilations. Setting the options larger than their defaults defers when a method gets compiled (or recompiled), possibly even preventing a method from ever getting compiled. Usually, setting these options to larger values also reduces performance (methods are interpreted), so it is important to monitor both performance and codecache usage when you adjust them. For the client JVM, tripling the default values of these options is a good starting point. For the server JVM, CompileThreshold is already set fairly high, so probably does not need to be adjusted further.

Reducing Compiled Method Sizes

There are a number of command-line options that reduce the size of compiled methods, but generally at some performance cost. Like the codecache reduction methods described in other sections, the key is finding a setting that gives good code cache usage reduction, without much performance loss.

The options described in this section all relate to reducing the amount of inlining the compiler does. Inlining is when the compiler includes the code for a called method into the compiled code for the method being compiled. Inlining can be done many levels deep, so if method a() calls b() which in turn calls c(), then when compiling a(), b() can be inlined in a(), which in turn can trigger the inlining of c() into a().

The JIT compiler uses multiple heuristics to determine if a method should be inlined. In general, the heuristics are tuned for optimal performance. However, you can adjust some of them to sacrifice some performance for less codecache usage. The more useful options to tune are described below:

InlineSmallCode

The value of this option determines how small an already compiled method must be for it to be inlined when called from a method being compiled. If the compiled version of this method is bigger than the setting for InlineSmallCode, then it is not inlined. Instead, a call to the compiled version of the method is generated.

MaxInlineLevel

This option represents the maximum nesting level of the inlining call chain. Inlining becomes much less useful at deeper levels, and can eventually be harmful to performance due to code bloat. The default value is 9. Setting MaxInlineLevel as low as 1 or 2 ensures that trivial methods, such as getters and setters, are inlined.

MaxInlineSize

This option represents the maximum bytecode size of an inlined method. It defaults to 35, but is automatically reduced at each inlining level. Setting it very small (around 6) ensures that only trivial methods are inlined.

MinInliningThreshold

The interpreter tracks invocation counts at method call sites. This count is used by the JIT to help determine if a called method should be inlined. If a method's number of invocations is less than MinInliningThreshold, the method is not inlined. Raising this threshold reduces the number of method call sites that are inlined. Trivial methods are always inlined and are not subject to the setting of MinInliningThreshold.

InlineSynchronizedMethods

This option can be used to disable the inlining of methods that are declared as synchronized. Because synchronizing is a fairly expensive operation, especially on multi-core devices, the benefits of inlining even small synchronized methods is greatly diminished. You might find you can disable the inlining of synchronized methods with little or no perceived performance degradation, but with a noticeable reduction in the codecache usage.