Diagnostics Guide

Part I Understanding the Oracle JRockit JDK

About the Oracle JRockit JDK

What is the JRockit JVM?

About the JDK

JRockit JDK Versions

What Platforms Does the JRockit JDK Support?

Compatibility Information

The Contents of a JRockit JDK Installation

Development Tools

Demo

C Header Files

Java Runtime Environment (JRE)

Java Virtual Machine

Standard Java SE JRE Features

Note on JRE Class Files

Additional Libraries

Sample

Attach API Support

Oracle JRockit Documentation

JRockit JVM Command Line Options

JRockit JDK and JRockit Mission Control Support

Understanding JIT Compilation and Optimizations

More than a “Black Box”

How the JRockit JVM Compiles Code

1. The JRockit JVM Runs JIT Compilation

2. The JRockit JVM Monitors Threads

3. The JRockit JVM Runs Optimization

An Example Illustrating Some Code Optimizations

Steps Taken to Optimize Class A

Understanding Memory Management

The Heap and the Nursery

Object Allocation

Garbage Collection

The Mark and Sweep Model

Mostly Concurrent Mark and Sweep

Parallel Mark and Sweep

Generational Garbage Collection

Dynamic and Static Garbage Collection Modes

Compaction

External and Internal Compaction

Sliding Window Schemes

Compaction Area Sizing

Understanding Threads and Locks

Understanding Threads

Default Stack Size for Java Threads

Default Stack Size for JVM Internal Threads

Understanding Locks

Spinning and Sleeping

Lock Chains

Lock Chain Types

Open Chains

Deadlock Chains

Blocked Chains

Migrating Applications to the Oracle JRockit JDK

About Application Migration

Why Migrate?

Migration Restrictions

Migration Support

Migration Procedures

Environment Changes

Other Tips

Tuning the JRockit JVM for Your Application

Testing the Application

Why Test?

How to Test

Replicating Tools Supplied with the Sun JDK

Command-line Option Compatibility Between the JRockit JVM and Sun

Submitting Migration Tips

Setting Up and Running the Oracle JRockit JDK

Installing the Oracle JRockit JDK

Setting Up and Checking Your Linux Environment

Linux on IA32

Using LD_ASSUME_KERNEL

Running in a chroot(3) Environment

Setting Up and Checking Your Windows Environment

Setting Up and Checking Your Sun Solaris Environment

Setting the Path to the License File

Part II Profiling and Performance Tuning

About Profiling and Performance Tuning

How to Tune: An Overview

What this Section Contains

Understanding the Tuning Trade-offs

Pause Times vs. Throughput

Concurrent vs. “Stop-the-World”

Compaction Pauses vs. Throughput

Performance vs. Memory Footprint

Heap Size vs. Throughput

Book Keeping vs. Pause Times

First Steps for Tuning the Oracle JRockit JVM

Step 1: Basic Tuning

Tuning the Heap Size

Tuning the Garbage Collection

Tuning the Nursery Size

Tuning the Pause Target

Step 2: Performance Tuning

Lazy Unlocking

Call Profiling

Large Pages

Step 3: Advanced Tuning

Tuning Compaction

Compaction Ratio

Compact Set Limit

Tuning the TLA size

Further Information

Best Practices

Oracle WebLogic Server

Oracle WebLogic SIP Server

Oracle WebLogic Event Server

Oracle Workshop

“Utility” Applications

“Batch” Runs

Tuning the Memory Management System

Setting the Heap and Nursery Size

Setting the Heap Size

Setting the Heap Size on 64-bit Systems

Setting the Nursery and Keep Area Size

Keep Area

Selecting and Tuning a Garbage Collector

Selecting a Dynamic Garbage Collection Mode

Throughput Mode

Pausetime Mode

Setting a Pause Target for Pausetime Mode

Deterministic Mode

Special Note for WLRT Users

Setting a Pause Target for Deterministic Mode

Selecting a Static Garbage Collection Strategy

Garbage Collector Strategy Selection Workflow

Changing Garbage Collection Strategy During Runtime

Tuning the Concurrent Garbage Collection Trigger

Tuning the Compaction of Memory

Fragmentation vs. Garbage Collection Pauses

Adjusting Compaction

Setting the Compaction Ratio

Setting the Compact Set Limit

Turning Off Compaction

Using Full Compaction

Optimizing Memory Allocation Performance

Setting the Thread Local Area Size

Tuning Locks

Lock Profiling

Disabling Spinning Against Fat Locks

Adaptive Spinning Against Fat Locks

Lock Deflation

Lazy Unlocking

Tuning For Low Latencies

Measuring Latencies

Tune the Garbage Collection

Dynamic Garbage Collection Mode Optimized for Deterministic Pauses

Dynamic Garbage Collection Mode Optimized for Short Pauses

Static Generational Concurrent Garbage Collection

Tune the Heap Size

Manually Tune the Nursery Size

Manually Tune Compaction

Tune When to Trigger a Garbage Collection

Tuning For Better Application Throughput

Measuring Your Application’s Throughput

Select Garbage Collector

Dynamic Garbage Collection Mode Optimized for Throughput

Static Single-Spaced Parallel Garbage Collection

Static Generational Parallel Garbage Collection

Tune the Heap Size

Manually Tune the Nursery Size

Manually Tune Compaction

Tune the Thread-Local Area Size

Tuning For Stable Performance

Measuring the Performance Variance

Tune the Heap Size

Manually Tune the Nursery Size

Tune the Garbage Collector

Tune Compaction

Tuning For a Small Memory Footprint

Measuring the Memory Footprint

Set the Heap Size

Select a Garbage Collector

Tune Compaction

Tune Object Allocation

Tuning For Faster JVM Startup

Measuring the Startup Time

Setting the Heap Size

Troubleshoot Your Application and the JVM

Part III JRockit JDK Tools

Introduction to Diagnostics Tools

What this Section Contains

Using Oracle JRockit Mission Control Tools

JRockit Mission Control Overhead

Architectural Overview of the JRockit Mission Control Client

JRockit Mission Control 3.0

JRockit Mission Control 2.0

JRockit Mission Control 1.0

The JRockit Browser (JRockit Mission Control 2.0 and later)

The JRockit Management Console

The JRockit Runtime Analyzer

Latency Analysis Tool (JRockit Mission Control 3.0)

JRA Sample Recordings

Opening a Sample Recording

To open a sample recording from within the JRockit Mission Control Client

To open a sample recording from the file system

The JRockit Memory Leak Detector

More Information on JRockit Mission Control Versions

Understanding Verbose Outputs

Memory Management Verbose Log Modules

Verbose Memory Module

Initial Verbose Memory Outputs

Verbose Memory Garbage Collection Outputs

Verbose Memory Page Faults Warning

Verbose Nursery Log Module

Verbose Nursery Young Collection Output

Verbose Nursery Size Adjustment Output

Verbose Memdbg Log Module

Initial Verbose Memdbg Output

Verbose Memdbg Parallel Old Collection Output

Verbose Memdbg Concurrent Old Collection Output

Verbose Memdbg Young Collection Output

Aborted Compactions

Parallel Sweep in Concurrent Old Collections

Verbose Compaction Log Module

Aborted Compaction Verbose Output

Verbose Gcpause Log Module

Verbose Gcpause Parallel Old Collection Output

Verbose Gcpause Concurrent Old Collection Output

Verbose Gcpause Young Collection Output

Verbose Gcreport Log Module

Verbose Refobj and Referents Log Modules

Verbose Refobj Output on Info Level

Verbose Referents Output and Verbose Refobj on Debug Level

Other Verbose Log Modules

Verbose Opt Log Module

Verbose Exceptions Log Module

Using Thread Dumps

Creating Thread Dumps

Reading Thread Dumps

The Beginning of The Thread Dump

Stack Trace for Main Application Thread

Locks and Lock Chains

Presentation of Locks Out of Order

JVM Internal Threads

Other Java Application Threads

Lock Chains

Thread Status in Thread Dumps

Life States

Run States

Special States

Troubleshooting with Thread Dumps

Detecting Deadlocks

Detecting Processing Bottlenecks

Viewing The Runtime Profile of an Application

Running Diagnostic Commands

Diagnostic Commands Overview

Using jrcmd

How jrcmd Communicates with the JRockit JVM

How to Use jrcmd

jrcmd Examples

Listing JRockit JVM Processes

Sending a Command to a Process

Sending Several Commands

Known Limitations of jrcmd

Ctrl-Break Handler

Available Diagnostic Commands

Getting Help

Oracle JRockit Time Zone Updater

Downloading the TZUpdater

Introduction to the TZUpdater

System Requirements to Run the TZUpdater

Using the TZUpdater

Command-line Options Described

Example of the Default way of Using TZUpdater

Error Handling

System-wide Usage

Determining Your TZUpdater Version

Removing TZUpdater Changes

Known Issues

Oracle JRockit Mission Control Use Cases

Analyzing System Behavior with the JRockit Management Console

Getting Started

Analyzing Memory Usage

Plotting Garbage Collection Times

Setting an Alert Trigger

Profiling Methods Online by Using the Console

Analyzing System Problems with the JRockit Runtime Analyzer

Getting Started

Creating the Recording

Looking at the Recording

Examining the Methods Tab

Examining the Top Hot Methods

Examine Garbage Collection Events

Examine the GC General Tab

Compare Object Statistics

Examine Lock Profiling Information

Detecting a Memory Leak

Getting Started

Analyze the Java Application

The Leak is Discovered

Part IV Diagnostics and Troubleshooting

About Diagnostics and Troubleshooting

What this Section Contains

Diagnostics Roadmap

Step 1. Eliminate Common Causes

Step 2. Observe the Symptoms

Step 3. Identify the Problem

Step 4. Resolve the Problem

Step 5. Send a Trouble Report (Optional)

The Oracle JRockit JVM Starts Slowly

Possible Causes Behind a Slow Start

Special Note If You Recently Switched JVMs to the JRockit JVM

Diagnosing a Slow JVM Startup

Diagnosing a Slow Application Startup

Timing with nanoTime() and currentTimeMillis()

System.nanoTime()

System.currentTimeMillis()

Milliseconds and nanotime at application startup

Recommended Solutions for a Slow Start

Tune for Faster Startup

Eliminate Optimization Problems

Eliminate Application Problems

Open a Case with Oracle Support

Long Latencies

The Problem is Usually with Tuning

Troubleshooting Tips

GC Trigger Value Keeps Increasing

GC Reason for Old Collections is Failed Allocations

Long Young Collection Pause Times

Long Pauses in Deterministic Mode

If All Else Fails, Open a Case With Oracle Support

Low Overall Throughput

The Problem is Usually with Tuning

If All Else Fails, Open a Case With Oracle Support

The Oracle JRockit JVM’s Performance Degrades Over Time

The Problem is Usually With Tuning

You Could be Experiencing Optimization Problems

You Could Be Experiencing a Memory Leak in Java

If All Else Fails, Open a Case with Oracle Support

The System is Crashing

Notifying Oracle Support

Classify the Crash

Using a Crash File

Determine the Crash Type

Out Of Virtual Memory Crash

Verify the Out Of Virtual Memory Error

Virtual Memory Maximums

Checking the Text dump File

Checking the Binary mdmp or core File

Troubleshoot the Out Of Virtual Memory Error

Upgrade to the Latest JRockit JVM Version Available

Reduce the Java Heap Size

Use the Windows /3GB Startup Option

Check for Memory Leaks in JNI Code

Record Virtual Memory Usage

Windows

Linux

If All Else Fails, Open a Case with Oracle Support

Stack Overflow Crash

Verify the Stack Overflow Crash

Troubleshoot a Stack Overflow Crash

Application Level Changes

Increase the Default Stack Size

Make the JRockit JVM More Robust Against Stack Overflow Errors

Unsupported Linux Configuration Crash

Verify that the OS Version is Supported

Verify that You Have Installed the Correct glibc Binary

Examine the Thread Library

JVM Crash

Code Generation Crash

Identify a Code Generation Crash

Troubleshoot the Code Generation Crash

Exclude the Offending Method

Creating and Using an Optfile

Verifying optfile Response

Setting and optfile with a Ctrl-Break Handler

Rules for Directives

The Problem Might Lie With an External Instrumentation Tool

If All Else Fails, Open a Support Case

Garbage Collection Crash

Identify a Garbage Collection Crash

Consider Upgrading to the Latest Version of the JRockit JVM

Try One of These Workarounds

Change the Garbage Collector

Disable Compaction

Disable Inlining

Use the Optimizing Compiler

If All Else Fails, Open a Case with Oracle Support

Understanding Crash Files

Differences Between Text dump Files and Binary core/mdmp Files

Binary Crash File Sizing

Location of Crash Files

Enabling Binary core Crash Files on Linux and Sun Solaris

Enabling Binary mdmp Crash Files on Windows

Disabling Crash Files

Disabling Text dump Files

Disabling the Binary Crash Files

Extracting Information From a Text dump File

Symptoms to Look For

Example of a Text dump File

The Beginning of the Text dump File

JRockit JVM Dump (Crash File) Produced

Support Case Information

Error Message from OS

JRockit JVM Version and Garbage Collector Information

CPU and Memory Information

Operating System Version Information

Thread and State Information

Command Line and Environment Information

Command Line Option Information

JAVA_HOME and _JAVA_OPTIONS

LD_LIBRARY_PATH

LD_ASSUME_KERNEL, C Heap, StackOverFlow, and OutOfMemory Information

Registers and Stack Information

Registers

Stack Information

The Oracle JRockit JVM is Freezing

Diagnosing Where the Freeze is Occurring

Java Application Freeze

Resolving a Java Application freeze

If This Did Not Help

JVM Freeze

Collect Information About the JVM Freeze

Collecting Information on Linux Systems

Force a Crash with the Diagnostics Command enableforce_crash

Use SIGABRT (modern Linux Systems)

Collecting Information on Windows Systems

Force a Crash with the Diagnostic Command enableforce_crash

Use windbg

Collecting State Information if the JRockit JVM is Running as a Service

Submit the Information to Oracle JRockit Support

Non-responding NFS Shares

Submitting Problems to Oracle Support

Check the Oracle JRockit JVM Forums First

Filing the Trouble Report

Trouble Reporting Process Overview

Identify Your Problem Type

Verify That You’re Running a Supported Configuration

Verify the Problem has Not Been Fixed in a Subsequent Version of the JRockit JVM

Collect Enough Information to Define Your Issue