The Oracle JRockit JVM is Freezing

When the Oracle JRockit JVM or Java application becomes unresponsive but hasn’t crashed, it is considered to be frozen. A frozen system occurs when the application stops answering requests but the process is still there. A system can freeze in either the JVM or the application. Your first action is to determine where it is freezing.

Most often, a JVM freeze requires that you contact Oracle JRockit Support for resolution. This section describes how to diagnose where a system is freezing and how to collect information that will assist the support team resolve your problem.

Diagnosing Where the Freeze is Occurring

To determine whether the freeze is occurring in the application or in the JVM, try to generate a thread dump by doing one of the following:

On all platforms you can also get the thread dump by using jrcmd; for example:

where nnnn is the PID of the Java process. To get a list of the PIDs of all Java processes running on the machine, run jrcmd without any command line parameters.

The result of the any of these procedures will indicate the kind of freeze happening:

Java Application Freeze

An application freeze occurs when something in the Java application causes the system to become unresponsive. This section describes how to handle a Java application freeze. It contains information on the following subjects:

Resolving a Java Application freeze

If This Did Not Help

If none of the suggested workarounds corrects the situation, you will need to open a case with Oracle Support, according to the process described in Submitting Problems to Oracle Support. When creating a support case for a Java application freeze, you need to include the following information:

JVM Freeze

If you cannot get thread dumps with Ctrl-Break or by sending SIGQUIT (kill -3) to the parent Java process PID after a few tries, the JVM has stopped handling signals and is really freezing. When this happens, you should open a case with Oracle Support and work with them to rectify the problem. This section describes the ways you can force the JRockit JVM to crash in order to create the necessary crash file. It contains information on the following subjects:

Collect Information About the JVM Freeze

Since resolving a JVM freeze requires that you open a case with Oracle Support, you need to collect as much information as you can about the state of the process that was running when the JVM froze. Normally, this information would be written to a crash file, but since the JRockit JVM technically hasn’t crashed, that file won’t be created unless you force the JVM to crash. This section explains how to collect the necessary state information based upon your JVM deployment.

Collecting Information on Linux Systems

To collect information for an JRockit JVM running on Linux, use one of these procedures:

Force a Crash with the Diagnostics Command enableforce_crash

On versions of the JRockit JVM later than R25, you can use the command enableforce_crash to force a crash and thus produce a crash file with the necessary state information. Add the following option to the command line and use jrcmd to force a crash:

Use SIGABRT (modern Linux Systems)

SIGABRT aborts the signal sent to the process and causes a crash, thus producing a crash file. Invoke SIGABRT as described here:

Find the PID of the JRockit JVM process by using the following procedure:

Note:

The following instructions are valid on 2.6 kernel-based Linux systems and on Red Hat Enterprise Linux 3.0.

Assuming that the JRockit JVM process runs as the Linux user webadmin, enter the following command:

pstree -p webadmin | grep java

If you get excessive garbage from pstree, try unsetting the LANG environment variable first:

unset LANG

If pstree gives you only one Java process, you're done. Go to step 2.
If pstree gives you several processes, print the command line parameters for any process by entering:

cat /proc/nnnn/cmdline | xargs --null -n1 echo

(where nnnn is the PID you are interested in; for example, 1234)

Use the command:

ls -l /proc/1234/cwd

to see where the core file (the core dump crash file) will be created (assuming that the PID is 1234).

Create the core file (and terminate the process) by entering:

ulimit -c unlimited

kill -SIGABRT 1234

(again assuming that the PID is 1234).

Collecting Information on Windows Systems

To collect information for JRockit JVM running on Windows, use one of these procedures:

Force a Crash with the Diagnostic Command enableforce_crash

On versions of JRockit JVM later than R25, you can use the diagnostic command enableforce_crash to force a crash and thus produce a crash file with the necessary state information. Add the following option to the command line and use jrcmd to force a crash:

Use windbg

_Thewindbg_{command can also create the necessary core file. Enter the command:}

Collecting State Information if the JRockit JVM is Running as a Service

If the JRockit JVM starts as a service, you can collect thread dumps by doing one of the following:

If you are collecting information from your own machine, run jrcmd with the print_threads Ctrl-Break handler. See Using jrcmd for background on jrcmd and Available Diagnostic Commands for information on using print_threads.
If you are collecting information from another machine, use Oracle JRockit Mission Control with the diagnostics bean.
If you are running the JRockit JVM with Oracle WebLogic Server, you might want to use beasvc – dump to obtain thread-dumps from the JVM. For more information, please refer to Setting Up a WebLogic Server Instance as a Windows Service.

If you want to see the messages that a server instance prints to standard out and standard error (including stack traces and thread dumps), redirect standard out and standard error to a file, please refer to Setting Up a WebLogic Server Instance as a Windows Service.
To cause the WebLogic Server instance to print a thread dump to standard out, do either of the following:

Use the weblogic.Admin THREAD_DUMP command.

Open a command prompt and enter the following command:

WL_HOME\bin\beasvc -dump -svcname:service-name

where WL_HOME is the directory in which you installed WebLogic Server and service-name is the Windows service that is running a server instance; for example.:

D:\bea\weblogic81\server\bin\beasvc -dump -svcname:mydomain_myserver

Submit the Information to Oracle JRockit Support

Open a case with Oracle Support, who will continue to troubleshoot your JVM freeze until it is resolved. Be sure to include any information pertinent to the freeze, including the all thread dumps you’ve collected. For information on communication with Oracle Support, please refer to Submitting Problems to Oracle Support.

Non-responding NFS Shares

In some cases, the freeze might be caused by a problem with the Network File System (NFS). NFS is a protocol used by Linux computers to share disks across a network. An NFS share is any disk on NFS set up to be shared by other computers. If you are using a machine that has NFS shares (look for nfs in /etc/fstab), your application might be freezing because a Java application is trying to access data on a non-responding NFS share. When that happens, the default behavior for the thread doing the file access is not to respond to any signals until the NFS server comes back up. This means that the Oracle JRockit JVM cannot suspend the thread, and it might freeze trying to suspend a thread until NFS services are restored.

Verify that your NFS server is configured correctly if you're having NFS problems.

Diagnostics Guide