| Oracle® Fusion Applications Administrator's Guide 11g Release 1 (11.1.2) Part Number E14496-03 |
|
|
PDF · Mobi · ePub |
| Oracle® Fusion Applications Administrator's Guide 11g Release 1 (11.1.2) Part Number E14496-03 |
|
|
PDF · Mobi · ePub |
This chapter describes how to diagnose Java applications in the middle tier using Oracle Enterprise Manager Cloud Control (Cloud Control):
This chapter includes the following topics:
Mission critical Java applications often suffer from availability and performance problems. Developers and IT administrators spend a lot of time diagnosing the root cause of these problems. Many times, the problems occurring in production environments either cannot be reproduced or may take too long to reproduce in other environments. This can cause severe impact on the business.
Oracle Enterprise Manager Cloud Control (Cloud Control) enables you to diagnose performance problems in Java applications in the production environment. By eliminating the need to reproduce problems, it reduces the time required to resolve these problems. This improves application availability and performance. Using Java Virtual Machine (JVM) diagnostics, you can identify the root cause of performance problems in the production environment without having to reproduce them in the test or development environment. It does not require complex instrumentation or restarting of the application to get in-depth application details. Application administrators will be able to identify Java problems or Database issues that are causing application downtime without any detailed application knowledge.
For a better understanding of JVM diagnostics, see the chapters in "JVM Diagnostics" part in the Oracle Enterprise Manager Getting Started with Fusion Middleware Management.
This section describes how to diagnose Java applications in the middle tier using Cloud Control:
The JVM Diagnostics Summary page in Cloud Control is the best starting place for diagnosing performance problems in Java applications. It provides diagnostics summary details for a selected period for an entire Oracle WebLogic Server domain and each of its Managed Servers and Administration Server.
Figure 11-1 shows the JVM Diagnostics Summary page for an Oracle WebLogic Server domains in the Oracle Fusion Applications environment.
The following charts display on this page:
Server State Charts For Selected Period: This section displays the Active Threads, CPU Utilization, and Heap Utilization of IO, CPU, lock, and network resources during the selected time. The number of Java threads (daemon and non-daemon) that are currently running in the virtual machine for this Oracle WebLogic Server
Active Threads: This chart displays the number of Java threads that are currently running in the for the domain or server. It is color-coded by thread state.
CPU Utilization: This chart shows the percentage of CPU used by the system and processes in the time period.
Heap Utilization: This chart shows the percentage of computer memory used by the Java heap in the time period.
Garbage Collections: This chart shows the number of times the JVM garbage collector was invoked in the time period. It includes both major and minor garbage collections.
Top Methods: This section shows the most expensive Java methods in the selected time period.
Top Requests: This section shows the top page requests in the selected time period.
The Compared with feature enables you to compare the diagnostics across two specified periods of time.
To access the JVM Diagnostics Summary page in Cloud Control:
From the Targets menu, choose Targets > Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain or one of its servers which has JVMD agents deployed.
The WebLogic Server Domain Home page displays.
From the WebLogic Domain menu, choose JVM Diagnostics > Summary.
Click the Summary tab, which is shown in Figure 11-1.
You can click the Details tab for a a more detailed view of the diagnostic data, as shown in the following figure:
If you have a slow-running application, locate the Java method causing the potential issue.
To find the top Java methods with Cloud Control:
From the Targets menu, choose Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain which has JVMD agents deployed.
The WebLogic Server Domain Home page displays.
From the WebLogic Domain menu, choose JVM Diagnostics > Summary.
Click the Details tab.
In the Top Methods section, review the list of methods ordered by their cost (the number of samples).
In the Top Methods section, click on a method to add it as a filter.
The Filter Options section auto-fills the information on the method and the charts update to reflect that method, for example:
Heavy use of IO, CPU, lock, and network resources (Active Threads chart)
Request causing it to be invoked (Top Requests chart)
SQL or database state it causes (Top DBStates and Top SQLs charts)
Click on the plus sign next to the method to investigate further.
The Method Stack Tree dialog displays with a call stack that you can traverse.
Click OK to close the Method Stack Tree dialog.
In Filter Options, clear out the Method text box filter and click anywhere to remove the filter.
If you suspect a slow SQL query is causing a network requests or IO issue, find the slowest SQL queries.
To find the top SQL calls using JVM diagnostics with Cloud Control:
From the Targets menu, choose Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain which has JVMD agents deployed.
The WebLogic Server Domain Home page displays.
From the WebLogic Domain menu, choose JVM Diagnostics > Summary.
Click the Details tab.
In the Top SQLs section, review the list of SQL calls ordered by their cost (the number of samples).
In the Top SQLs section, click on a SQL call to view the charts for that call.
The Filter Options section auto-fills the information on the method and the charts update to reflect that method. Adding the statement as a filter enables you to see everything related to that SQL call, for example:
Methods that invoke it (Top Methods chart)
Request causing it to be invoked (Top Requests chart)
Database state it causes (Top DBStates chart)
In Filter Options, clear out the SQL field and click anywhere to remove the filter.
If application users report a spinning status indication after clicking in the application, investigate the stuck threads.
To find the top SQL calls using JVM diagnostics with Cloud Control:
From the Targets menu, choose Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain which has JVMD agents deployed, or click on a specific server having problems.
From the WebLogic Domain menu or WebLogic Server menu, choose JVM Diagnostics > Threads > Real-Time Analysis.
In the JVMs section, click on a thread in the upper section show details in the Threads section.
In the Threads section, look for a thread having the prefix [STUCK THREAD] and click on it.
Look at the fields Current Call, File Name, Line, and State for the thread.
This information provides you with the key information on how to locate the code that is causing the problem:
Current Call: This field displays the name of the method call where the code is stuck.
File Name: This column identifies the file with the problem.
Line: This column identifies the line number in the file where the problematic code is.
State: This column displays the state of the thread (for example, CPU, IO, Network, DB Wait, Lock, and so on).
Look for the Thread State and the Locks in the details view.
If the stuck thread is in the DB Wait state then click on the hyperlink and go directly to the database session to see what that thread is doing in the database, or use the technique described in Section 11.2.5.
If you issue an SQL query and it does not return, then analyze the SQL statement.
To analyze SQL from Cloud Control:
From the Targets menu, choose Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain which has JVMD agents deployed, or click on a specific server having problems.
From the WebLogic Domain menu, choose JVM Diagnostics > Summary.
Click the Details tab.
In the Top SQLs section, click on a SQL call to view the charts for that call.
In Filter Options, in the SQL field, copy the ID of the SQL call.
Click on Databases in secondary navigator.
Select the relevant target database.
Click the Performance tab.
In the Additional Instance Monitoring Links section, click Search SQL.
Paste the ID of the SQL call into the relevant field with any other choices you may need and then click Search.
Analyze the SQL.
To find and analyze memory leaks, you can use Cloud Control to take and analyze snapshots of the heap.
Analyzing heap requires a large amount of free space in the Oracle Database tablespace being used. As a standard practice, ensure you have five times the size of heap dump file being loaded in the tablespace. Since you know the size of your dump file, make sure that there is adequate space to accommodate the dump file before it is loaded into the database.
To use heap snapshot to analyze memory leaks:
To create a snapshot of the heap for later loading and examination for leaks:
From the Targets menu, choose Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain which has JVMD agents deployed, or click on a specific server having problems.
From the WebLogic Domain menu, choose JVM Diagnostics > Heap > Take Snapshot Now.
The dump Java heap page displays with the name of the file, heapdump#.txt in /tmp on Unix and C:\ on Windows. This file lists all the heap objects and their relationships that you can load and examine for leaks.
To modify the default location of the heapdump#.txt file:
From the WebLogic Domain menu, choose JVM Diagnostics > Setup.
Click the JVMs tab.
Select the edit icon for a JVM ID.
In the Edit JVM Information page, modify the Heap Dump Dir field.
Click Save.
To create a snapshot of the heap for later loading and examination for leaks:
From the Targets menu, choose Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain which has JVMD agents deployed, or click on a specific server having problems.
From the WebLogic Domain menu, choose JVM Diagnostics > Setup.
Click the Downloads tab.
For Load Heap, click on the Download icon.
Transfer the loadheap.zip file to a computer that has Oracle Database client installed.
Unzip loadheap.zip to some known location, for example, ~/jvmd.
Steps 1 - 8 are one-time steps. The next heap dump that needs to be loaded does not require the above steps to be repeated in order to upload a heap snapshot.
Transfer the heap dump files to the same host, for example, /scratch/jvmd/heapdump22.txt.
In the known location, go to the loadheap directory. For example:
cd ~/jvmd/loadheap
Run the loadpheap utlity for the file you created:
(UNIX) loadheap.sh HEAP_FILE_NAME (Windows) loadheap.cmd HEAP_FILE_NAME
where HEAP_FILE_NAME is the heap dump file you created in Task 0, "Take a Heap Snapshot".
./loadheap.sh /scratch/jvmd/heapdump22.txt
While running the command, you will be asked to enter in the connect information to your repository (host name, SID, username and password). This command can be time consuming process. After the command is finished, scroll up to examine if there has been any error.
If you want to compare heaps, you run loadheap at least twice to get a pair of heaps loaded and compared.
To analyze heap snapshots:
From the Targets menu, choose Middleware.
The Middleware target home page displays.
Search for the Oracle WebLogic Server domains:
From the Search area, click Advanced Search.
From the Type list, choose Oracle WebLogic Domain.
Click Search.
Click on a domain which has JVMD agents deployed, or click on a specific server having problems.
From the WebLogic Domain menu, choose JVM Diagnostics > Heap > Saved Heap Snapshots.
The Available Heaps table lists all the snapshots that have been loaded into the repository.
Examine one of the heaps by clicking on the number in the % column.
The Heap Usage page displays.
Click the Roots tab to see the objects reachable by roots, which are objects that are directly reachable from the JVM itself.
The Show Heap Usage by Roots page displays.
Click a root to drill down and view the objects that consume a lot of memory. They are most likely to be in the Dictionary root.
The Top 40 Objects in Threadlocals page displays.
Click the Compare Heaps tab to compare the current heap with another previously taken heap dump.
When comparing heaps, load the bigger one first. Otherwise you may see negative deltas.
Select the second heap by clicking on the number in the % column.
Compare both the heaps. Compare the number of objects and the occupied memory size in each heap dump. This measure indicates the objects that are growing over the period of time when the snapshots were taken.
Drill down into the row which had the largest delta (Adjusted) in order to find the biggest memory leak.
|
 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Legal Notices |
|
