This section describes the different Identity Manager and third-party debugging tools you can use to debug performance issues.
The information is organized into the following sections:
Tracing affects system performance. To help ensure optimal performance, specify the minimum tracing level or turn tracing off after debugging the system.
This section provides instructions for accessing the Identity Manager Debug pages and describes how to use these pages to identify and debug Identity Manager performance issues.
See the following sections for information:
Provisioning Threads for Administrator Configurator (Show_Provisioning.jsp)
XML Resource Adapter Caches Flushed and Cleared (Clear_XMLResourceAdapter_Cache.jsp)
You must have the Debug, Security Administrator, or Waveset Administrator capabilities to access and execute operations from the Identity Manager Debug pages. Administrators and Configurator are assigned this capability by default.
If you do not have the Debug capability, an error message results.
Open a browser and log in to the Administrator interface.
Type the following URL:
http:// host:port /idm/debug
where:
host is the application server on which you are running Identity Manager.
port is the number of the TCP port on which the server is listening.
When the System Settings page displays, type the .jsp file name for the debug page you want to open.
For example:
http:// host:port /idm/debug/pageName.jsp
Some debugging utilities are not linked from the System Settings page, but you can use them to enhance your ability to gather data for product performance and usability. For a complete list of debug pages, open a command window and list the contents of the idm/debug directory.
Use the Control Timings page to collect and view call timer statistics for different methods. You can use this information to track bottlenecks to specific methods and invoked APIs. You can also use options on the Call Timings page to import or export call timer metrics.
Call timing statistics are only collected while trace is enabled.
Open the Control Timings page, and click Start Timing & Tracing to enable trace and timing.
To stop the timing, click Stop Timing & Tracing or click Stop Timing.
The page re-displays and populates the Show Timings table with a list of methods for which statistics are available and the methods’ aggregate call timer statistics (not broken down by caller).
This table contains the following information:
Method name (Click a method name to see which methods it calls)
Total time
Average time
Minimum time
Maximum time
Total calls
Total errors
To clear the list, click Clear Timing.
You can also use the callTimer command to collect call timer data from the Console. This command is useful when you are debugging performance issues during an upgrade or in other situations where Identity Manager is not running on the application server.
Use the Edit Trace Configuration page to enable and configure tracing for the Java classes provided with your Identity Manager installation.
Specifically, you can use this page to configure the following trace settings:
Choose methods, classes, or packages to trace and specify the level of trace you want to capture.
Send trace information to a file or to standard output.
Specify the maximum number of trace files to be stored and the maximum size for each file.
Specify how dates and times are formatted in the trace output file.
Specify the maximum number of methods to be cached.
Indicate how to write data to the trace file.
Write data to the trace file as the data is generated, or queue the data and then write it to the file.
If you are not using a data source, you can use the Host Connection Pool page to view connection pool statistics. These statistics include the pool version, how many connections were created, how many are active, how many connections are in the pool, how many requests were serviced from the pool, and how many connections were destroyed.
You can also use the Host Connection Pool page to view a summary of the connection pools used to manage connections to the Gateway. You can use this information to investigate low-memory conditions.
Use the List Cache Cleared page to clear recently used XML parsers from the cache and to investigate low memory conditions.
Use the Method Timings page to quickly detect and assess hotspots at a method level.
The following information is gathered from Identity Manager methods and displayed on the Method Timings page:
Method names
How many times the methods were called
How many times the methods exited with an error status
Average time consumed by the methods
Minimum and maximum times consumed by invocations of each method
The Method Timings page also contains a table with the following links. You can click these links to view additional information.
Details. Shows call stack information.
History. Shows a graph of call duration compared with the time of the most recent calls.
History data. Shows a list of the most recent calls, showing what time the call was made and the duration of the call.
Identity Manager does not keep stack history by default. To keep stack history and to control its depth, edit Waveset.properites and look at the MethodTimer keys.
The Clear ALL option on the Method Timings page clears all results. This option is enabled by default.
Use the Object Size Summary page to detect problematically large objects that can affect your system.
The Object Size Summary page shows information about the size of objects (in characters) stored in the repository. These objects are listed by type, along with the total number of objects of each type, and the objects’ total combined size, average size, maximum size, and minimum size.
Click an entry in the Type column to view additional size information about that object type. For example, click Configuration to view the ID, name, and size of the largest configuration objects in the repository.
You can also access this size information from the Console command line.
Open the console.
At the command prompt, type:
showSizes [ type[limit ]]
For upgrades, existing objects will report a size of 0 until they have been updated or otherwise refreshed.
Use the Provisioning Threads for Administrator Configurator to view a summary of the provisioning threads in use by the system. This summary is a subset of the information available in Show_Threads.jsp.
Looking at just a single thread dump can be misleading.
Use the System Cache Summary page to view information about the following items to help you investigate low-memory conditions:
Administrator-associated object caches
System object cache
User login sessions
XML parser cache
Use the System Memory Summary page to view how much total and free memory you have available in Mbytes. When you are using memory-intensive functionality such as Reconciliation, this information can help you determine whether there is sufficient memory allocated to the JVM.
You can also use this page to launch garbage collection or to clear unused memory in the JVM for investigating heap usage.
The System Properties page provides information about your environment, including software versions, paths and environmental variables.
Use the System Threads page to see which processes are running so you can verify that automated processes (such as reconciliation or Active Sync) are running.
This page includes information about the process type, process name, its priority, if the process is a daemon, and if the process is still running.
Looking at just a single thread dump can be misleading.
Use the Session Pool Cleared page to clear all of the cached sessions for users who have recently logged in and to investigate low memory conditions.
Use the Waveset Properties page to view and temporarily edit properties in the Waveset.properties file. You can test different property settings for a particular server on which the Waveset.properties file resides without having to restart the server to pick up the changes. The edited property settings only remain in effect until the next time you restart the server.
Use the XML Resource Adapter Caches Flushed and Cleared page to clear test XML resource adapters from the cache and to investigate low memory conditions.
You can use the following Sun Microsystems’ and third-party tools to identify potential performance bottlenecks:
These tools can be particularly useful if your deployment uses custom Java classes.
Identity Manager provides a Profiler utility to help you troubleshoot performance problems in your deployment.
Customized forms, Java, rules, workflows, and XPRESS can cause performance and scale problems. The Profiler profiles how much time is spent in these different areas, enabling you to determine whether these forms, Java, rules, workflows, or XPRESS objects are contributing to performance and scale problems and, if so, which parts of these objects are causing the problems.
For more information about Profiler, see Working with the Identity Manager Profiler in Sun Identity Manager 8.1 Release Notes.
The DTrace facility is a dynamic tracing framework for the Solaris 10 operating system that enables you to monitor JVM activity.
DTrace contains more than 30,000 probes and uses integrated user-level and kernel-level tracing to give you a view into your production system. You can also trace arbitrary data and expressions by using the D language, which is similar to C or awk. The DTrace facility also includes special support for monitoring the JVM, and enables you to watch your whole system and span outside the JVM.
DTrace is easiest to use with Java 6 because probes are built into the JVM. The facility also works with Java 1.4 and Java 5, but you must download JVM PI or JVM TI agents from the following URL:
https://solaris10-dtrace-vm-agents.dev.java.net/
The following example shows how to write a DTrace script.
#!/usr/sbin/dtrace -Zs #pragma D option quiet hotspot$1::: { printf("%s\n", probename); } |
In this example, you would replace $1 with the first argument to the script, which is the PID of the Java process you want to monitor. For example:
# ./all-jvm-probes.d 1234
The following table describes the commands you can use to enable different DTrace probes.
Table 4–3 DTrace Commands
Because DTrace causes additional work in the system, enabling this facility affects system performance. The effect is often negligible, but can become substantial if you enable many probes with costly enablings.
Instructions for minimizing the performance effect of DTrace are provided in the “Performance Considerations” chapter of the Solaris Dynamic Tracing Guide.
For more information about DTrace, see /usr/demo/dtrace and man dtrace.
Identity Manager enables you to use Java Management Extensions (JMX) to capture and expose operational statistics for certain resource adapter operations. You can use this data for diagnostic and predictive purposes, such as to monitor system health and reports.
This statistical data includes the following:
The number of times the action was performed
The minimum, maximum, and average duration of the operations
Objects |
Actions Monitored |
---|---|
For Accounts |
|
For Actions |
Run |
For Other Objects |
|
JMX creates MBeans for each resource adapter, by server, and registers these beans with names that match the following pattern:
serverName=server name, resourceAdapterType=Resource Adapter Type, resourceAdapterName=Resource Adapter Name |
Identity Manager records statistics for all completed operations, whether they completed successfully or with errors. However, Identity Manager does not record statistics for incomplete operations, such as any operations that throw exceptions.
You can configure excludes as follows:
From the Administrator interface, select Configure -> Servers.
On the Configure Servers page, perform one of the following tasks:
Click the Edit Default Server Settings button to edit the default server settings.
Click a server link to edit the policy for that server.
Click the JMX tab and enable the JMX Enable Resource Adapter Monitor box to turn on resource monitoring.
To exclude specific resources, add regular expressions to the JMX Resource Adapter Monitor Excludes list.
To exclude monitoring specific actions, add regular expressions to the JMX Resource Adapter Monitor Operation Excludes list.
All excludes use regular expressions. For excluding certain resources, JMX just matches on the resource name. For example, if you have adapters named
resource1 resource2 resource3 resource10 resource11 |
and you specify the following pattern
.*1$ |
which means, match any 0 or more of any character (.*) until something that ends with a 1 (1$). JMX will exclude resource1 and resource11.
For operations, the process is similar. If your operations have the following names, the patterns must match against those names.
ACCOUNT_CREATE ACCOUNT_UPDATE ACCOUNT_DELETE ACCOUNT_GET ACCOUNT_AUTHENTICATE OBJECT_CREATE OBJECT_UPDATE OBJECT_DELETE OBJECT_GET OBJECT_LIST ACTION_RUN |
For example, the ^ACCOUNT.* pattern excludes all operations that start with ACCOUNT. Or, using this pattern excludes updates and deletes:
.*UPDATE$ .*DELETE$ |
For more information about configuring and using JMX, see Configuring JMX Monitoring and The JMX Publisher Type in Sun Identity Manager 8.1 Business Administrator’s Guide.
The Java Monitoring and Management Console (JConsole) is a Java Management Extension (JMX) technology-compliant graphical management tool that is co-packaged with at least JDK 5. JConsole connects to a running JVM and gathers information from the JVM MBeans in the connected JMX agent.
Specifically, you can use JConsole to perform the following tasks:
Detect low memory and deadlocks
JConsole accesses the memory system, memory pools, and MBeans garbage collector to provide information about memory use, including memory consumption, memory pools, and garbage collection statistics.
Enable or disable garbage collection
Enable or disable verbose tracing
Monitor local and remote applications
Monitor and manage MBeans including current heap memory use, non-heap memory use, and how many objects are pending for finalization
View information about performance, resource consumption, and server statistics
View summary information about the JVM and monitored values, threads running on the application, and loaded classes
View information about operating system resources (Sun’s platform extension), such as:
CPU process time
How much total and free physical memory is available
The amount of committed virtual memory (how much virtual memory is guaranteed to be available to the running process)
How much total and free swap space is available
The number of open file descriptions (UNIX®only)
For more information about using JConsole to monitor applications on the Java platform, see the Sun Developer Network (SDN) article titled Using JConsole to Monitor Applications, which is available from the following URL:
http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
Identity Manager supplies some JMX MBeans that provide information about the following:
Identity Manager Server Cluster
Data Exporter
Scheduler
You can use the Java Runtime Analysis Toolkit (JRat), an open-source performance profiler for the Java platform, to identify potential performance bottlenecks, especially if your deployment uses custom Java classes. JRat monitors your application’s execution and persists the application’s performance measurements.
For example, if you have a custom workflow for provisioning, you can use JRat to see which classes are being invoked and how much time is required to run your workflow compared to the default Identity Manager provisioning workflow.
For more information about JRat, see http://jrat.sourceforge.net.