test

Sun Identity Manager 8.1 System Administrator's Guide

Debugging Performance Issues

This section describes the different Identity Manager and third-party debugging tools you can use to debug performance issues.

The information is organized into the following sections:

Working With Identity Manager Debug Pages

Note –

Tracing affects system performance. To help ensure optimal performance, specify the minimum tracing level or turn tracing off after debugging the system.

This section provides instructions for accessing the Identity Manager Debug pages and describes how to use these pages to identify and debug Identity Manager performance issues.

See the following sections for information:

Accessing the Debug Pages

Note –

You must have the Debug, Security Administrator, or Waveset Administrator capabilities to access and execute operations from the Identity Manager Debug pages. Administrators and Configurator are assigned this capability by default.

If you do not have the Debug capability, an error message results.

ProcedureTo Access the Identity Manager Debug Pages

  1. Open a browser and log in to the Administrator interface.

  2. Type the following URL:

    http:// host:port /idm/debug

    where:

    • host is the application server on which you are running Identity Manager.

    • port is the number of the TCP port on which the server is listening.

  3. When the System Settings page displays, type the .jsp file name for the debug page you want to open.

    For example:

    http:// host:port /idm/debug/pageName.jsp

    Note –

    Some debugging utilities are not linked from the System Settings page, but you can use them to enhance your ability to gather data for product performance and usability. For a complete list of debug pages, open a command window and list the contents of the idm/debug directory.

Control Timings (callTimer.jsp)

Use the Control Timings page to collect and view call timer statistics for different methods. You can use this information to track bottlenecks to specific methods and invoked APIs. You can also use options on the Call Timings page to import or export call timer metrics.

Note –

Call timing statistics are only collected while trace is enabled.

ProcedureTo View Call Timer Statistics

  1. Open the Control Timings page, and click Start Timing & Tracing to enable trace and timing.

  2. To stop the timing, click Stop Timing & Tracing or click Stop Timing.

    The page re-displays and populates the Show Timings table with a list of methods for which statistics are available and the methods’ aggregate call timer statistics (not broken down by caller).

    This table contains the following information:

    • Method name (Click a method name to see which methods it calls)

    • Total time

    • Average time

    • Minimum time

    • Maximum time

    • Total calls

    • Total errors

  3. To clear the list, click Clear Timing.

    Note –

    You can also use the callTimer command to collect call timer data from the Console. This command is useful when you are debugging performance issues during an upgrade or in other situations where Identity Manager is not running on the application server.

Edit Trace Configuration (Show_Trace.jsp )

Use the Edit Trace Configuration page to enable and configure tracing for the Java classes provided with your Identity Manager installation.

Specifically, you can use this page to configure the following trace settings:

Host Connection Pool (Show_ConnectionPools.jsp)

If you are not using a data source, you can use the Host Connection Pool page to view connection pool statistics. These statistics include the pool version, how many connections were created, how many are active, how many connections are in the pool, how many requests were serviced from the pool, and how many connections were destroyed.

You can also use the Host Connection Pool page to view a summary of the connection pools used to manage connections to the Gateway. You can use this information to investigate low-memory conditions.

List Cache Cleared (Clear_XMLParser_Cache.jsp)

Use the List Cache Cleared page to clear recently used XML parsers from the cache and to investigate low memory conditions.

Method Timings (Show_Timings.jsp)

Use the Method Timings page to quickly detect and assess hotspots at a method level.

The following information is gathered from Identity Manager methods and displayed on the Method Timings page:

The Method Timings page also contains a table with the following links. You can click these links to view additional information.

Note –

The Clear ALL option on the Method Timings page clears all results. This option is enabled by default.

Object Size Summary (Show_Sizes.jsp)

Use the Object Size Summary page to detect problematically large objects that can affect your system.

The Object Size Summary page shows information about the size of objects (in characters) stored in the repository. These objects are listed by type, along with the total number of objects of each type, and the objects’ total combined size, average size, maximum size, and minimum size.

Click an entry in the Type column to view additional size information about that object type. For example, click Configuration to view the ID, name, and size of the largest configuration objects in the repository.

You can also access this size information from the Console command line.

ProcedureTo Access Object Size Information from the Command Line

  1. Open the console.

  2. At the command prompt, type:

    showSizes [ type[limit ]]

    Note –

    For upgrades, existing objects will report a size of 0 until they have been updated or otherwise refreshed.

Provisioning Threads for Administrator Configurator (Show_Provisioning.jsp)

Use the Provisioning Threads for Administrator Configurator to view a summary of the provisioning threads in use by the system. This summary is a subset of the information available in Show_Threads.jsp.

Note –

Looking at just a single thread dump can be misleading.

System Cache Summary (Show_CacheSummary.jsp)

Use the System Cache Summary page to view information about the following items to help you investigate low-memory conditions:

System Memory Summary (Show_Memory.jsp)

Use the System Memory Summary page to view how much total and free memory you have available in Mbytes. When you are using memory-intensive functionality such as Reconciliation, this information can help you determine whether there is sufficient memory allocated to the JVM.

You can also use this page to launch garbage collection or to clear unused memory in the JVM for investigating heap usage.

System Properties (SysInfo.jsp)

The System Properties page provides information about your environment, including software versions, paths and environmental variables.

System Threads (Show_Threads.jsp)

Use the System Threads page to see which processes are running so you can verify that automated processes (such as reconciliation or Active Sync) are running.

This page includes information about the process type, process name, its priority, if the process is a daemon, and if the process is still running.

Note –

Looking at just a single thread dump can be misleading.

User Session Pool Cleared (Clear_User_Cache.jsp)

Use the Session Pool Cleared page to clear all of the cached sessions for users who have recently logged in and to investigate low memory conditions.

Waveset Properties (Show_WSProp.jsp)

Use the Waveset Properties page to view and temporarily edit properties in the Waveset.properties file. You can test different property settings for a particular server on which the Waveset.properties file resides without having to restart the server to pick up the changes. The edited property settings only remain in effect until the next time you restart the server.

XML Resource Adapter Caches Flushed and Cleared (Clear_XMLResourceAdapter_Cache.jsp)

Use the XML Resource Adapter Caches Flushed and Cleared page to clear test XML resource adapters from the cache and to investigate low memory conditions.

Working With Other Debugging Tools

You can use the following Sun Microsystems’ and third-party tools to identify potential performance bottlenecks:

These tools can be particularly useful if your deployment uses custom Java classes.

Identity Manager Profiler

Identity Manager provides a Profiler utility to help you troubleshoot performance problems in your deployment.

Customized forms, Java, rules, workflows, and XPRESS can cause performance and scale problems. The Profiler profiles how much time is spent in these different areas, enabling you to determine whether these forms, Java, rules, workflows, or XPRESS objects are contributing to performance and scale problems and, if so, which parts of these objects are causing the problems.

Note –

For more information about Profiler, see Working with the Identity Manager Profiler in Sun Identity Manager 8.1 Release Notes.

Using DTrace

The DTrace facility is a dynamic tracing framework for the Solaris 10 operating system that enables you to monitor JVM activity.

DTrace contains more than 30,000 probes and uses integrated user-level and kernel-level tracing to give you a view into your production system. You can also trace arbitrary data and expressions by using the D language, which is similar to C or awk. The DTrace facility also includes special support for monitoring the JVM, and enables you to watch your whole system and span outside the JVM.

DTrace is easiest to use with Java 6 because probes are built into the JVM. The facility also works with Java 1.4 and Java 5, but you must download JVM PI or JVM TI agents from the following URL:

https://solaris10-dtrace-vm-agents.dev.java.net/

The following example shows how to write a DTrace script.

Example 4–2 Example DTrace Script


#!/usr/sbin/dtrace -Zs 
#pragma D option quiet
hotspot$1::: 
{
  printf("%s\n", probename); 
}

In this example, you would replace $1 with the first argument to the script, which is the PID of the Java process you want to monitor. For example:

# ./all-jvm-probes.d 1234

The following table describes the commands you can use to enable different DTrace probes.

Table 4–3 DTrace Commands

Command 

Description 

-XX:+DTraceMonitorProbes

Enables JVM support in Java 6 (patches for Java 1.4 and Java 5) 

-XX:+ExtendedDTraceProbes

Provides the following information: 

  • JVM startup (begin and end) and shutdown

  • Thread starting and stopping

  • Class loading and unloading

  • Garbage collection (several options available)

  • JIT compilation begin and end

  • Compiled method loading and unloading

  • Monitor contention, wait, and notify

  • Method entry, method return, and object allocation

/usr/sbin/dtrace -n ’hotspot*:::’

Enables all JVM probes for all Java processes on the system 

/usr/sbin/dtrace -n ’hotspot1234:::’

Enables all JVM probes for only the Java process with PID 1234

/usr/sbin/dtrace -n ’hotspot1234:::gc-begin’

Enables only the probe that starts when garbage collection for process 1234 begins

Note –

Because DTrace causes additional work in the system, enabling this facility affects system performance. The effect is often negligible, but can become substantial if you enable many probes with costly enablings.

Instructions for minimizing the performance effect of DTrace are provided in the “Performance Considerations” chapter of the Solaris Dynamic Tracing Guide.

For more information about DTrace, see /usr/demo/dtrace and man dtrace.

Using JMX

Identity Manager enables you to use Java Management Extensions (JMX) to capture and expose operational statistics for certain resource adapter operations. You can use this data for diagnostic and predictive purposes, such as to monitor system health and reports.

This statistical data includes the following:

Objects 

Actions Monitored 

For Accounts 

  • Create

  • Update

  • Delete

  • Get

  • Authenticate

For Actions 

Run 

For Other Objects 

  • Create

  • Update

  • Delete

  • Get

  • List

JMX creates MBeans for each resource adapter, by server, and registers these beans with names that match the following pattern:


serverName=server name, resourceAdapterType=Resource Adapter Type,
resourceAdapterName=Resource Adapter Name

Identity Manager records statistics for all completed operations, whether they completed successfully or with errors. However, Identity Manager does not record statistics for incomplete operations, such as any operations that throw exceptions.

You can configure excludes as follows:

  1. From the Administrator interface, select Configure -> Servers.

  2. On the Configure Servers page, perform one of the following tasks:

    • Click the Edit Default Server Settings button to edit the default server settings.

    • Click a server link to edit the policy for that server.

  3. Click the JMX tab and enable the JMX Enable Resource Adapter Monitor box to turn on resource monitoring.

    • To exclude specific resources, add regular expressions to the JMX Resource Adapter Monitor Excludes list.

    • To exclude monitoring specific actions, add regular expressions to the JMX Resource Adapter Monitor Operation Excludes list.

All excludes use regular expressions. For excluding certain resources, JMX just matches on the resource name. For example, if you have adapters named


resource1
resource2
resource3
resource10
resource11

and you specify the following pattern


.*1$

which means, match any 0 or more of any character (.*) until something that ends with a 1 (1$). JMX will exclude resource1 and resource11.

For operations, the process is similar. If your operations have the following names, the patterns must match against those names.


ACCOUNT_CREATE
ACCOUNT_UPDATE
ACCOUNT_DELETE
ACCOUNT_GET
ACCOUNT_AUTHENTICATE
OBJECT_CREATE
OBJECT_UPDATE
OBJECT_DELETE
OBJECT_GET
OBJECT_LIST
ACTION_RUN

For example, the ^ACCOUNT.* pattern excludes all operations that start with ACCOUNT. Or, using this pattern excludes updates and deletes:


.*UPDATE$
.*DELETE$
Note –

For more information about configuring and using JMX, see Configuring JMX Monitoring and The JMX Publisher Type in Sun Identity Manager 8.1 Business Administrator’s Guide.

Using JConsole

The Java Monitoring and Management Console (JConsole) is a Java Management Extension (JMX) technology-compliant graphical management tool that is co-packaged with at least JDK 5. JConsole connects to a running JVM and gathers information from the JVM MBeans in the connected JMX agent.

Specifically, you can use JConsole to perform the following tasks:

Note –

For more information about using JConsole to monitor applications on the Java platform, see the Sun Developer Network (SDN) article titled Using JConsole to Monitor Applications, which is available from the following URL:

http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

Identity Manager supplies some JMX MBeans that provide information about the following:

Using JRat

You can use the Java Runtime Analysis Toolkit (JRat), an open-source performance profiler for the Java platform, to identify potential performance bottlenecks, especially if your deployment uses custom Java classes. JRat monitors your application’s execution and persists the application’s performance measurements.

For example, if you have a custom workflow for provisioning, you can use JRat to see which classes are being invoked and how much time is required to run your workflow compared to the default Identity Manager provisioning workflow.

For more information about JRat, see http://jrat.sourceforge.net.