Chapter 5 Tracing and Troubleshooting

This chapter explains how you can use tracing to help fix errors, solve performance issues, and understand the flow within Identity Manager. This chapter also describes methods for troubleshooting problems that might occur with different Identity Manager components.

This chapter covers the following topics:

• Before You Begin Tracing or Troubleshooting

• Tracing Identity Manager Objects and Activities

• Tracing the Identity Manager Gateway Objects and Activities

• Troubleshooting and Fixing Common Problems

Before You Begin Tracing or Troubleshooting

Review the information in the following sections before tracing or troubleshooting Identity Manager:

• Intended Audience

• Important Notes About Tracing and Troubleshooting Identity Manager

• Before Calling Support

• Related Documentation and Web Sites

• Process Overview

Intended Audience

This chapter is intended for application server and database administrators, front-line support engineers, and partners who are responsible for maintaining Identity Manager in a deployment environment.

Before troubleshooting problems with Identity Manager, you must

• Be familiar with Java 5.0 (required for Sun Identity Manager 8.1)

• Understand the components that you are trying to troubleshoot

Important Notes About Tracing and Troubleshooting Identity Manager

Before you start tracing or troubleshooting Identity Manager, note the following:

• Tracing affects system performance. To ensure optimal performance, use the minimum tracing level needed or turn off tracing after debugging the systems.

• Do not enable tracing for the com.waveset class. The com.waveset class is verbose and has many classes, so tracing this class might cause your server to hang.

• Do not configure Identity Manager to trace at the advanced level unless instructed to do so by Sun Support.

• If you set exception.trace in the Waveset.properties file to debug a specific problem, do not use it for an extended period of time and be sure you disable it after debugging. Setting exception.trace in the Waveset.properties file can significantly affect system performance.

Before Calling Support

Before you call Sun Technical Support for assistance with a problem, consider the following suggestions for narrowing the problem area:

• Try using web browser search tools to research the problem.

• Use resource-specific support options when available. Your problem might be related to a resource and fixed using a resource patch.

• Remove Identity Manager from the equation by using an appropriate client tool when possible. For example, by using a Java LDAP browser.

Also, you must collect and have the following information available before calling Sun Support for assistance.

Related Documentation and Web Sites

In addition to the information provided in this chapter, consult the documents and web sites listed in this section for information related to tracing and troubleshooting Identity Manager.

Recommended Reading

See the following documents for information related to tracing and troubleshooting Identity Manager:

• Testing Your Customized Form in Sun Identity Manager Deployment Reference for the recommended method of tracing XPRESS functions.

• The article titled Using JConsole to Monitor Applications for information about using JConsole to monitor applications that run on the Java platform. This article is available from the following URL:

  http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

Useful Web Sites

The following table describes web sites related to troubleshooting Identity Manager.

Process Overview

Generally, you can identify and resolve problems in your deployment if you follow the steps shown in the following illustration.

Figure 5–1 Tracing and Troubleshooting a Problem

See Tracing Identity Manager Objects and Activities for information about how to enable tracing for different Identity Manager objects and activities.

See Troubleshooting and Fixing Common Problems for information about how to troubleshoot problems that commonly occur for those objects and activities.

Tracing Identity Manager Objects and Activities

Trace output can be very helpful when you are trying to identify and resolve problems, or when you are developing custom resource adapters.

This section describes how to enable tracing for a variety of Identity Manager objects and activities. The information is organized as follows:

• How To Configure Tracing

• How to View Trace Files

• Tracing the Identity Manager Server

• Tracing Adapters

• Tracing Custom Code

• Tracing Exceptions

• Tracing Forms

• Tracing Global XPRESS

• Tracing PasswordSync

• Tracing Sun Identity Manager Service Provider Delegated Administration

• Tracing Reconciliation

• Tracing the setRepo Command

• Tracing SPML

• Tracing the Task Scheduler

• Tracing Workflows

• Locating Version Information

Tracing affects system performance. To ensure optimal performance, specify the minimum tracing level or turn tracing off after debugging the system.

How To Configure Tracing

You can enable and configure tracing from several locations within Identity Manager. Instructions are provided in the following sections:

• To Enable Trace from the System Settings Page

• To Configure Tracing From Individual Debug Pages

• To Enable Tracing From the Identity Manager Console

To Enable Trace from the System Settings Page

The System Settings page is the primary Identity Manager Debug page.

From this page, you can

• View and edit objects in the repository

• Clear caches

• Set specialized traces

• Reload the Waveset.properties file

Use the following steps to enable tracing from the Identity Manager System Settings page:

1. Open a browser and log in to the Identity Manager Administrative interface.

2. Type the following URL:

   http://hostport/idm/debug

   where:

   • host is the local server on which you are running Identity Manager.

   • port is the number of the TCP port on which the server is listening.

3. When the System Settings page displays, click Show Trace (located near the bottom of the page) to manage a single trace configuration, where you can create, modify, or delete up to ten trace settings.

   ---

   Note –

   The remaining instructions assume you have only one trace configuration.

   To manage multiple trace configurations, click the Show Trace List button instead. When the Trace Configuration page displays, click the configuration name to edit the current settings.

   Identity Manager supplies one configuration, called Global, by default. However, if you have multiple servers in an Identity Manager instance, defining different configurations for particular servers might be useful. If the name of a trace configuration matches the name of the current host, the host configuration overrides the Global configuration.

   ---

Next Steps

After you configure tracing, you can view and edit the default global trace configuration object or create a new configuration object as described in the following sections:

• To Edit the Default Configuration Object

• To Create a New Trace Configuration Object

To Edit the Default Configuration Object

1. From the Edit Trace Configuration page, click the Trace Enabled box to enable tracing.

   ---

   Note –

   Deselecting this box stops tracing, but keeps your configuration. You can turn tracing on and off without having to remember and retype the classes you were tracing.

   ---

2. Specify which classes, packages, or methods to trace by typing their names into the table.

   For example:

   • To trace all classes in the waveset.repository package, enter com.waveset.repository.

   • To trace the AbstractDataStore class in the waveset.repository package, enter com.waveset.repository.AbstractDataStore.

   • To trace the list method in the AbstractDataStore class in the waveset package, enter com.waveset.repository.AbstractDataStore#list.

   Do not enable trace for the com.waveset class. The com.waveset class is verbose and has many classes, so tracing this class might cause your server to hang.

3. In the same table, choose a Method/Class tracing level from the Level menu.

   Each level captures different types of information, as described in the following table:

   Trace Level	Description
   0	Minimum debugging output, traces exceptions and error information only
   1	Trace Level 0 events, plus entries and exits for public methods
   2	Trace Level 1 events, plus entries and exits for non-public methods
   3	Trace Level 2 events, plus decision points and significant variables
   4	Maximum debugging output

   ---

   Note –

   Method/Class tracing produces a predictable, but possibly very large volume of trace output. Try to be as specific as possible when specifying methods and classes to trace.

   ---

4. (Optional) To enable subcall tracing, choose a level from the Subcall Trace menu. This menu uses the same trace numbering scale described in the previous table.

   ---

   Note –

   • The default Subcall Tracing level is none, which disables subcall tracing on a per-method or per-class basis.

   • Subcall Tracing levels are independent of the Method/Class tracing levels you specified in the previous step.

   ---

   When you enable Subcall Tracing for a particular method that supports subcall tracing, you are automatically setting the tracing level for methods that are called by this method. Subcall Tracing enables you to produce a short, but detailed burst of trace output that is triggered by the entry into a specified method and terminated by the method’s exit.

   For example, assume you created a trace configuration setting for the com.waveset.adapter.NewRes#init method, set Method/Class tracing to level one and set Subcall tracing to level three.

   Also, assume that the init method calls two other methods:

   • NewRes#subcallA

   • NewRes#subcallB

     When the init method runs, the com.waveset.adapter.NewRes#init method produces trace output at level one until reaching subcallA. When subcallA begins executing, the trace level changes to three and continues at that level until subcallA exits. The com.waveset.adapter.NewRes#init method returns to the init method and restores the trace level to one. Later, when init calls subcallB, there is another burst of level three trace detail that lasts until subcallB exits. Finally, when init exits, level one tracing stops.

5. Send the trace results to a specified file location or to stdout.

   If you choose output to a file, the Trace File field displays. Use this field to specify an alternate location and file name for trace output files. The default location and file name is

   path_to_idm_install\export\pipeline\config\WSTrace.log

6. Specify the maximum number of trace files to store (default is 2).

7. Specify the maximum size for each file (default is 512K).

8. Specify whether to write the trace output file as generated (synchronously) or to queue the data and then write it to the trace file (asynchronously).

9. Save your changes.

To Create a New Trace Configuration Object

Use the following steps to create a new trace configuration object:

1. Decide which package or method you want to trace.

   Generally, you specify a resource adapter name or use information that was revealed in an error message.

2. Log in to the Identity Manager Administrator interface and open the System Settings page as described in To Enable Trace from the System Settings Page.

3. On the Systems Setting page, click Show Trace List.

4. When the Identity Manager Trace Configuration page displays, click New.

5. Enable tracing from the Edit Trace Configuration page.

   Choose one of the following options from the Trace Configuration menu:

   • Global. Select to enable tracing for all servers.

   • Server_name. Select a server name to enable tracing for a particular server.

6. Select the Trace Enabled box to enable tracing for this object and configure the remaining parameters on this page as described in To Edit the Default Configuration Object.

7. Save your changes.

To Configure Tracing From Individual Debug Pages

This section describes how to enable tracing from an Identity Manager Debug page.

1. Open a browser and log in to the Identity Manager Administrative interface.

2. Type the following URL:

   http:// host:port /idm/debug/pageName.jsp

   where:

   • host is the local server on which you are running Identity Manager.

   • port is the number of the TCP port on which the server is listening.

   • pageName.jsp is the particular Debug page you want to open.

     For example, to trace adapter classes and methods for a custom adapter, you can open the Edit Trace Configuration page by typing the following URL:

     http://host :port/idm/debug/Show_Trace.jsp

To Enable Tracing From the Identity Manager Console

This section explains how to enable tracing from the Identity Manager console.

1. Set $WSHOME.

   For example, to set the variable to the default installation directory:

   set WSHOME=C:\Program Files\tomcat\webapps\idm

2. Type lh console to open the Identity Manager console from the bin directory.

3. From the console, type trace to see a detailed summary of available trace options, including enable and disable.

   The syntax for this command is:

   trace [ -s server ] $subcommand

How to View Trace Files

By default, Identity Manager sends tracing information to a file named WSTrace#.log that is stored in the path_to_idm_install\export\pipeline\config\ directory. If necessary, you can specify alternate file names and locations when you configure tracing for an object.

Each log file is numbered sequentially, up to the maximum number of files specified on the Edit Trace Configuration page. For example, if you specify a maximum of three files, the files are named WSTrace1.log, WSTrace2.log, and WSTrace3.log.

Use one of the following methods to view these log files:

• If you are sending trace information to a file, open the trace file from the specified location. For example:

  path_to_idm_install \export\pipeline\config\WSTrace2.log

• If you are sending trace information to stdout, open your application server’s stdout file in a text editor to view the trace output logs.

Tracing the Identity Manager Server

Identity Manager is a Java-based product whose executables consist of Java classes grouped as packages. When the code is implemented, many classes can supply trace output.

Tracing the Identity Manager server can provide helpful information such as where a server is failing, having problems, or not running. You can use the Identity Manager Debug pages to enable package-level and method-level tracing in a running Identity Manager server.

Configure Identity Manager to trace at this advanced level only if instructed by Sun Support.

Tracing Adapters

You can use trace information to verify that a resource adapter has started, to verify that all setting changes were saved, and to diagnose problems with the adapter.

This section contains information to help you configure adapter tracing, and is organized into the following topics:

• General Notes About Tracing Adapters

• Instrumenting Code with Trace Points

• Other Tracing Guidelines

General Notes About Tracing Adapters

When you enable tracing for an adapter, you must identify the methods that you want to trace by typing:

com.waveset.adapter.sample.MyResourceAdapter

You must also provide calls to create log entries for any new methods in your custom resource adapter so the adapter can write its resource settings to a log file.

In some cases, you can specify additional logging parameters for the resource instance, such as:

• Maximum Log Archives

• Maximum Active Log Age

• Log File Path

• Maximum Log File Size

• Log Level

• To further debug the synchronization process, you can configure synchronization logging for an ActiveSync adapter. Instructions are provided in the Identity Manager online help.

• Tracing custom Java code can be useful when you are writing your own resource adapter. See Tracing Custom Code for more information.

• You can instrument code with trace points to help assess and resolve problems more effectively. See Instrumenting Code with Trace Points for more information.

• For more information about tracing specific adapter methods, see the Sun Identity Manager 8.1 Resources Reference.

Instrumenting Code with Trace Points

The Identity Manager trace facility enables you to instrument code with trace points in the adapters component (resource adapters). This section contains some guidelines to help you to consistently use trace points to assess and resolve adapter problems.

Some guiding principles for instrumenting code with trace points are as follows:

• Make trace points as useful as possible for troubleshooting in both production and development environments by displaying relevant information.

• Create one entry point and one exit or exception trace point for every significant method.

• Use trace points as efficiently as possible during normal operations and when tracing is enabled by

  • Minimizing the number of trace level checks

  • Minimizing object creation when tracing is enabled

You can specify a trace level for a trace point to control how much information displays. The following table describes each of the trace levels that are defined in the com.sun.idm.logging.TraceManager interface.

Use the com.sun.idm.logging.Trace class to prevent circular dependencies on the com.sun.idm.logging package that implements the com.sun.idm.logging.TraceManager interface.

When adding trace points to code, be aware of the following information:

• Trace arguments to the method and return values from the method.

  If the arguments “print large,” where the text representation is not brief, specify whether the Object is null or not. If the argument or return value is an array or a list, and the value could print large, showing the array size or list size is sufficient. You can use the com.waveset.util.Util.length() utility method to guard against null values.

• To format trace information or to create objects that present trace information in a meaningful way, put an if statement around the trace statements, using the Trace.getLevel method and the isLogging(level) method, so that Identity Manager does not marshal the trace point data unless the specified trace level is enabled.

  When using this construct, specify the Trace.ALWAYS trace level for trace points inside the condition. This trace level prevents an unnecessary check for the trace level when that check has already been performed in the condition.

• If you are tracing an object value that requires a method call to get a printable representation of the object, such as obj.getName(), guard against a null object to prevent a NullPointerException occurring if trace becomes enabled.

  ---

  Note –

  In general, do not use trace points for simple getter and setter methods or for any other trivial methods.

  ---

• When tracing a WSUser object, use the toString() method instead of the getName() method to present the value of the object. The toString()) method is more robust and prints more useful information than the getName() method. Instead of tracing user.getName(), trace user, which calls toString() implicitly.

• If a method or constructor is just a wrapper, where that method or constructor does nothing significant except call another method or constructor with the same name but a different signature, just put trace points into the destination method or constructor.

• If performance is critical and the method is called many times, use information trace points to indicate errors. Do not add entry or exit trace points to the method. Use this option carefully.

• Before adding a trace point, ask yourself if that trace point will provide useful information when determining problems in the field.

The following sections describe specific trace levels in greater detail and provide examples showing how to use trace points in code.

• Using Entry and Exit Trace Points

• Using Information Trace Points

• Using Exception Trace Points

• Using Trace Point Templates

Using Entry and Exit Trace Points

Consider these guidelines before adding entry and exit trace points to your code:

• Use Trace.Level1 entry or exit trace points for all Identity Manager component external interface methods, which are the public methods declared in the resource adapter interface.

• Use Trace.Level2 entry or exit trace points for significant non-public methods including constructors and static methods or methods not included in the Identity Manager component external interface.

• Specify arguments in the trace points.

  If the argument is not a primitive type or java.lang.String, and the argument requires any kind of object creation or method calls, conditionalize the entry trace point so that the formatting work is only done when tracing is enabled.

• When specifying an exit trace point, show the return value if the method is a primitive type or java.lang.String. If the return value requires some sort of formatting, conditionalize the object formatting as specified in these guidelines. Use the following:

  int getLevel() int getLevel (Method) boolean isLogging(level,method) boolean level1 (method) boolean level2 (method) boolean level31 (method) boolean level4 (method)

• If you are tracing information that requires appending several java.lang.Strings together, use java.lang.StringBuffer instead of java.lang.String. Appends are faster using java.lang.StringBuffer. Also, conditionalize as in the preceding bullet item.

• When adding entry or exit trace points for significant constructors, remember that you cannot place a trace point before a this() or super() method call within a constructor. Put the entry trace point immediately after the this() or super() method call.

• Do not use an exit trace point to trace an exception condition. If the method throws an exception, use an exception trace point at the same trace level as the entry or exit trace point just before the exception is thrown. See Using Exception Trace Pointsfor more information.

  If a method contains an entry trace point, that method also must contain a code path that executes either a throwing method, a caught method, or an exit.

Following are some simple entry and exit trace statements. For these examples, assume the following CLASS variables are declared for each statement.

private static final String CLASS = 
"com.waveset.adapter.MyResourceAdapter"; 
protected static Trace _trace = Trace.getTrace();

Example 5–1 Entry Trace Point Example

final String METHOD = methodName; 
_trace.entry(_trace.LEVEL1, CLASS, METHOD);
_trace.entry(_trace.LEVEL1, CLASS, METHOD, user);
if (_trace.level1(CLASS, METHOD)) {    
_trace.entry(_trace.ALWAYS, CLASS, METHOD,
user= + user); 
}

// Show the size of an array argument
// ASSUME: password is an argument to the method
// Note the use of the Util.length() method. It is
// a convenience method that guards against null.
if (_trace.level1(CLASS, METHOD)) {
    StringBuffer sb = new StringBuffer(32);
    sb.append(password length=);
    ab.append(Util.length(password));
    _trace.entry(_trace.ALWAYS, CLASS, METHOD, sb.toString());
}

Example 5–2 Exit Trace Point Example

_trace.exit(_trace.LEVEL1, CLASS, METHOD);

_trace.exit(_trace.LEVEL1, CLASS, METHOD, returnValue);
if (_trace.level1(CLASS, METHOD)) {
    _trace.exit(_trace.ALWAYS, CLASS, METHOD,
      returnValue != null ? returnValue.getName() : returnValue);
}

// Show the size of an array 
String[] accounts = ... 
if (_trace.level1(CLASS, METHOD)) {    
    StringBuffer sb = new StringBuffer(32)
    sb.append(accounts length=);    
    ab.append(accounts.length);    
    _trace.exit(_trace.ALWAYS, CLASS, METHOD, sb.toString()); 
}

Using Information Trace Points

Consider these guidelines before adding information trace points to your code:

• Generally, use Trace.Level3 or Trace.Level4 for information, variable and data trace points. The trace level depends upon the significance, expense, and the verbosity of the information presented.

• Use Trace.Level3 and Trace.Level4 information and variable trace points to trace significant conditions, branches, and other information as they occur in the code path.

• A variable trace point is a convenience method that is the same as an information trace point, except that a variable trace point prints out variable=<variable> instead of just printing the argument value. In addition, this method does not format the information unless the trace level specified in the argument is matched.

• Do not use an information trace point if the method contains an exception condition but does not rethrow the exception. Use the caught method. For an example, see Using Exception Trace Points.

• Use Trace.Level3 and Trace.Level4 data trace points to show information in byte arrays. You can use a lower trace level if the information is appropriate for that trace level and the information presented is brief.

• You can use Trace.Level1 and Trace.Level2 information, variable, and data trace points to trace significant, but brief, information. However, you should rarely use these trace points at these levels.

Following is an example of a simple information trace statement. For this example, assume the following CLASS variables are declared.

private static final String CLASS = 
"com.waveset.adapter.MyResourceAdapter"; 
protected static Trace _trace = Trace.getTrace();

Example 5–3 Information Trace Point Example

_trace.info(_trace.LEVEL3, CLASS, METHOD, Some Message);
WavesetResult result = new WavesetResult(); 
try {   
   someMethod(); 
}  catch(Exception e) {   
   String msg = Some Error Message;   
   WavesetException we = new Waveset(msg, e);   
   _trace.caught(_trace.LEVEL3, CLASS, METHOD, e);   
   result.addException(we); 
}

Using Exception Trace Points

Consider these guidelines before adding exception trace points to your code:

• Use exception trace points in all methods where entry or exit trace points are specified, and in methods that catch and rethrow exceptions. Identity Manager sometimes throws exceptions in which a new exception wraps the original exception. Use the same trace level being used by the entry and exit trace points.

• Use an exception trace point if an exception is created and thrown from the current method.

• Use an exception trace point caught method if an exception is caught and handled. For example, when the exception is not thrown from the current method. This situation typically occurs when Identity Manager catches and handles the exception by placing it in a container, such as a WavesetResult, to be examined later.

• If the exception thrown from the current method is not a checked exception, which is an exception that extends java.lang.RuntimeException or java.lang.Error, the method exit point will not be traced unless other measures are taken. One way to set an exception trace point in this situation is to use a try/catch block around the critical section of the method and rethrow the exceptions that occur. In general, this means that you should use Trace.Level3 and higher when it is critical to know whether a method has succeeded or failed.

Following is an example of a simple exception trace statement. For this example, assume the following CLASS variables are declared:

private static final String CLASS = 
"com.waveset.adapter.MyResourceAdapter"; 
protected static Trace _trace = Trace.getTrace();

Example 5–4 Exception Trace Point Example

try {   
   someMethod(); 
}  catch(Exception e) {   
   _trace.throwing(_trace.ALWAYS, CLASS, METHOD, e);   
   throw e; 
}


try {   
   someMethod(); 
}  catch(Exception e) {   
   _trace.throwing(_trace.ALWAYS, CLASS, METHOD, e);   
   WavesetException we = new WavesetException(Some Message, e);   
   throw we; 
}


if (error) {   
  WavesetException e = new WavesetException(Some Error Message.; 
  _trace.throwing(_trace.LEVEL3, CLASS, METHOD, e);   
  throw e;
}

try {
   someMethod();
}  catch(Exception e) {
   _trace.caught(_trace.LEVEL1, CLASS, METHOD, e);
}
// execution continues.
someOtherMethod();

Using Trace Point Templates

Using trace point templates can help you provide consistent and useful trace points in Identity Manager resource adapters. Identity Manager provides Eclipse templates in the following location:

src/wps/doc/eclipse-trace-templates.xml

To import these templates into Eclipse, select Window -> Preferences -> Java -> Editor -> Templates.

You can create similar templates if you are using Emacs or IDEA.

Other Tracing Guidelines

This section describes some additional tracing guidelines, including the following:

• Tracing Inner Classes

• Tracing Static Initializers

Tracing Inner Classes

Remember to trace significant methods in inner classes. Be sure to declare a final static CLASS variable if there are any trace methods in the inner class.

Example 5–5 Example of Using a CLASS Variable in Inner Classes

private final static String CLASS = 
com.waveset.adapter.SomeResourceAdapter$AdapterInnerClass;

Tracing Static Initializers

In general, do not use the trace facility in static initializers. Identity Manager executes these initializers when the class is loaded, which can occur when the repository is initialized. Use the Debug class to trace anything significant in static initializers.

Tracing Auditor

You can trace the following methods to troubleshoot issues with Identity Auditor:

• com.sun.idm.auditor.policy to trace issues with Audit Scans.

• com.sun.idm.auditor.accessreview to trace issues with Access Reviews.

• com.sun.idm.auditor.report to trace issues with Audit Reports.

• com.sun.idm.auditor.view to trace issues with Auditor Views.

To Enable Tracing:

1. Open a browser and log in to the Administrator interface.

2. Select Configure -> Servers.

3. When the Configure Servers page displays, click the server name in the Server column to edit the settings.

4. On the Edit Server Settings page, click the Scheduler tab.

5. Select the Tracing Enabled box to activate Scheduler debug tracing and write the results to stdout.

6. Save your changes.

Tracing Custom Code

You can use the Identity Manager tracing facility in custom-written code to provide seamless integration when troubleshooting your deployment.

Use the following classes to provide this tracing facility:

• Use com.sun.idm.logging.trace.Trace objects to interface with the tracing subsystem.

• Use com.sun.idm.logging.trace.TraceManager as a factory for these objects.

You can also use these com.sun.idm.logging.trace facilities to record trace output. See the Javadoc^TM for more information about these facilities and classes.

Be aware that the com.sun.idm.logging.trace facilities were not available prior to the Identity Manager Version 6.0 SP1 release. For earlier versions of Identity Manager, you must use the com.waveset.util.Trace class instead.

Tracing Exceptions

Exception logs are stack traces that you can view from the Identity Manager Debug pages or from the config/Waveset.properties file. Trace data does not include all exceptions by default, but exception logging can be an important and informative troubleshooting tool.

Use one of the following methods to enable exception logging:

• Open the Waveset Properties page (debug/Show_WSProp.jsp) in the Identity Manager Administrator interface. Locate the exception.trace key and change the Value to true.

• Open the config/Waveset.properties file in a text editor, and change the exception.trace key value to true. The following figure shows an example exception trace.

  

Identity Manager sends exception logs to stdout on the web application instance, which is often the application server console.

When you are finished, disable exception logging to stop unnecessary output to the application server logs. To disable exception logging, set the exception.trace key value back to false.

Tracing Forms

You can enable tracing to troubleshoot edited forms and to check for expression statement errors within your form fields.

Use either of the following methods to enable tracing:

• Open the Waveset Properties page (debug/Show_WSProp.jsp) in the Identity Manager Administrator interface. Locate the form.trace key and change the Value to true.

• Open the config/Waveset.properties file in a text editor, and change the form.trace key value to true.

Identity Manager reports any problems with form expression syntax to standard output.

The form.trace key is disabled by default because it produces trace information for every field on every page, including the Accounts List page, which affects system performance. Consider using a more targeted form and field tracing method.

When you are finished troubleshooting your forms, remember to disable tracing by changing the form.trace key value back to false.

Global XPRESS tracing might also be helpful while you are developing and updating forms and form processes. Although Global XPRESS tracing produces a large amount of output that affects system performance, this tracing method shows XPRESS output and might expose where problems are happening in your form.

For more information, see

• Tracing Global XPRESS

• Testing Your Customized Form in Sun Identity Manager Deployment Reference

Tracing Global XPRESS

While not generally recommended, you can use global XPRESS tracing to trace any and all XPRESS code, wherever the code is located. For example, you can trace XPRESS code in forms, views, and workflows. The resulting trace shows XPRESS output that can expose potential problems.

XPRESS tracing is disabled by default because it produces a large amount of output, which affects system performance.

See Testing Your Customized Form in Sun Identity Manager Deployment Reference for more information about tracing XPRESS functions.

To Enable Global XPRESS Tracing:

1. Open a command window.

2. Change directories to config/Waveset.properties in the default Identity Manager installation directory.

3. Open the config/Waveset.properties file and edit the xpress.trace line to read:

   xpress.trace=true

4. Save the Waveset.properties file.

5. Restart your application server or reload the Waveset.properties file from the Identity Manager debug pages.

6. Replicate the XPRESS trace output to a file by adding this line to the Waveset.properties file:

   xpress.traceFile=FileName.txt

   xpress.traceFileOnly=true

   When you set xpress.traceFileOnly=true in Waveset.properties, all XPRESS statement evaluations will generate trace messages to a file specified by xpress.traceFile. Otherwise, when xpress.traceFile has a value, trace messages are redirected to both the console and a file.

Tracing PasswordSync

This section describes how to enable tracing for PasswordSync and how to configure tracing in Direct access or JMS modes.

To Enable Trace for PasswordSync

You can use the following methods to configure tracing for Identity Manager’s PasswordSync feature:

• Using the PasswordSync Configuration Tool

• Editing the Registry Keys

Using the PasswordSync Configuration Tool

This section describes how to configure tracing from the PasswordSync Configuration Trace tab.

For more information about installing and configuring PasswordSync, see Chapter 11, PasswordSync, in Sun Identity Manager 8.1 Business Administrator’s Guide.

The first time you run the configuration tool, the wizard does not allow you to configure tracing. Subsequently, when you run the configuration tool, the wizard displays a Trace tab where you can configure tracing.

The following figure shows the PasswordSync Configuration tool Trace tab.

Figure 5–2 Trace Tab in the PasswordSync Configuration Tool

From this tab, you can specify the following:

• Use the Trace Level field to specify the level of detail you want PasswordSync to provide when writing to the trace log. A value of 0 turns tracing off, while a value of 4 shows full detail.

• Use the Max File Size field to specify a maximum size for the log file.

  When the trace file exceeds the size specified in the Max File Size (MB) field, PasswordSync starts a new trace file and appends .bk to the old trace file name. For example, if your trace level is set to 100 Mbytes, and your trace file writes to C:\logs\pwicsvc.log, when the trace file exceeds 100 Mbytes PasswordSync renames the file to C:\logs\pwicsvc.log.bk. PasswordSync then creates a new C:\logs\pwicsvc.log file where new trace file messages are written.

• Use the Trace File field to specify a location for the PasswordSync trace file.

Editing the Registry Keys

To enable additional PasswordSync configuration settings, edit the following PasswordSync registry keys using the PasswordSync configuration tool.

Using PasswordSync configuration tool is the safest method for editing PasswordSync registry keys. Editing these keys directly in the Windows Registry is discouraged.

The PasswordSync registry keys are located in the following location:

HKEY_LOCAL_MACHINE\SOFTWARE\Waveset\Lighthouse\PasswordSync

Other keys are present in this location.

To Collect Logs for the Different Modes

PasswordSync trace logs are the same, whether you are using a direct access mode or JMS mode configuration. However, these trace logs might only provide partial information. You must configure different classes for each configuration to collect logs on the server side, as described in the following sections.

Tracing in Direct Mode

When using PasswordSync with a direct access mode configuration, the trace logs show failures, but not all logged failures are real failures. For example, in some circumstances the view check-in takes a long time, which shows as a failure in the log. You must trace on the server side to see this information.

In Direct mode, PasswordSync talks to the servlet that generates the view to be checked into the repository. You can trace the com.waveset.rpc.GenericMessageHandler class at level 4 to view all phases of password synchronization, from receiving the password change to the response generated and returned to the servlet. Level 4 is the only level that supplies enough detail for troubleshooting.

Tracing in JMS Mode

When using PasswordSync with a JMS mode configuration, the logs only show successful or failed deliveries to the JMS server. From this point on, you must rely on server side logs. JMS tracing is a little more complex.

You can trace the com.waveset.rpc.PasswordSyncHandler class at level 4 to convert the messages generated by the PasswordSync dll into a JMS message and add those messages to the JMS queue. Limited tracing is available in this class, and only level 4 provides enough information to help with troubleshooting.

If PasswordSync successfully delivers the JMS message to the JMS queue, the tracing will not help you find the cause of a problem. The next, and final step is to trace the JMS adapter. See the Sun Identity Manager 8.1 Resources Reference for instructions.

Tracing Rule-Driven Members Caches

You can trace rule-driven members caches and use the results to tune cache properties in Waveset.properties.

If you trace com.waveset.server.RuleDrivenMembersCache at Level 1, the resulting information includes the number of adds, removes, hits to the cache, and so forth. Use this information to evaluate cache sizes and decide whether tuning the cache properties in Waveset.properties is necessary.

You can use the following properties in Waveset.properties to control the rule-driven members cache:

• Use ruledrivenmemberslistcache.size = value to specify the maximum number of object lists to be cached per subject. (Default is 20.)

• Use ruledrivenmemberslistcache.rowlimit = value to specify the maximum total number of objects to be cached per subject. (Default is 100000.)

By default, Identity Manager evaluates the User Members Rule associated with a specified organization and creates a user members list cache containing a dynamic list of users. However, you can also create a user members list cache containing a specified user's set of dynamic member organizations for a given subject. The key to this cache is the object type concatenated with the object ID. For example, the User object type concatenated with the User#ID#Configurator object ID. The value for each key is a list of object groups for which this object is dynamically a member.

To determine whether the object being evaluated is a dynamic member, the cache evaluates the same User Members Rule per organization as that used by the list cache. If the object is a dynamic member, Identity Manager adds that object to the list and then caches the list. Identity Manager caches both empty and non-empty lists to ensure the highest cache hit rate.

For this cache, you can use these properties in Waveset.properties to control memory requirements affecting performance:

• Use ruledrivenmembersobjectcache.size = value to specify the number of member object group lists to be cached per subject. The default value is 100.

• Use ruledrivenmembersobjectcache.rowlimit = value to specify the maximum total number of member object groups to cache per subject. The default value is 100000.

Tracing Sun Identity Manager Service Provider Delegated Administration

Enabling Identity Manager tracing at Method/Class Level 2 for the following classes allows you to trace authorization flows when listing or accessing Sun Identity Manager Service Provider (Service Provider) users and when AdminRoles are dynamically assigned when Service Provider users login.

• Trace com.sun.idm.idmx.view.IDMXBrowseViewer when searching for Service Provider users.

• Trace com.sun.idm.idmx.view.IDMXUserViewer when creating, editing, or deleting Service Provider users.

• Trace com.sun.idm.idmx.api.IDMXServerUtils when using both of the preceding classes.

• Trace com.waveset.security.authn.WSSPELoginModule when you are logging in as a Service Provider user.

You configure Service Provider tracing from the Identity Manager debug pages. If necessary, review Tracing the Identity Manager Server for instructions.

Tracing Reconciliation

If you are having problems with a reconciliation task, you can use the standard tracing facility on com.waveset.task.Reconciler to trace the Reconciler.

Use either of the following methods to enable tracing:

• Open the Waveset Properties page (debug/Show_WSProp.jsp) in the Identity Manager Administrator interface. Locate the exception.trace key and change the Value to true.

• Open the config/Waveset.properties file in a text editor, and change the exception.trace key value to true.

You can also enable tracing from the System Settings page and trace the following reconciliation methods, at the Method/Class trace Levels noted, to view useful debugging information.

Remember, the higher the tracing level, the larger the trace file. Also tracing all of these methods at the same time will create a very large trace file.

When you are finished troubleshooting, remember to disable tracing by setting the exception.trace key value back to false.

Tracing the setRepo Command

If you see errors while you are using the setRepo command to configure the Identity Manager repository, use the following flags to isolate and debug the problem:

-Dtrace.enabled=true
-Dtrace.level.com.waveset.repository.AbstractDataStore=2
-Dtrace.level.com.waveset.repository.DefaultTypeHandler=4
// Use one of the following based on your repository type
-Dtrace.level.com.waveset.repository.OracleDataStore=4
-Dtrace.level.com.waveset.repository.SqlServerDataStore=4
-Dtrace.level.com.waveset.repository.MysqlDataStore=4
-Dtrace.level.com.waveset.repository.DB2DataStore=4

Identity Manager sends output from the setRepo command to the default $WSHOME/config/WSTrace.log file.

Tracing SPML

This section describes methods for enabling trace for SPML Version 1.0 and SPML Version 2.0.

To Enable Tracing for SPML 1.0

SPML 1.0 provides the following options for turning on trace output so you can log Identity Manager’s SPML traffic and diagnose problems.

Method 1: Enable the setTrace Method

You can use the setTrace method, provided by the SpmlClient and LighthouseClient classes, to enable tracing for SPML 1.0.

When you enable this setTrace method, the XML for the request sent by the client and the XML for the response received from the server are printed to the client console as they are sent and received.

The setTrace method takes a Boolean argument. For example:

SpmlClient client = new SpmlClient();
 client.setURL("http://localhost:8080/idm/spml");
 client.setTrace(true);

Method 2: Initializing the org.openspml.server.SOAPRouter Servlet

You can enable tracing when initializing the org.openspml.server.SOAPRouter servlet, which is a third-party, open source class from the OpenSPML organization. This servlet controls the output of RPC traffic information for the servlet handling SPML requests.

To enable this tracing method, add the following to the WEB-INF/web.xml file:

<servlet>
    <servlet-name>rpcrouter2</servlet-name>
    <display-name>OpenSPML SOAP Router</display-name>
    <description>no description</description>
    <servlet-class>
        org.openspml.server.SOAPRouter
    </servlet-class>
    <init-param>
        <param-name>trace</param-name>
        <param-value>true</param-value>
    </init-param>
    ...
  </servlet>

The following is sample output for an SPML 1.0 trace:

SpmlClient: sending to http://example.com:8080/idm/servlet/rpcrouter2
<spml:addRequest xmlns:spml='urn:oasis:names:tc:SPML:1:0'
xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core'>
  <spml:operationalAttributes>
    <dsml:attr name='session'>
     

<dsml:value>session token</dsml:value>

    </dsml:attr>
  </spml:operationalAttributes>
  <spml:identifier type='urn:oasis:names:tc:SPML:1:0#GUID'>
    <spml:id>suetonius</spml:id>
  </spml:identifier>
  <spml:attributes>
    <dsml:attr name='objectclass'>
      <dsml:value>person</dsml:value>
    </dsml:attr
    <dsml:attr name='password'
      <dsml:value>password</dsml:value>
    </dsml:attr>
    <dsml:attr name='gn'>
      <dsml:value>Suetonius</dsml:value>
    </dsml:attr>
    <dsml:attr name='sn'>
      <dsml:value>Tranquillus</dsml:value>
    </dsml:attr>
    <dsml:attr name='email'>
      <dsml:value>twelve@example.com</dsml:value>
    </dsml:attr>
  </spml:attributes>
</spml:addRequest>


SpmlClient: received
<?xml version='1.0' encoding='UTF-8'?>
<SOAP-ENV:Envelope
  xmlns:SOAP-ENV='http://schemas.xmlsoap.org/soap/envelope/'>
<SOAP-ENV:Body>
<spml:addResponse xmlns:spml='urn:oasis:names:tc:SPML:1:0'
xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' result='urn:oasis:names:tc:
SPML:1:0#success'>
  <spml:operationalAttributes>
    <dsml:attr name='session'>
     
<dsml:value>session token</dsml:value>
    </dsml:attr>
  </spml:operationalAttributes>
  <spml:identifier type='urn:oasis:names:tc:SPML:1:0#GUID'>
    <spml:id>suetonius</spml:id>
  </spml:identifier>
</spml:addResponse>
/SOAP-ENV:Body>
</SOAP-ENV:Envelope>

For more information about the SOAP rpcrouter servlet, refer to your OpenSPML Toolkit documentation.

Method 3: Pass the trace Operational Attribute

You can enable tracing for an individual SPML RPC request by passing a trace operational attribute to the RPC request on the server side.

Tracing occurs during servlet initialization, and it controls how information is output for the RPC traffic of a servlet handling SPML Version 1.0 requests. For example, the trace prints the raw XML that is sent back and forth on whatever the System.out is for that servlet (which is a function of the Application container). For example:

AddRequest ar = new AddRequest();
 ar.setOperationalAttribute("trace", "true");

When you use the trace attribute, how the attribute affects server operation is vendor-specific. Currently, Identity Manager prints the raw request and response data to the server console, which is useful if the client application is not associated with a console window.

For more information consult your OpenSPML Toolkit product documentation.

To Enable Tracing for SPML 2.0

SPML 2.0 provides the following options for turning on trace output so you can log Identity Manager’s SPML traffic and diagnose problems.

Method 1: Using the org.openspml.v2.transport.RPCRouterServlet Servlet

As with SPML 1.0, you can enable tracing for SPML 2.0 when initializing the org.openspml.v2.transport.RPCRouterServlet class, which controls the output of RPC traffic information for the servlet handling SPML 2.0 requests.

To enable this tracing method, add the following to the WEB-INF/web.xml file:

<servlet>
    <servlet-name>openspmlRouter</servlet-name>
    <display-name>OpenSPML SOAP Router</display-name>
    <description>A router of RPC traffic - nominally SPML 2.0 over SOAP</description>
    <servlet-class>
        org.openspml.v2.transport.RPCRouterServlet
    </servlet-class>
    <init-param>
      <param-name>trace</param-name>
      <param-value>true</param-value>
    </init-param>
    ...
  </servlet>

The following example illustrates output from an rpcrouter servlet trace:

RPCRouterServlet:
<?xml version='1.0' encoding='UTF-8'?><SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header/>
<SOAP-ENV:Body><lookupRequest
xmlns='urn:oasis:names:tc:SPML:2:0' requestID='random name' executionMode='synchronous'
returnData='everything'>
<openspml:operationalNameValuePair xmlns:openspml='urn:org:openspml:v2:util:xml' name='
session'value=session token'/>
<psoID ID='random name' targetID='spml2-DSML-Target'/>
</lookupRequest>
</SOAP-ENV:Body></SOAP-ENV:Envelope>

RPCRouterServlet:  response:
<?xml version='1.0' encoding='UTF-8'?>
<SOAP-ENV:Envelope

  xmlns:SOAP-ENV='http://schemas.xmlsoap.org/soap/envelope/'>
<SOAP-ENV:Body>
<lookupResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='failure' requestID='random 
name'error='noSuchIdentifier'>
<openspml:operationalNameValuePair xmlns:openspml='urn:org:openspml:v2:util:xml' 
name='session'

value=session token/>
<errorMessage>Item User:random name was not found in the repository, it may have been 
deleted in another session.</errorMessage>
</lookupResponse>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

For more information see Method 2: Initializing the org.openspml.server.SOAPRouter Servlet, in the To Enable Tracing for SPML 1.0 section.

Method 2: Using the SPML Access Log

SPML 2.0 has a simple text access log that can be a useful troubleshooting tool. This log is always available and it enables you to view information, such as what kind of requests have been received, how long it took to process those requests, and whether the requests were successful without having to enable tracing.

Instructions for configuring this SPML text access log are provided in Configuring SPML Tracing in Sun Identity Manager 8.1 Web Services

Tracing the Task Scheduler

If a Scheduler task is having problems, you can use the standard tracing facility on com.waveset.task.Scheduler to trace the task scheduler. The output shows detailed information about task scheduling.

To Enable Tracing:

1. Open a browser and log in to the Administrator interface.

2. Select Configure -> Servers.

3. When the Configure Servers page displays, click the server name in the Server column to edit the settings.

4. On the Edit Server Settings page, click the Scheduler tab.

5. Select the Tracing Enabled box to activate Scheduler debug tracing and write the results to stdout.

6. Save your changes.

   ---

   Note –

   • In a clustered environment, tracing occurs on each instance.

   • See Tracing the Identity Manager Server for more information about defining and editing trace configuration objects, and about viewing trace files.

   ---

Tracing Workflows

Enabling workflow tracing can help you resolve problems with workflow activities and understand the workflow process.

• In a clustered environment, tracing occurs on each instance.

• To debug workflows in a multiple server deployment environment, consider shutting down all but one server. If all servers are running, you cannot determine which server in the environment is executing the workflow, which causes troubleshooting problems.

To Enable Tracing for Workflows

Use the following steps to enable workflow tracing:

To Enable Workflow Tracing:

1. Open a browser and log in to the Identity Manager Administrator interface.

2. From the System Setting page, choose Configuration from the List Objects Type menu.

3. Click the List Objects button.

4. When the List Objects of type: Configuration page displays, scroll down the list of objects to locate the System Configuration object and click the edit link.

5. When the Checkout Object: Configuration, #ID#CONFIGURATION:SYSTEMCONFIGURATION page displays, you can edit any of the following workflow options in the SystemConfiguration object:

   ---

   Note –

   Typically, you enable only one option, but it is possible to enable more than one option at a time.

   These attributes are not dependent on each other. You can turn on one type of trace while the other types are turned off.

   ---

   • Specify workflow.consoleTrace=true to redirect workflow trace messages to the application server console, which can be useful when workflows are terminating due to a fatal exception because this attribute prints more trace output than workflow.fileTrace. (Default value is false.)

   • Specify workflow.fileTrace=PathtoFileto redirect workflow trace messages to a file that is easy to read. This attribute does not have a value by default.

     <Attribute name=’fileTrace’/>

     Add a value tag to the workflow.fileTrace attribute and, using Unix-style forward slashes, enter the path to a log file. Identity Manager stores relative pathnames relative to the application server’s installation directory. For example:

     • On Windows. <Attribute name=’fileTrace’ value=’C:\mydir\workflow.txt’/>

     • On Solaris/UNIX. <Attribute name=’fileTrace’ value=’/mydir/workflow.txt’/>

   • Specify workflow.traceLevel=tracingLevel to specify the level of workflow tracing you want to see.

     Specify workflow.Trace=true to trace workflow processing. You must restart the application server to start tracing with this option. Identity Manager stores the trace results in the task’s WavesetResult object. Use this tracing option when file system access is unavailable. (Default value is false.)

   • Trace messages in the task’s WavesetResult. (Default value is 1.)

   Specifying workflow.Trace=true appends trace messages into one long, unformatted string that is difficult to read. Use this option only when you do not have access to the file system.

   ---

   Note –

   With the first two options, you might lose some of the workflow trace if a fatal exception occurs.

   ---

6. Save the SystemConfiguration object.

7. Restart your application server, or reload the SystemConfiguration object from the Identity Manager debug area.

To Enable Tracing for a Designated Workflow

Use one of the following methods to enable tracing for a designated workflow:

• Editing the WFProcess definition. To enable trace unconditionally for a particular process, edit the XML of the TaskDefinition object by adding the trace=’console ’ attribute to the <WFProcess> element.

• Editing the workflow variable. Gives you more control over the timing of tracing by using a workflow variable. The workflow engine will look for a top-level workflow variable named trace.

The following example shows how to trace a workflow variable.

Example 5–6 Tracing a Workflow Variable

<Variable name=’trace’>
   <cond><eq><ref>accountId</ref><s>jfaux</s></eq>
     <s>workflowTrace.txt</s>
   </cond>
 </Variable>

The trace variable turns tracing on only if the workflow is operating on a user named jfaux. You could also specify trace in a form field to control tracing interactively.

In this example, the trace output is written to the workflowTrace.txt file.

Locating Version Information

You can use one of the following methods to get information about the Identity Manager version you are currently using:

• Hover your cursor above the Help button in the upper right corner of the Identity Manager application window. A pop-up is displayed that contains the version number.

• Open the Waveset.properties file, and check the information provided at the top of the file.

Use the Identity Manager System Properties page (/debug/SysInfo.jsp) to get information about which JVM versions, and other system software versions, you are currently using.

Tracing the Identity Manager Gateway Objects and Activities

This section describes how to trace objects and activities in Sun Identity Manager Gateway, the information is organized as follows:

• How to Configure Tracing from the Gateway Debug Page

• How to Configure Tracing for the PowerShellExecutor.dll Add-On

• How to Capture Dr. Watson Logs

• When viewing or editing a Gateway trace file, use Notepad to avoid file restrictions.

• When you start the Gateway, the program appends new trace entries to the trace file instead of deleting entries. To locate the point at which the Gateway trace entries begin, look for a Gateway version string.

• The Gateway version is output in the trace automatically when you start the Gateway. You can also type gateway -v from the command line to get the version.

How to Configure Tracing from the Gateway Debug Page

You can enable tracing from the Gateway Debug page (Gateway.jsp) or from the command line to debug problems with Windows accounts on Identity Manager.

Instructions are provided in the following sections:

• From the Gateway Debug Page

• From the Command Line

From the Gateway Debug Page

Enable tracing from the Gateway Debug page (Gateway.jsp) if you cannot access the Gateway. You can specify and retrieve Gateway trace files from this debug page.

To Enable Tracing:

1. Log in to the Identity Manager Administrator interface.

2. Type the following URL in to your browser to open the Gateway Debug page:

   http://host :port/idm/debug/Gateway.jsp

3. Choose a resource to trace from the Gateway Resource list.

4. If necessary, modify existing settings.

   Click the following buttons to modify the settings:

   • Get Version. Returns the Gateway version and the operating system of the machine on which you are running the Gateway.

   • Get Trace File. Returns the contents of the trace file.

   • Get Trace Parameters. Returns the path of the trace file, the trace level, and the maximum size of the trace file.

   • Set Trace Parameters. See To Create a New Trace Configuration Object for information about these options.

   • Get Loaded Modules. Returns the load addresses of modules (DLLs) being used by the Gateway.

   The Get Loaded Modules list consists of load addresses, followed by module names and only includes loaded modules. The list does not include delay-loaded modules that have not been called.

   The Get Loaded Modules option only supports Active Directory and Domino.

From the Command Line

Enabling trace from the command line is useful if you want a wider range of options.

To Enable Tracing:

1. Open a command window.

2. Start the Gateway, specifying the necessary trace command arguments.

   The following table describes the Gateway tracing command line arguments.

   Argument	Description
   -f	Specify the path to the trace file
   -l	Specify the level of tracing:  Level 0. Disables tracing. (Default) Level 1. Traces the flow of control between components and generally defines a low-detail trace point that includes entry and exit from high-level functional methods. Level 2. Generally defines a medium-detail trace point that includes entry and exit from every method, and information and data trace points for high-level functional methods. Level 2 adds the flow of control within each component, major decision points, and items of information. Level 3. Generally defines a high-detail trace point that includes entry and exit from every method, information and data trace points for high-level functional methods, and significant subroutines. Level 3 adds lower-level decision points and items of information. Level 4. Generally defines a hyper-detail trace point that includes everything traced in the other trace levels. Level 4 traces at a very low level and provides a level of detail that is seldom needed but might be useful in characterizing complex behaviors of some components. NOTE: Not all components support level 4. Trivial methods, such as getters and setters, generally do not have entry or exit trace points because they add overhead.
   -m	Specify the maximum trace file size in kilobytes  When the trace file reaches -m Kbytes, Identity Manager closes the current trace file, deletes any existing back-up files, renames the current trace file to the name specified by the -f argument with .bk appended, and opens a new trace file with the -f argument name. For example, if you specified -f beeble.trc on the command line, the following two files result after -m Kbytes are recorded: beeble.trc.bk beeble.trc Where beeble.trc contains the most recent traces.

   Usage: gateway -f name -l -m

   For example:

   cd %WSHOME%\bin\winnt gateway -d -p 11319 -f %CD%\gateway.trc -l 2 -m 500

   The preceding invocation starts the Gateway with the following characteristics:

   • -d – Use regular application (not a service)

   • -p 11319 – Use port 11319

     You must configure this port for Gateway resources from the Identity Manager resource configuration. For example, for an Active Directory resource

   • -f %CD%\gateway.trc – Directory to which the trace output is written. Identity Manager writes the trace output to a text file in this directory.

   • -l 2 – Output level 2 of Gateway tracing.

   • -m – Maximum size in Kilobytes of trace log file.

   ---

   Note –

   If specified, Identity Manager saves -f, -l, and -m values in the registry, so the next time you run Gateway from the command line or as a service, the same values are used.

   Identity Manager sends the Gateway trace output to the console and to a trace file.

   ---

How to Configure Tracing for the PowerShellExecutor.dll Add-On

The PowerShellExecutor.dll is an add-on that implements communication between the gateway and Microsoft PowerShell. The PowerShell is used to manage Exchange Server 2007 accounts. This add-on cannot share tracing facilities with the rest of the gateway and provides a similar stand-alone tracing facility as the rest of the gateway.

The trace configuration for the PowerShellExecutor is stored in the same registry key as the other gateway registry keys:

HKEY_LOCAL_MACHINE\Software\Waveset\Lighthouse\Gateway

You create this base key when you configure tracing through the Identity Manager debug pages or when you start the gateway with trace command arguments.

On shut down, the gateway writes the current PowerShellExecutor settings for the tracing to the registry. These settings include:

• traceFileName

  Content. File name for the trace output (registry type REG_SZ)

  Default. " "

  Name of the trace file to generate for the PowerShellExecutor tracing. Where the name:

  • Can be a fully qualified path, including the filename

  • Cannot end in a slash (\)

    The full path, except the file, provided in the traceFileName must exist.

    If configured, log rotation adds a timestamp to the configured filename after rotation, when the file is no longer active. This timestamp displays in the following format:

    yyyyMMddHHmmss

• traceLevel

  Content. Trace level (registry type REG_DWORD)

  Default. 0 (no tracing)

  Allowed. 0–4

  This key is shared with the rest of the gateway. The whole gateway always provides tracing at the same level.

• traceMaxSize

  Content. Maximum file size in bytes (registry type REG_DWORD or REG_QWORD)

  Default. 100000 bytes

  Minimum. 100000 bytes

  Tracing text is written as UTF–8 encoded text with a byte order mark to make it portable to other systems.

• traceMaxFiles

  Content. Number of trace files (registry type REG_DWORD)

  Default. 2

  Minimum. 1

  This setting controls the number of trace files to keep on the system. Setting the maximum number of files to keep to 1, causes the file to be overwritten when the maximum size is reached. The oldest file, based on last write time, is removed when the maximum number of files is reached.

• traceConfigInterval

  Content. Time out in milliseconds (registry type REG_DWORD)

  Default. 300000 ms (5 minutes)

  Minimum. 60000 ms (1 minute)

All trace settings are reread from the registry based on this timeout value. In a production environment, consider setting this value to a large value, such as 24 hours, to minimize overhead.

How to Capture Dr. Watson Logs

If the Gateway encounters a serious problem and exits abnormally, you can send the resulting Dr. Watson logs to Sun Support for analysis.

You must have administrator privileges on the system to view these logs.

To Capture a Dr. Watson Log:

1. Open the Windows Event Viewer.

2. Open the application log.

3. Look for an event with DrWatson source.

4. Open the event to view detailed information.

5. Ensure that the Bytes option is selected for Data.

6. Right-click in the display dialog and choose Select all from the menu.

7. Type Ctrl-C to copy the information.

8. Paste the information into Notepad and save the file.

9. Send the file in an email to Sun Support with a detailed description of the problem. Be sure to indicate which version of the Identity Manager Gateway you are running.

Troubleshooting and Fixing Common Problems

Use the information provided in the following sections to help diagnose and fix problems you might encounter as you work with Identity Manager:

• Working with Debugging Tools

• Debugging Errors Displayed in the Browser

• Troubleshooting Adapters

• Troubleshooting Auditor

• Troubleshooting ClassNotFound Exceptions

• Troubleshooting Form Problems

• Troubleshooting the Gateway

• Troubleshooting Java Code Problems

• Troubleshooting Low Memory Conditions

• Troubleshooting PasswordSync Problems

• Troubleshooting Reconciliation Problems

• Troubleshooting Repository Connection Problems

• Troubleshooting Server-Related Problems

• Troubleshooting Service Provider Problems

• Troubleshooting an SPML Configuration

• Troubleshooting Upgrades

For additional troubleshooting information, including the Identity Manager FAQ, go to the following URL:

https://sharespace.sun.com/gm/document-1.26.2296/IdM_FAQ.html?

You must sign up for a Share Space ID to access information provided on this site.

Working with Debugging Tools

You can use several different debugging tools to help identify and fix problems in your Identity Manager deployment. These tools include:

• Using the Identity Manager Debug Pages

• Using Identity Manager IDE

• Using Identity Manager System Monitoring

• Working With Adapter Logs

• Debugging with DTrace

• Debugging with JConsole

Using the Identity Manager Debug Pages

You can use Identity Manager Debug pages to help identity and fix problems in your deployment. For example, you can enable or disable tracing for various activities and objects, collect statistical information, verify that processes are running, or investigate bottlenecks and memory problems.

The following table describes the most commonly used Debug pages and their actual .jsp file names.

For a comprehensive list of all Identity Manager Debug pages, open a command window and list the contents of the idm/debugdirectory.

See Working With Identity Manager Debug Pages for more information about these Debug pages.

To Access Individual Identity Manager Debug Pages

Before You Begin

You must have the Debug capability to access and execute operations from the Identity Manager Debug pages. If you do not have the Debug capability, an error message results. Administrators and the Configurator are assigned the Debug capability by default.

1. Open a browser and log into the Administrator interface.

2. Type the following URL to open the System Settings page:

   http://host:port/idm/debug

   where:

   • host is the local server on which you are running Identity Manager.

   • port is the number of the TCP port on which the server is listening.

     From this page, you can enable or disable tracing for various Identity Manager activities and objects and use the information displayed on these pages to troubleshoot problems in your deployment.

     Some Debug pages are not linked to the System Settings page, and you must type the page’s .jsp file name to open the page. For example:

     http:// host:port/idm/debug/pageName.jsp

     Where pageName.jsp is the particular Debug page you want to open.

Using Identity Manager IDE

The Sun^TM Sun Identity Manager Integrated Development Environment (Identity Manager IDE) is Java application that enables you to view, customize, and debug Sun Identity Manager (Identity Manager) objects in your deployment.

Specifically, the Identity Manager IDE provides a graphical Debugger that you can use to debug Identity Manager forms, rules, and workflows. You can use this Debugger to set breakpoints and watches, step through code, examine and modify variables, examine classes and the callstack, follow threads, and run multiple sessions.

Instructions for installing and configuring the Sun Identity Manager Integrated Development Environment (Identity Manager IDE) are now available from the following URL:https://identitymanageride.dev.java.net.

Using Identity Manager System Monitoring

You can configure Identity Manager system monitoring to track system events. System monitoring collects and aggregates statistics at various levels to present a real-time view of system events, based on your specifications.

Viewing this information in dashboard graphs enables you to quickly assess system resources, view abnormalities, understand historical performance trends, and interactively isolate problems before looking at audit logs. Although dashboards do not provide as much detail as audit logs, dashboards can indicate where to look for problems in the logs.

For more information about dashboards and system monitoring, see Chapter 8, Reporting, in Sun Identity Manager 8.1 Business Administrator’s Guide.

Working With Adapter Logs

Adapter logs capture information about the adapter that is currently processing. You can use this information to monitor the adapter’s progress and to diagnose and debug adapter problems.

You must enable tracing and identify the methods for which tracing is requested before any logging can occur. Also, your customized adapter must include calls that create log entries for new methods.

Nearly every adapter has its own log file, path, and log level. You can specify the level of detail captured by the adapter log, along with these other values in the Logging section of the Synchronization Policy for the appropriate Identity Manager or Service Provider user type.

For more information about using adapter log files as a debugging tool, see Troubleshooting Adapters.

Debugging with DTrace

DTrace is a comprehensive, dynamic tracing framework for the Solaris operating environment. DTrace provides more than 30,000 probes into your production system and integrates user- and kernel-level tracing. You can use DTrace to monitor JVM activity. This facility also allows you to use the D language (similar to C or awk) to trace arbitrary data and expressions.

Debugging with JConsole

The Java Monitoring and Management Console (JConsole) is a Java Management Extension (JMX) technology-compliant graphical management tool bundled with JDK 5 (and later). JConsole connects to a running JVM and gathers information from the JVM MBeans in the connected JMX agent.

For example, you can use JConsole to

• Detect low memory

• Enable or disable garbage collection

• Enable or disable verbose tracing

• Detect deadlocks

• Control Identity Manager log levels

• Access information about operating systems resources (Sun’s platform extension)

• Monitor and manage MBeans

• View information about the JVM and monitored values, threads running on the application, and class loading

For more information about JConsole, see the article titled, Using JConsole to Monitor Applications. You can view this article from the following URL:

http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

Debugging Errors Displayed in the Browser

If a red error message displays in the Identity Manager interface after you have performed an action, you might be able to view more complete information and further analyze the error by viewing and saving the page source.

To view the page source

• If you are using Internet Explorer, select View -> Source from the menu bar.

• If you are using Netscape^TM, select View -> Page Source from the menu bar.

If you still need help resolving the problem,

1. View the page source, and then select File -> Save to save the file to your system.

2. Locate the error in your saved file.

3. Send the error information, the URL from the page where the problem occurred, and a description of the problem in an email to Sun Support for resolution assistance.

Troubleshooting Adapters

To troubleshoot an adapter, review the adapter’s log file. Almost all adapters write their resource settings to a log file, and you can use this information to confirm that the adapter started and that all setting changes have been saved.

You must enable tracing and identify the methods for which tracing is requested before any logging can occur. Also, your custom adapter must include calls that create log entries for new methods.

If necessary, review Tracing the Identity Manager Server for instructions about how to enable tracing.

Most adapter log files are located in the $WSHOME/config directory and they are named WSTrace1.log.

Active Sync-enabled adapters that make log calls to the ActiveSyncUtil instance create a log file or set of log files in a directory specified by the Log File Path resource attribute. Be sure to check these log files for additional Active Sync-related log entries.

The information in this section is organized as follows:

• To Debug an Adapter

• To Debug LoginConfig Changes

• To Debug Adapter Connection Problems

To Debug an Adapter

Follow these general steps to debug your custom adapter.

1. Create a test program for your adapter, and be sure this Java file performs the following basic functions:

   1. Create a new resource

   2. Create a user

   3. Get a user

   4. Update a user

   5. Delete a user

   6. Perform create, get, update and delete operations on multiple users

      ---

      Note –

      A sample test file (SkeletonResourceTests.java) is provided in the /REF directory on your installation CD.

      ---

2. Set an appropriate logging level for debugging.

   For example, for the first debugging pass, increase the logging level to 4 (maximum debugging output), set the log file path, and specify a maximum file size.

   When you start the adapter, all of the resource settings are written to the log file. You can use this information to validate that the adapter started and that all setting changes were saved.

3. Compile and test your adapter.

   • To compile the test program, open a command window and enter the javac -d . test/filename.java command. This command creates the class file in the appropriate com/waveset/adapter/test directory.

   • To test your new adapter using this class file, be sure that your compiled adapter is in the com/waveset/adapter directory and use the following command to run the adapter:

     java– D waveset.home=path com.waveset.adapter.test. MyResourceAdapter

4. Create an HTML help file for your resource.

   ---

   Note –

   • Example help files are supplied in the idm.jar file located in the com/waveset/msgcat/help/resources directory.

   • See Sun Identity Manager Deployment Reference for information about how to include online help with the application.

   ---

5. (For Active Sync-enabled adapters only) To reset synchronization on the last resource, delete the XmlData SYNC_resourceName object.

6. Read the error log and modify the adapter.

7. Reset the logging level.

   For example, specifying Level 2 debugging yields information about the adapter settings and any errors, but limits the amount of log detail to a manageable level.

8. Before starting Identity Manager, you must identify the new adapter in the $WSHOME/config/Waveset.properties file by placing the adapter name under the resource.adapters entry or Identity Manager cannot recognize the adapter.

9. Install your adapter and its associated help file into Identity Manager.

   ---

   Note –

   Before Identity Manager can recognize an instance of a new adapter in the display, you must create a new resource of that type from the List Resource page.

   From this page, select New -> new adapter and use the Resource Wizard to create the new adapter.

   ---

10. Use Identity Manager to create a resource and a user on that resource.

    ---

    Tip –

    When troubleshooting an Active Sync-enabled adapter, if you edit the XmlData SYNC_resourceName object to remove the MapEntry for the Active Sync synchronization process from the Debug page, the adapter starts over from the first detected change.

    If you used the IAPI event, you must set the Property() method to store synchronization state for the resource, such as a last change processed value. Setting this method is very useful for troubleshooting adapters. You can set the adapter to run and ignore past changes. Subsequently, you can modify the adapter and see your change results in the adapter log file.

    ---

    If your resource is an Active Sync resource, you might see additional information if you enable logging on the resource edit page. Set the logging level (0-4) and the file path where the log file will be written (as resource_name.log).

11. (For Active Sync-enabled adapters only) Restart synchronization for the last resource.

To Debug LoginConfig Changes

To debug LoginConfig-related changes to your adapter, you must

1. Enable trace for the selected files and trace the following classes at Method/Class Level 1 trace:

   • com.waveset.security.authn.WSResourceLoginModule

   • com.waveset.session.LocalSession

   • com.waveset.session.SessionFactory

   • com.waveset.ui.LoginHelper

   • com.waveset.ui.web.common.ContinueLoginForm

   • com.waveset.ui.web.common.LoginForm

2. Test Single Sign-On (SSO) pass-through authentication login through Telnet as follows:

   1. After correctly configuring the SSO login module, telnet directly to the HTTP port and send an HTTP request to login.jsp.

   2. Paste the following request, which contains an SSO login module that looks for the sm_user HTTP header, into your telnet session:

      HEAD /idm/login.jsp HTTP/1.0 Accept: text/plain,text/html,*/* Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 Host: LOCALHOST sm_user: Configurator

      A trace displays to indicate that your user has logged in correctly. For example:

      2003.07.08 14:14:16.837 Thread-7 WSResourceLoginModule#checkForAuthenticatedResourceInfo() Found authenticated resource accountId, ’Configurator@Netegrity SiteMinder’on Identity Manager user ’Configurator’. null null 2003.07.08 14:14:16.837 Thread-7 WSResourceLoginModule#checkForAuthenticatedResourceInfo() Exit null null 2003.07.08 14:14:16.837 Thread-7 WSResourceLoginModule#login() Exit, return code = true null null 2003.07.08 14:14:16.847 Thread-7 LocalSession#login() Login succeeded via Netegrity SiteMinder null null 2003.07.08 14:14:16.847 Thread-7 LocalSession#login() Overall authentication succeeded null null 2003.07.08 14:14:16.897 Thread-7 LocalSession#checkIfUserDisabled() Entry null null 2003.07.08 14:14:16.897 Thread-7 LocalSession#checkIfUserDisabled() Exit null null 2003.07.08 14:14:16.927 Thread-7 LocalSession#login() Exit null null

To Debug Adapter Connection Problems

This section describes methods for debugging some common adapter connection problems.

The topics in this section are organized as follows:

• Adapter Authentication Problems

• Active Sync Adapter Problems

• Domino Gateway Adapter Problems

• Mainframe Host Adapter Problems

• PeopleSoft Adapter Problems

• SAP Adapter Problems

• UNIX Adapter Problems

Generally, you can identify adapter connection issues by tracing the adapter class com.waveset.adapter.adapter_classname. For example:

com.waveset.adapter.ADSIResourceAdapter

If necessary, review instructions for enabling trace in Tracing the Identity Manager Server.

Adapter Authentication Problems

Some common authentication problems include

• Missing authentication properties

  You must include a property name for the specified DataSource type in the set of names output in the trace.

• An Identity Manager user is missing a matching resource account

  If authentication succeeds for the resource adapter, but an exception occurs indicating that no Identity Manager user could be found with a matching resource account ID, be sure that the resource accountId associated with the user is the same as the accountId returned by your resource adapter’s authenticate method.

  To verify the Identity Manager user’s resource accountId, review Identity Manager’s Edit Trace Configuration page (debug/Show_Trace.jsp). If a mismatch exists, change the content of the name being returned by your authenticate method or change your resource’s ID template. The template must generate a resource accountId that matches the accountId being returned by the authenticate method.

Active Sync Adapter Problems

The most common problems with custom Active Sync adapters are form-related. These errors generally occur because you have not provided necessary information, such as password or email information, in a required field.

Identity Manager prints form validation errors to the adapter log after the final XML of the view. For example:

20030414 17:23:57.469: result from submit (blank means no errors):
 20030414 17:23:57.509: Validation error: missing required field password

Identity Manager also prints all messages to the adapter log. These messages include account creation and update times, adapter errors, and a summary of the schema map data.

Active Sync resource adapters store information about the last change processed in the SYNC.resourceName XMLData object.

Domino Gateway Adapter Problems

Following are some common Domino gateway and adapter configuration errors and instructions for fixing these problems:

• If an error message states the ’New Domino Gateway’ resource is not accessible and the connection is refused, try stopping and restarting the Identity Manager Gateway.

• If an error message states no ID file name is specified and the path to the userID file is set incorrectly, specify a target location for the userID file and edit the resource adapter to set this attribute to a correct path. Typically, the target location for the userID file is the directory into which you installed the Gateway.

• If an error message states you are not authorized to use the server, you have not set the correct access permissions for the ID file. Specify the correct permissions for this file and retry.

Mainframe Host Adapter Problems

When RACF, ACF2, or TopSecret host adapters fail to reuse or cache connections, users are forced to log in frequently, which negatively impacts performance. Generally, the cache timeout setting causes this problem.

To check the cache timeout setting, trace Identity Manager’s adapter connection pool as follows:

1. From Identity Manager’s Edit Configuration Object page, trace the com.waveset.adapter.HostConnPool#reapConnections method at level 4.

   If necessary, review instructions for enabling trace in Tracing the Identity Manager Server.

2. Capture trace for a sufficiently long period of time (at least 30-60 minutes), while the adapter performs operations.

3. Review the trace output in the application server stdout or trace file and look for Info reaping connection entries.

   If this entry occurs more than once every 30 minutes, you have a good indication that connections are being timed out unnecessarily.

   To resolve this problem, increase the Idle Timeout resource attribute value to prevent connections from being reaped too frequently. The Idle Timeout attribute controls how long a connection remains idle before the connection is logged out. The default value is 90 seconds, which causes new logins to occur frequently.

   Ideally, specify a value that is greater than the average idle time for your deployment environment. For example, adjust the Idle Timeout attribute to 30 minutes (1800000 milliseconds) or more.

PeopleSoft Adapter Problems

This section describes methods for troubleshooting the following PeopleSoft adapter problems:

• Executing a “Test Connection” from the PeopleSoft resource adapter causes a failure with a generic exception.

  1. Open the Waveset.properties file and set exception.trace=true.

  2. Retry the test connection, and if you see the following results:

     FormState: expansion complete java.lang.NullPointerException: PSProperties not loaded from file WavesetException: WavesetException: ==> com.waveset.util.WavesetException: FormState: derivation Log in to the PeopleSoft web interface to verify that you are using the correct UID and password.

• The PeopleSoft application server logs are not showing login attempts.

  This problem generally occurs because you did not use the psjoa.jar file supplied with the PeopleSoft installation to which you are connecting. (See the Sun Identity Manager 8.1 Resources Reference for more information about the PeopleSoft resource adapter).

SAP Adapter Problems

If an error results when you try to test the connection from an SAP or SAP HR Active Sync adapter to the SAP system, open a command window and run this command from your installation directory WEB-INF/lib:

java -jar sapjco.jar

The sapjco.jar command shows which version of the SAP Java Connector (JCO) is installed and whether the adapter is installed correctly. The command also returns the Java^TM Native Interface (JNI^TM) platform-dependent and the RFC libraries that communicate with the SAP system.

If these platform-dependent libraries are not found, consult the SAP documentation to find out how to correctly install the SAP Java Connector.

UNIX Adapter Problems

This section contains information about debugging some common problems with UNIX adapters.

• If you see timeout errors when provisioning to a UNIX resource adapter, you can determine where the provisioning process is failing by tracing the com.waveset.adapter.ScriptedConnection method at Method/Class level 4, which gives you the maximum logging output.

• When becoming root to perform administrative commands, if the resource adapter executes a su root command instead of a su - root command, the environment does not inherit any of the custom environment variables defined for root; including any custom prompts (environment variable of PS1).

  When configuring a UNIX adapter, you can determine which prompt to enter into the Root Shell Prompt field as follows:

  1. Telnet or ssh to the system as the user you specified in the Login User field.

  2. After typing the password and logging in, type su root without a dash and press return.

  3. Type the root password.

  4. The next prompt displayed is the prompt you must enter into the Root Shell Prompt field.

Troubleshooting Auditor

You can trace the following methods to troubleshoot issues with Identity Auditor:

• com.sun.idm.auditor.policy to trace issues with Audit Scans.

• com.sun.idm.auditor.accessreview to trace issues with Access Reviews.

• com.sun.idm.auditor.report to trace issues with Audit Reports.

• com.sun.idm.auditor.view to trace issues with Auditor Views.

In addition, you can set the following hidden flags by modifying Forms or TaskDefinitions:

• Audit Policy Scan Task - maxThreads (int). Number of concurrent threads used. (Defaults to 5).

• Audit Policy Scan Task - userLock (int). Time (in milliseconds) to wait to acquire a lock on the User object. (Defaults to 5000)

• Audit Policy Scan Task - scanDelay (int). Time (in milliseconds) to delay between launching a new scan thread. (Defaults to 0, no delay)

• Audit Policy Scan Task - clearuserLocks (boolean). If true, the scanner attempts to clear the user lock before scanning.

In addition, the Show Timings page (/debug/callTimer.jsp) provides the following information:

• How long it takes to fetch the initial User view, with no resources, during the scan

• How long it takes to refresh the User view, including resources, during the scan

• How long it takes to evaluate the policy on the User view

• How long each user scan takes, including User view fetch, policy evaluation, and so forth

• How long it takes to fetch a list of users for access review

• How long it takes to evaluate the attestation rule in access review

Troubleshooting ClassNotFound Exceptions

In the event of a ClassNotFound exception error, verify that the missing class is included in the server's classpath. If the classpath is configured properly, try configuring your application server such that the application class loader loads before the parent class loader. Sometimes loading the application classpath before the server classpath can resolve this issue. Consult your application server documentation for instructions.

Troubleshooting Form Problems

This section describes some common form problems and how to fix these problems.

• You added a <script> tag to your user form, but see the following exception when trying to access the user form:

  java.lang.NoClassDefFoundError: org/mozilla/javascript/NativeScript

  You must put the WEB-INF/lib/javascript.jar file into the application server’s classpath. For example:

  • If you installed Tomcat as a service (Windows only), edit the following registry key:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tomcat\Parameters\JVM Option Number 0

    Append the path to your jar file to the end of the JVM Option Number 0 key string value, wherever it is located. For example:

    ;C:\tomcat\lib\javascript.jar

  • If you just started Tomcat from the command line, move the jar file into the tomcat/lib directory.

• You customized a Tabbed User Form, but when you try accessing the form through User Creation, you see the following exception:

  com.waveset.util.WavesetException: Maximum form stack depth exceeded

  You are limited to pushing 1000 hard-coded elements on the stack and you exceeded this limit.

  If you added a check to detect Field containers that include a FieldRef or Field that matches the name of the Field container, you might have inadvertently created a circular reference on the form. To fix this problem, change the FieldRef or Field name.

• You used a MultiSelect field in a form for LDAP users, assigned groups to a user in this field, edited the user again, and then see the same group on both sides of the MultiSelect. For example:

  • Left side value. cn=Group1,dc=test,dc=Com

  • Right side value. cn=Group1,dc=test,dc=com

    In this example, the LDAP baseContext resource field is set to dc=test,dc=com and the LDAP groups are listed as dc=test,dc=Com, which causes a problem because LDAP is not case sensitive, but the MultiSelect widget is case-sensitive.

    To alleviate this problem, change the baseContext on the LDAP resource to match the case in your LDAP resource dc=test,dc=Com or use the <upcase> XPRESS function to uppercase both the left and right side displays.

Troubleshooting the Gateway

When troubleshooting the Sun Identity Manager Gateway, it is often useful to run the Gateway from the command line. Using command line options allows you to input a wider range of start-up options, which includes starting the Gateway as a normal application instead of a service and running the Gateway on a different port.

You must kill the Identity Manager Gateway as a service before running it from the command line. For example, type

gateway.exe -k

The following table describes the Gateway command line arguments.

Usage: gateway -i n -r -s -k -t n -d -p n -f name -l n -m n -v.

• The -d and -s options are mutually exclusive.

• See From the Command Line for more information about the Gateway tracing levels.

You can also use the Identity Manager Gateway Debug page (debug/Gateway.jsp ) to troubleshoot the Gateway. See How to Configure Tracing from the Gateway Debug Page for more information.

Troubleshooting Java Code Problems

If you have the basic Java programming skills required to work with Identity Manager, you should be able to diagnose and resolve most Java code problems.

However, a fairly common problem occurs where someone opens a connection to the database but does not close the connection properly. If you do not close the connection properly, performance issues result.

Troubleshooting Low Memory Conditions

This section describes tools that you can use to investigate low memory conditions, including:

• From the Identity Manager Debug Pages

• From JConsole

From the Identity Manager Debug Pages

You must have the Debug capability to access and execute operations from the Identity Manager Debug pages. Administrators and the Configurator are assigned this capability by default.

If you do not have the Debug capability, an error message results.

You can open the following Identity Manager Debug pages from the Administrator interface to monitor how much memory is being used by your system:

• Host Connection Pool page (debug/Show_ConnectionPools.jsp). View a summary of connection pool statistics (if you are not using a data source), including the pool version, how many connections were created, how many are active, how many connections are in the pool, how many requests were serviced from the pool, and how many connections were destroyed.

  You can also use the Host Connection Pool page to view a summary of the connection pools used to manage connections to the Gateway. You can use this information to investigate low-memory conditions.

• List Cache Cleared (debug/Clear_XMLParser_Cache.jsp ). Clear the cache of recently used XML parsers.

• Private collection pool (debug/Show_JDBC.jsp). View a summary of the cache of Java DataBase Connectivity (JDBC^TM) connections being used by the repository and some resource adapters.

• System Memory Summary page (debug/Show_Memory.jsp). View the used and total memory in the system. You must click the Garbage Collect button to get the most current used memory value.

• System Memory Summary page (debug/Show_Memory2.jsp). View an updated Show_Memory.jsp page that allow you to clear all unused memory in the JVM so you can investigate heap usage.

• User Session Pool Cleared (Clear_User_Cache.jsp). Clear the cached sessions for recently logged in users.

• XML Resource Adapter Caches Flushed and Cleared (Clear_XMLResourceAdapter_Cache.jsp). Clear the cache of the test XML resource adapter.

From JConsole

Use the Java Monitoring and Management Console (JConsole) to detect low memory and deadlocks. JConsole is a Java Management Extension (JMX^TM) technology-compliant graphical management tool that is co-packaged with JDK 5 (and later).

JConsole accesses the memory system, memory pools, and MBeans garbage collector to provide information about memory use such as memory consumption, memory pools, and garbage collection statistics. In addition, You can use JConsole to monitor MBeans for information about current heap memory use and non-heap memory use.

For information about using JConsole to monitor applications that run on the Java platform, see Using JConsole to Monitor Applications. This document is available from the following URL:

http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

Troubleshooting PasswordSync Problems

When you are trying to troubleshoot problems with PasswordSync, review the following logs for information:

• PasswordSync Error Logs. PasswordSync writes all failures to the Windows Event Viewer. (For more information about Event Viewer, see Windows’ Help.) The source name for error log entries is PasswordSync.

• PasswordSync Trace Logs. PasswordSync writes all trace logs to the file location specified when you configured tracing. See Using the PasswordSync Configuration Tool.

Some common PasswordSync problems and solutions include

• PasswordSync is not propagating password changes from the Windows server to Identity Manager.

  PasswordSync relies on the registry settings when creating a connection from Active Directory to the Identity Manager Server. PasswordSync reads the registry and processes the settings, but PasswordSync does not perform any checks to see if it can create a connection.

  The following example shows a registry entry for a PasswordSync server. This example includes the default registry setting values, but does not show all of the settings used by PasswordSync.

  [HKEY_LOCAL_MACHINE\SOFTWARE\Waveset\Lighthouse\PasswordSync] "reinitIntervalMilli"=dword:0001d4c0 "securityIgnoreCertRevoke"=dword:00000000 "securityAllowInvalidCert"=dword:00000000 "directMode"=dword:00000001 "lhuser"="config" "lhcred"="rsVtQZpa5Ys=" "endpointURL"="http://10.10.10.10:8080/idm/servlet/PasswordSync" "installdir"="C:\\Program Files\\Sun Microsystems\\Sun Identity Manager PasswordSync" "tracelevel"=dword:00000000 "tracemaxKB"=dword:00002710 "tracefile"="C:\\Program Files\\Sun Microsystems\\Sun Identity Manager PasswordSync\\trace.log"

  If you have not enabled tracing at an appropriate level, PasswordSync does not log connection failures in much detail. To see more detailed trace information, edit the PasswordSync registry settings as described on Editing the Registry Keys. Specify tracelevel 4 to output the maximum trace information, and change the tracefile value to point to a writable file. For example:

  "tracelevel"=dword:00000004
  "tracefile"="C:\\Program Files\\Sun\\IdentityManager\\PasswordSync\\pwicsvc.log"

  The registry settings will be reread based on the <i>reinitIntervalMilli</i> setting in the registry. After rereading the registry settings, PasswordSync automatically starts or stops tracing, depending on the trace parameters set in the registry. For each intercepted password change, PasswordSync logs the actions taken to push the password to Identity Manager.

• If a connection fails during creation, you might encounter the following situations:

  ---

  Note –

  Each of these situations has its own error code and set of log entries. Identity Manager removes the date, time stamp, and process number from these entries to keep them short.

  ---

  • An incorrect or unreachable URL error that occurs when the server cannot be reached, is not running, or does not reply with a correct response.

    Check that PasswordSync can access the server and page.

    • Be sure the server is running and that you have configured your firewalls and routers correctly.

    • Check the application server to be sure it is running, and that PasswordSync can connect to the endpointURL without the application path. If PasswordSync can does not return a page or an error, the application server is not running.

    • Check the servlet response by opening the endpointURL in a standard browser. If you do not see an error that starts with: com.waveset.util.WavesetException see if the servlet is compiling and available..

    • JMS usage for PasswordSync relies on the jms.jar being available in the classpath. The following exception message displays if you access the endpointURL without the correct file in place:

      com.waveset.util.WavesetException: A JMS request arrived, but JMS PasswordSync is unavailable. Is JMS jar file available?

  • An incorrect user name error generally occurs when the userID stored in the lhuser entry is incorrect. Use the Configure.exe utility to replace the user or replace the lhuser registry key value with a valid userID.

  • An incorrect password error generally occurs when the password stored in the lhcred entry is not correct when used in combination with the userID stored in lhuser. Use the Configure.exe utility to replace the password, but do not manually edit the lhcred registry key.

  • A garbage in the password entry error generally occurs when the registry key is corrupted and or when the registry key is manually edited, which causes garbage in the password entry.

  • This situation causes the process to hang in RAEncryptor::Decrypt3DES and PasswordSync cannot decrypt the entry. Use the Configure.exe utility to replace the password.

Troubleshooting Reconciliation Problems

When you are trying to troubleshoot problems with a reconciliation task, review the Reconciliation Status Debug page ( debug/Show_Reconciler.jsp) to see what the resource threads are working on.

Some common reconciliation problems include

• Reconciliation does not start.

  1. Open the System Threads Debug page (debug/Show_Threads.jsp) to see if the Reconcilerserver_name task is running.

  2. If the task is not running, open the System Settings page and click Trace Scheduler.

     Running the Scheduler restarts the Reconciler server_name task.

  3. Check the System Threads page again, and if the Reconciler server_name task has not restarted, the Scheduler might be hung.

  4. Restart your application server.

• Reconciliation fails for a certain user object.

• If the object exists in Identity Manager, try editing the object directly. If the Identity Manager account does not exist, load the one account from the resource.

• Verify the reconciliation user has the proper access rights.

• Try extracting the object to a file using the same code path as reconciliation.

• Open the System Memory Summary Debug page (debug/Show_Memory.jsp or debug/Show_Memory2.jsp) and verify you have enough free memory.

Troubleshooting Repository Connection Problems

Identity Manager’s lh commands are very useful when you are troubleshooting connection problems. These commands use Identity Manager’s web application installation, but remove the application server from the equation.

This section describes the following

• Using lh Commands to Debug Problems

• Testing DataSource Connections

Using lh Commands to Debug Problems

This section describes how to use the lh commands; starting with using the more basic commands and progressing to using commands that exercise most of Identity Manager.

• Using lh setRepo -c -n

• Using lh setRepo -c -v

• Using setRepo

• Using lh console

After becoming familiar with these debugging tools, you can develop your own variations for using these lh commands.

Using lh setRepo -c -n

Use the lh setRepo -c -n command to perform the most basic connection test, which allows you to examine the current repository location without connecting. You can use this command to verify that parameters, such as URL and JDBC driver, are correct.

• If the connection is successful, you can read the ServerRepository.xml bootstrap file.

• If the connection fails, try to solve this failure first. A decryption error is the most common cause of this failure. For example, you might have a J2EE mismatch or classpath conflict.

Using lh setRepo -c -v

Use the lh setRepo -c -v command to connect to and examine the current repository location. (The -v provides verbose output.) You can use this command to exercise almost all of the Repository code without requiring the Identity Manager server.

• If the connection is successful, then you are successfully connected to the current repository location.

• If the connection fails, try to solve this problem first. Solving your connection problem can be very helpful in resolving DNS, firewall, or remote connection privilege problems.

For more information, see Testing DataSource Connections.

Using setRepo

Use the setRepo command throughout the debugging process, to specify a new repository location or to set the repository to the same location.

You can use this command to confirm that all of the necessary components, such as the JAR files, are in place. The setRepo command also lets you vary connection information, such as userid and password, to debug table ownership or privilege problems.

Using lh console

Use this command to actually start an Identity Manager Server using the JAR files in the WEB-INF/lib and the classes in WEB-INF/classes under WSHOME. The lh console command uses your Identity Manager installation environment and actually starts the Identity Manager application, but removes the application server from the equation.

• If the connection is successful, the problem is specific to the application server environment or configuration.

• If the connection fails, review the failure messages.

• If the connection failure is the same as the application server failure, you have reproduced this failure with significantly fewer variables.

• If the failure appears to be different from the application server failure, try fixing the Identity Manager connection problem first because there are fewer variables and more of these variables are under Identity Manager control.

Testing DataSource Connections

If you are testing a DataSource connection, the lh setRepo -c command might fail.

This failure is especially likely if you configured Identity Manager to use the application server’s database connectivity service or the application server’s directory service. These services often work only in the environment that a running application server provides to a web application.

Initially, approach the DataSource configuration you want in a step-by-step manner. Once you are comfortable with these steps, you can adapt your approach to suit your needs.

1. Try using a direct JDBC DriverManager connection, such as a non-DataSource connection, that bypasses the application server’s database connectivity service.

2. Use a DataSource, but store the DataSource object in a directory service other than application server’s directory service.

   ---

   Note –

   If you have no other directory service available, you can download a free directory service, including the reference implementation of JNDI that uses only the local file system.

   ---

   If these steps work, you have localized the problem to the application server.

   Then, if useful, you can add the application server’s database connectivity service or the application server’s directory service, whichever service works outside of the environment that the application server provides to web applications.

Troubleshooting Server-Related Problems

You can analyze your application server logs for fatal errors and other server-related problems.

To troubleshoot server problems, use the application server’s Administrative Console to increase the logging level for each module. For more information, see the product documentation supplied with your server.

Most application servers have a standard location for standard out files (stdout) and standard error files (stderr) for the JVM running the application server. To analyze your application server logs, locate the logs directory or the log files specified for your Identity Manager application server.

• Open the stdoutfile to view minor messages and other tracings.

• Open the stderr file to view fatal and critical exceptions.

You will see Identity Manager start up and shut down the messages in this trace output.

Beginning with Identity Manager Version 7.1, a sealing violation exception occurs in the application server log when you use Identity Manager with Oracle 10g on Sun Application Server Enterprise Edition 8.2.

This exception generally occurs if you are using more than one Java Archive file (JAR file) containing Oracle JDBC drivers.

To prevent this problem, be sure the CLASSPATH contains only one JDBC driver JAR file, and that you use the ojdbc14_g.jar provided with Oracle 10g. In addition, you must use the ojdbc14_g.jar provided by the Oracle 10g installation to ensure correct operation.

Troubleshooting Service Provider Problems

If you are using the Sun Identity Manager Service Provider End User Login page in WebSphere, and a javax.servlet.UnavailableException occurs with a 404 error displayed in your browser, you must reset some properties in the IBM 1.5 JDK.

Use the following steps:

1. In the was-install/java/jre/lib directory, rename the jaxb.properties.sample to jax.properties and uncomment these two lines:

   javax.xml.parsers.SAXParserFactory= org.apache.xerces.jaxp.SAXParserFactoryImpl javax.xml.parsers.DocumentBuilderFactory= org.apache.xerces.jaxp.DocumentBuilderFactoryImpl

2. Save the file and restart the application server.

Troubleshooting an SPML Configuration

To test an SPML configuration:

1. Open the Connect page and click Test.

   A dialog indicating that the connection was successful pops up.

2. Open the Schema page and click Submit.

   The system displays a hierarchical view of the schemas supported by the Identity Manager server.

   If you cannot establish a successful connection

   • Double-check the URL you entered.

   • If the error you receive contains phrases such as no response or connection refused, then the problem is most likely the host or port used in the connection URL.

   • If the error suggests that a connection was made, but the web application or servlet could not be located, the problem is most likely in the WEB-INF/web.xml file.

Troubleshooting Upgrades

If you encounter problems during the upgrade, check the upgrade log files located in the $WSHOME/patches/logs directory. The file names for the logs are based on a timestamp and the stage of the upgrade.

If, following an upgrade, Identity Manager fails to start with the following exception, your JDK/JRE may be the problem:

java.lang.IllegalStateException: Error attempting to decrypt: 
Given final block not properly padded

Verify that you are using a JDK/JRE supplied by the same vendor that you were using previously. For example, you cannot upgrade to a Sun JDK if previously you were using a JDK from IBM. To fix this problem, uninstall the JDK/JRE and install the JDK or JRE from your previous vendor.