Chapter 17 Troubleshooting and Monitoring Instant Messaging

This chapter lists the common problems that might occur during installation and deployment of Instant Messaging and provides an overview of the watchdog. The log information generated by the various system components on their operation can be extremely useful when trying to isolate or troubleshoot a problem. In addition, you can use the monitoring framework agent to monitor the general health of Instant Messaging processes to help prevent problems before they occur, assess usage levels to help you scale your deployment, and to prevent downtime. This chapter contains information in the following sections:

• Obtaining Instant Messenger Runtime Information

• Obtaining Instant Messenger Logs

• Problems and Solutions

• Troubleshooting Instant Messaging and LDAP

• Monitoring Instant Messaging

• Managing the Watchdog Process

For details and more information on managing server, multiplexor, watchdog, Calendar agent, and client logging, and for default log file locations, see Chapter 11, Managing Logging for Instant Messaging.

Obtaining Instant Messenger Runtime Information

You can obtain information about a client system from the Instant Messenger client.

To Obtain Instant Messenger Runtime Information from the About Dialog

Steps

1. In Instant Messenger, select Help->About.

   The About dialog box appears.

2. Select the Details tab.

   The Details tab contains information about the client system that you can use when troubleshooting problems.

Obtaining Instant Messenger Logs

In order to obtain Instant Messenger logs, you first need to enable logging on the client host. See Administering Instant Messaging Client Logging for instructions.

Problems and Solutions

Listed below are some problems and their possible causes, and clues to help troubleshoot these problems:

• Cannot Forward Mail to Offline Users

• Calendar Pop-up Reminders Do Not Work

• Single Sign-on Does Not Work

• Instant Messenger Does Not Load or Start

• Connection Refused or Timed Out

• Authentication Errors

• Instant Messenger Channel Display Error

• Instant Messaging Content is not Archived

• Server-to-Server Communication Fails to Start

• Catastrophic Installation Failure Leaves Server in an Inconsistent State

• Instant Messaging Services Do Not Appear in the Access Manager Console (amconsole)

Instant Messenger Resource Customizations Lost After patchrm and patchadd

(Issue Number: 6361796) The patchrm and patchadd processes redeploy the client resources. When this occurs, all customized files are overwritten. You need to back up any customized files you want to save before performing these actions.

Cannot Forward Mail to Offline Users

By default, Instant Messaging uses the mail attribute to determine the email address to which it forwards instant messages when a recipient is offline. If your directory does not use the mail attribute for email addresses, you need to configure Instant Messaging to use the same attribute as your directory.

To Configure the Attribute Used for User Email Addresses

Steps

1. Open iim.conf.

   See iim.conf File Syntax for instructions on locating and modifying iim.conf.

2. Change the value of the iim_ldap.user.mailattr parameter to the attribute your directory uses to contain email addresses in user entries.

Calendar Pop-up Reminders Do Not Work

If Calendar pop-ups are not being delivered as expected, you can troubleshoot the configuration as described in this section. For instructions on setting up Calendar pop-ups, see Chapter 14, Using Calendar Pop-up Reminders.

The most common error in Calendar pop-up configuration is incorrectly entered parameter names in the configuration files. This includes typos and misspelled parameter names. Ensure that you have correctly entered all of the configuration parameters and values in iim.conf and ics.conf. If you have already configured pop-ups, use Table A–10 to compare your entries with the required parameters.

If your Instant Messaging and Calendar Server configuration files are correct, but pop-ups are still not arriving as expected, ensure the Calendar client and Instant Messenger are configured correctly.

To Troubleshoot Calendar Client and Instant Messenger Configuration for Pop-Ups

Steps

1. Log into the Calendar client.

2. Ensure that the time zone settings are correct.

   If you are using Calendar Express, select Tools->Options->Settings from the menu.

3. Schedule an email reminder.

   If you are using Calendar Express, select Tools->Options->Settings from the menu.

4. Save your settings.

5. Log into Instant Messenger with the same user.

6. Select Tools->Settings.

   The Settings dialog box appears.

7. Select the Alerts tab.

8. Check the Show Calendar Reminders checkbox and click OK.

9. Leave the Instant Messenger user logged in.

10. Check to see whether or not the user received the email alert and pop-up at the time configured in the Calendar client.

    If you did not receive the email alert, the Calendar client is incorrectly configured. Refer to the Calendar client documentation for further troubleshooting information.

    If you received the email alert, but not the Calendar pop-up, and you are sure that you have configured both servers and clients correctly, check the xmppd.log for further information. You may need to set this log to a more verbose setting, for example DEBUG. For instructions on changing the log level, see To Set Log Levels for Instant Messaging Components Using iim.conf Parameters.

Single Sign-on Does Not Work

If you are using SSO with Sun Java^TM System Access Manager, the Access Manager server and Instant Messaging server must be configured to use the same web container.

Instant Messenger Does Not Load or Start

The following are the possible causes for this problem:

• Wrong codebase in the applet page.

• Application/x-java-jnlp-file MIME type not defined in the web container configuration.

• Plug-in of Java Web Start not installed or functional.

• No compatible Java version available.

• Security exception, cannot verify signature of .jar files.

Where to get the necessary information:

• In the Java Web Start or plug-in errors (exception stack trace, launch page.)

• In the applet page source on the browser.

Connection Refused or Timed Out

The following are the possible causes for this problem:

• Either the Instant Messaging server or the multiplexor is not running.

• Incorrect multiplexor host or port names used in the Applet descriptor file (.jnlp or .html.)

• Different SSL settings used between Instant Messenger and the multiplexor.

• Client and server version mismatch.

Where to get diagnostic information:

• Instant Messaging server and multiplexor log files.

• Instant Messenger logs.

• Instant Messenger About dialog box, Details tab.

Authentication Errors

The following are the possible causes for this problem:

• Problems while accessing the LDAP server, such as the LDAP server is not running, or a provisioning error, such as a schema violation, has occurred.

• End user not found.

• Invalid credentials.

• Invalid Instant Messenger session.

Where to get diagnostic information:

• Instant Messaging server, Identity authentication, and LDAP log files.

• In deployments using Sun Java^TM System Access Manager, ensure that the user entries in your Directory contain the iplanet-am-managed-person objectclass. The Instant Messaging server uses this object class when it searches for valid users in an Access Manager deployment. For more information about this object class and how Access Manager uses it, refer to the Sun Java^TM System Access Manager documentation.

Instant Messenger Channel Display Error

The following are the possible causes for this problem:

• The server cannot validate the session token.

• Instant Messaging channel is not configured properly. For example, incorrect Instant Messaging server host, port, or both.

• Plug-in or Java Web Start is not installed or is not functional.

• End user not found and the Instant Messaging server cannot find the end user when performing an LDAP lookup.

Where to get diagnostic information:

Instant Messaging server and Instant Messaging channel logs.

Instant Messaging Content is not Archived

The following are the possible causes for this problem:

• Content is actually archived but the end user has insufficient rights to access it.

• The content has not yet been committed to the database.

• The archive provider has been disabled in the Instant Messaging server.

Where to get diagnostic information:

Instant Messaging server and the archive log files.

Server-to-Server Communication Fails to Start

The following are the possible causes for this problem:

• Incorrect server identification.

• Mismatch in the SSL settings.

Where get diagnostic information:

The Instant Messaging server log file for both servers.

Catastrophic Installation Failure Leaves Server in an Inconsistent State

If a catastrophic error occurs while installing or uninstalling Instant Messaging, the system might be left in an inconsistent state. This results in both install and uninstall being unable to complete. In this circumstance, you must manually remove all the Instant Messaging components so that a fresh install can be attempted. The clean up procedure consists of removing packages and registry information.

To Manually Remove All Instant Messaging Components

Steps

1. Back up any information you might need in a future installation.

   See Backing Up Instant Messaging Data for instructions.

2. Manually edit the product registry information.

   For Solaris 9, issue the following command:

   prodreg(1)

   For all other operating systems:

   1. Edit productregistry.xml and remove all Instant Messaging XML elements from the file.

      By default, the productregistry XML file is stored in the following locations:

      • Solaris: /var/sadm/install/productregistry

      • Linux: /var/tmp/productregistry

   2. Remove the following packages or RPMs if they are still present:

      • SUNWiim

      • SUNWiimc

      • SUNWiimd

      • SUNWiimid

      • SUNWiimin

      • SUNWiimjd

      • SUNWiimm

Instant Messaging Services Do Not Appear in the Access Manager Console (amconsole)

If Instant Messaging uses Access Manager policies in a Sun Java^TM System Application Server deployment, you need to restart the Application Server when you finish configuring Instant Messaging. If you do not restart the Application Server, Instant Messaging services will not appear in the Access Manager console (amconsole).

Troubleshooting Instant Messaging and LDAP

The following LDAP issues might arise in a given deployment. Change the LDAP parameters in iim.conf accordingly.

Using a Directory That Does not Permit Anonymous Bind

By default, Instant Messaging server performs an anonymous search of the LDAP directory. However, it is common for sites to prevent anonymous searches in their directory so that any random person cannot do a search and retrieve all the information. If your site’s directory is configured to prevent such anonymous searches, and you didn't provide bind credentials during post-installation configuration, you need to configure the Instant Messaging server needs with a user ID and password it can use to bind and perform searches.

Use the iim_ldap.usergroupbinddn and iim_ldap.usergroupbindcred parameters to configure the necessary credentials.

To Configure Bind Credentials for the Instant Messaging Server

Steps

1. Open iim.conf.

   See iim.conf File Syntax for instructions on locating and modifying iim.conf.

2. Specify the DN you want the server to use to bind to the directory as the value for iim_ldap.usergroupbinddn.

   iim_ldap.usergroupbinddn=bind_DN

3. Specify the password that corresponds to the bind DN as the value for iim_ldap.usergroupbindcred

   iim_ldap.usergroupbindcred=password

4. Save and close the file.

Displaying Contact Names Using an Attribute Other than cn

You can customize how Instant Messenger displays contact names. The default attribute used byInstant Messenger to display contact names is cn. Contact names appear as First Name, Last Name. For example, Frank Smith, Mary Jones, and so on.

Use the iim_ldap.userdisplay and iim_ldap.groupdisplay parameters to specify which attribute to use to display contact names.

To Change the Attribute Used to Display Contact Names

Steps

1. Open iim.conf.

   See iim.conf File Syntax for instructions on locating and modifying iim.conf.

2. Specify the attribute you want to use to display user names as the value for iim_ldap.userdisplay.

   iim_ldap.userdisplay=user_name_attribute

3. Specify the attribute you want to use to display group names as the value for iim_ldap.groupdisplay

   iim_ldap.groupdisplay=group_name_attribute

4. Save and close the file.

Searching the Directory Using Wildcards

If your directory is indexed to allow the use of wildcards, and you want to be able to use wildcards while searching for contact names, you need to configure the Instant Messaging server to allow wildcard searches. However, allowing wildcard searches can impact performance unless User IDs are indexed for substring search. See Modifying How Client Users Search for Contacts for instructions on allowing wildcard searches in Instant Messaging.

Using Nonstandard Objectclasses for Users and Groups

If your directory uses nonstandard objectclasses to define users and groups you need to change the appropriate iim_ldap.* parameters, replacing inetorgperson and groupofuniquenames with your values.

See LDAP and User Registration Configuration Parameters for a list of LDAP parameters.

To Change the Objectclasses Used to Specify Users and Groups

Steps

1. Open iim.conf.

   See iim.conf File Syntax for instructions on locating and modifying iim.conf.

2. Search for and replace inetorgperson with the object class used to define users in your directory.

3. Search for and replace groupofuniquenames with the object class used to define groups in your directory.

4. Save and close the file.

Using an Attribute Other than uid for User Authentication

If your directory does not use the uid attribute for user authentication, you need to configure the Instant Messaging server with the attribute used by your directory. By default, Instant Messaging uses uid. You also need to change each filter parameter that contains uid in its value.

Use the iim_ldap.loginfilter parameter to specify which attribute to use for user authentication.

To Change the Attribute Used for User Authentication

Steps

1. Open iim.conf.

   See iim.conf File Syntax for instructions on locating and modifying iim.conf.

2. Search for and replace uid with the attribute you want to use for user authentication in the following parameters:

   • iim_ldap.loginfilter

   • iim_ldap.usergroupbyidsearchfilter

3. Save and close the file.

Using an Attribute Other than uid for User IDs

If your directory does not use the uid attribute for user IDs, you need to configure the Instant Messaging server with the attribute used by your directory. By default, Instant Messaging uses uid. In addition, you should index the attribute in the directory to help offset any performance degradation caused by searching on unindexed attributes.

Use the iim_ldap.user.uidattr parameter to specify which attribute to use for user IDs.

To Change the Attribute Used for User IDs

Steps

1. Open iim.conf.

   See iim.conf File Syntax for instructions on locating and modifying iim.conf.

2. Specify the attribute you want to use for user IDs as the value for iim_ldap.user.uidattr.

   The default value is uid.

   For example, to use the loginname attribute, set the iim_ldap.user.uidattr attribute as follows:

   iim_ldap.user.uidattr=loginname

3. Save and close the file.

4. Add the index directive to the indexing rules in LDAP:

   index loginname eq

Troubleshooting Connectivity Issues in a Multi-Node Deployment (Server Pool)

If you are receiving errors where presence status is not being shared between servers in a server pool:

• Ensure that the nodes are configured correctly to enable server-to-server communication. See Configuring Server-to-Server Communication Between Instant Messaging Servers in a Server Pool for a list of configuration parameters and appropriate values.

• Check for server-to-server session establishment errors in the log file.

Monitoring Instant Messaging

Instant Messaging provides an agent to help you monitor activity. This agent is called the monitoring framework management agent, or mfwk agent. The mfwk agent is contained within the Common Agent Container (CAC). The CAC and the mfwk agent are installed when you installed Instant Messaging.

The mfwk agent makes XMPP module statistics available through the Java Monitoring and Management Console (JConsole). Table 17–1 describes the Instant Messaging services for which the agent exposes state and performance metrics.

Table 17–1 Instant Messaging Services Monitored by the mfwk Agent

Category	Services	Description
Authentication	auth	Authentications.
Discovery	disco	Discovery requests.
Messages	message	Information about alerts and one-on-one chat sessions between two clients such as the speed at which messages are sent.
Conference	muc-presence, muc-admin, and muc-message	Conference statistics such as leaving or joining a conference, conference administrative requests, and conference (group chat) messages relayed.
Presence	presence-subscribe, presence-unsubscribe, presence-probe, and presence-authorize	Presence informations such as updates and subscriptions.
Privacy	private-get, private-set, privacy-get, and privacy-set	Privacy details.
Roster	roster-get and roster-set	Roster information.
Search	search	Search statistics.

This section provides information about administering and troubleshooting the mfwk agent and JConsole, and how you can use the agent and JConsole to monitor Instant Messaging in the following sections:

• Enabling and Disabling Monitoring for Instant Messaging

• Administering the mfwk Agent

• Viewing Monitoring Data

• Troubleshooting the mfwk Agent

• Troubleshooting JConsole

Enabling and Disabling Monitoring for Instant Messaging

Before you can use the mfwk agent, you need to configure the Instant Messaging server to make its activities available (turn on instrumentation) for use by mfwk as described in this section. Monitoring is disabled by default when you install Instant Messaging.

To Enable or Disable Monitoring for Instant Messaging

Steps

1. Open iim.conf.

   See iim.conf File Syntax for more information.

2. Change the value of the iim_server.monitor.enable parameter.

   • To enable monitoring:

     iim_server.monitor.enable=true

   • To disable monitoring:

     iim_server.monitor.enable=false

3. Save and close the file.

4. Refresh the server.

   imadmin refresh server

   ---

   Caution –

   Do not use the imadmin start, imadmin stop, or imadmin refresh commands in an HA environment with Sun Cluster. Instead, use the Sun Cluster administrative utilities.

   ---

Administering the mfwk Agent

You use the mfwkadm command-line utility to administer the mfwk agent. For example, you can start, stop, and restart the agent, and set up new and view current performance monitoring jobs performed by the agent. Detailed instructions on using this utility are available in the mfwkadm man page. In addition, the agent runs inside the CAC. For information on the CAC, refer to the cacaoadm and cacao man pages. This section provides instructions for locating these man pages.

Table 17–2 lists the paths to the man pages.

Table 17–2 mfwkadm and CAC man page paths

Component	Solaris Path	Linux Path
mfwkadm	/opt/SUNWmfwk	/opt/sun/mfwk
CAC	/opt/SUNWcacao/man	/opt/sun/man

To Access the mfwkadm and CAC man Pages

Steps

1. On the command line, check your MANPATH environment variable to see if the correct paths are already there.

2. If the correct path is not there, append the location of the mfwkadm utility and CAC man pages to your MANPATH environment variable.

   For example, on Solaris:

   MANPATH=/usr/dt/man:/usr/man:/opt/SUNWmfwk:/opt/SUNWcacao/man

   On Linux, update /etc/man.config with the path to the man pages.

3. Verify that the man pages are accessible.

   For example:

   man mfwkadm

Viewing Monitoring Data

Use JConsole to view the information exposed by the mfwk agent. JConsole is a graphical console tool that enables you to monitor and manage Java applications and virtual machines in your network. Using JConsole, you can browse the server JVM and also observe the Instant Messaging services described in Table 17–1.

For more information about using JConsole, see the JConsole man page and the “Using JConsole” chapter of Monitoring and Management for the Java Platform.

To View Instant Messaging Monitoring Information Using JConsole

Steps

1. Log in as root.

2. Set the CLASSPATH to include the location of the CAC, JConsole, and the JMX jar file.

   ---

   Note –

   The line should be entered as a single line without line breaks.

   ---

   On Solaris:

   /opt/SUNWcacao/lib/cacao_cacao.jar:/opt/SUNWjdmk/5.1/lib /jmxremote_optional.jar:/usr/jdk/entsys-j2se/lib/jconsole.jar

   On Linux:

   /opt/sun/cacao/share/lib/cacao_cacao.jar:/opt/sun/jdmk/5.1/lib /jmxremote_optional.jar:/usr/jdk/entsys-j2se/lib/jconsole.jar

3. Run JConsole.

   ---

   Note –

   On both platforms, enter the command as a single line with a space between JConsole and “service.

   ---

   On Solaris:

   /usr/jdk/entsys-j2se/bin/java sun.tools.jconsole.JConsole "service:jmx:cacao-jmxmp://localhost;wellknown=true;username=root"

   On Linux:

   /usr/jdk/entsys-j2se/bin/java sun.tools.jconsole.JConsole "service:jmx:cacao-jmxmp://localhost;wellknown=true;username=root"

4. On the MBeans tab, expand the XMPP tree.

   The service attributes and their values are listed within the tree. See Table 17–1 for a complete list of Instant Messaging services visible through JConsole.

Troubleshooting the mfwk Agent

If you are experiencing trouble using the mfwk agent to monitor Instant Messaging, ensure the following:

• The dependencies are installed. In particular, check to make sure the JDMK, CAC, and the mfwk agent are installed. To check, use the following commands:

  Solaris:

  • pkginfo SUNWmfwk-agent (For the mfwk agent.)

  • pkginfo SUNWcacao (For the CAC.)

  • For JDMK, if the CAC is running, then the JDMK is installed. You can also check that the jar files are installed under /opt/SUNWjdmk/version/lib and /opt/SUNWjdmk/version/bin, where version is the version number of the JDMK, for example 5.1.

  Linux:

  • rpm -qi sun-mfwk-agent-1.0 (For the mfwk agent.)

  • rpm -qi sun-cacao-1.0 (For the CAC.)

  • For JDMK, if the CAC is running, then the JDMK is installed. You can also check that the jar files are installed under /opt/sun/jdmk/version/lib and /opt/sun/jdmk/version/bin, where version is the version number of the JDMK, for example 5.1. In addition, you can use the rpm command.

    rpm -qi sun-jdmk-runtime-5.1

• The CAC is running. To get status on the CAC, use the following commands:

  Solaris: /opt/SUNWcacao/bin/cacaoadm status

  Linux: /opt/sun/cacao/bin/cacaoadm status

  If the CAC is not running, start it as follows:

  Solaris: /opt/SUNWcacao/bin/cacaoadm start

  Linux: /opt/sun/cacao/bin/cacaoadm start

• The XMPP module is loaded within the CAC and running:

  /opt/SUNWcacao/bin/cacaoadm status com.sun.im.service.xmpp

• The mfwk agent is loaded within the CAC:

  Solaris: /opt/SUNWcacao/bin/cacaoadm list-modules

  Linux: /opt/sun/cacao/bin/cacaoadm list-modules

  The module name for the mfwk agent is com.sun.mfwk.mfwk_module.

• The mfwk agent is running. To get status on the mfwk agent, use the following commands:

  Solaris: /opt/SUNWcacao/bin/cacaoadm status com.sun.mfwk.mfwk_module

  Linux: /opt/sun/cacao/bin/cacaoadm status com.sun.mfwk.mfwk_module

  If the mfwk agent is not running, start it as follows:

  Solaris: /opt/SUNWmfwk/bin/mfwkadm start

  Linux: /opt/sun/mfwk/bin/mfwkadm start

Troubleshooting JConsole

If you cannot bring up JConsole, ensure the following:

• The path to JConsole is correctly entered in your CLASSPATH.

• You logged in as root before attempting to run JConsole.

Managing the Watchdog Process

The watchdog process monitors the server and multiplexor components and attempts to restart a component if it determines that the component is not running.

For the server, the watchdog determines whether the server is running by periodically attempting to make a connection, either directly to the server or through the multiplexor, based on the current configuration of the server. The watchdog tries to poll the server’s operational status and if it cannot determine the status, it then tries to make a connection to the server. If both operations fail, the watchdog stops and then restarts the server.

Before you use the watchdog, verify that it is enabled and running using the imadmin status command. By default, the watchdog is enabled and running when you install Instant Messaging.

More information about the imadmin utility is available in Appendix C, Instant Messaging imadmin Tool Reference.

Determining the Status of the Watchdog

You use the imadmin command-line utility to check the status of the watchdog.

To Determine the Status of the Watchdog

Steps

1. Change to the directory that contains the imadmin command-line utility.

   cd im_svr_base/sbin

2. Run imadmin status:

   ./imadmin status watchdog

   The imadmin utility returns the current status of the watchdog.

Enabling and Disabling the Watchdog

By default, the watchdog is enabled when you install Instant Messaging. You can disable or enable the watchdog by setting a configuration parameter in iim.conf.

To Enable or Disable the Watchdog

Steps

1. Open iim.conf.

   See iim.conf File Syntax for instructions on locating and modifying iim.conf.

2. Enable or disable the watchdog by setting the iim_wd.enable parameter as follows:

   To enable the watchdog: iim_wd.enable=true

   To disable the watchdog: iim_wd.enable=false

3. Save and close the iim.conf file.

4. Refresh the Instant Messaging server configuration:

   cd im_svr_base/sbin

   ./imadmin refresh

Managing Logging for the Watchdog

You manage logging for the watchdog the same way you manage logging for the server, multiplexor, and the Calendar agent. The watchdog log file is saved as im_db_base/log/iim_wd.log.

For more information on setting logging levels for all Instant Messaging components including the watchdog, see Chapter 11, Managing Logging for Instant Messaging.