Configuring and Managing WebLogic Store-and-Forward

     Previous  Next    Open TOC in new window  Open Index in new window  View as PDF - New Window  Get Adobe Reader - New Window
Content starts here

Troubleshooting WebLogic SAF

This release of WebLogic Server includes the WebLogic Diagnostic Service, which is a monitoring and diagnostic service that runs within the WebLogic Server process and participates in the standard server life cycle. This service enables you to create, collect, analyze, archive, and access diagnostic data generated by a running server and the applications deployed within its containers. This data provides insight into the runtime performance of servers and applications and enables you to isolate and diagnose faults when they occur. WebLogic SAF takes advantage of this service to provide enhanced runtime statistics, notifications sent to imported SAF queues and topics, message life cycle logging JMS messages, and debugging to help you keep your WebLogic domain running smoothly.

For more information on monitoring SAF agent statistics, see Monitoring SAF Agents.

The following sections explain how to troubleshoot the WebLogic SAF service.


Debugging WebLogic SAF

Once you have narrowed the problem down to a specific application, you can activate WebLogic Server's debugging features to track down the specific problem within the application.

Enabling Debugging

You can enable debugging by setting the appropriate ServerDebug configuration attribute to true. Optionally, you can also set the server StdoutSeverity to Debug.

You can modify the configuration attribute in any of the following ways.

Enable Debugging Using the Command Line

Set the appropriate properties on the command line. For example,


This method is static and can only be used at server startup.

Enable Debugging Using the WebLogic Server Administration Console

Use the WebLogic Server Administration Console to set the debugging values:

  1. If you have not already done so, in the Change Center of the Administration Console, click Lock & Edit (see Use the Change Center).
  2. In the left pane of the console, expand Environment and select Servers.
  3. On the Summary of Servers page, click the server on which you want to enable or disable debugging to open the settings page for that server.
  4. Click Debug.
  5. Expand default.
  6. Select the check box for the debug scope or attributes you want to modify. For example, select the DebugMessaging check box for SAF debugging.
  7. Select Enable to enable (or Disable to disable) the debug scopes or attributes you have checked.
  8. To activate these changes, in the Change Center of the Administration Console, click Activate Changes.
    Not all changes take effect immediately—some require a restart (see Use the Change Center).
  9. This method is dynamic and can be used to enable debugging while the server is running.

Enable Debugging Using the WebLogic Scripting Tool

Use the WebLogic Scripting Tool (WLST) to set the debugging values. For example, the following command runs a program for setting debugging values called

   java weblogic.WLST

The main scope, weblogic, does not appear in the graphic; saf is a sub-scope within weblogic. Note that the fully-qualified DebugScope for DebugSAFSendingAgent is messaging.saf.admin.

The program contains the following code:

connect(user, password, url)

Note that you can also use WLST from Java. The following example shows a Java file used to set debugging values:

import weblogic.jndi.Environment;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;

public class test {
public static void main(String args[]) {
try {
WLSTInterpreter interpreter = null;
String user="user1";
String pass="pw12ab";
String url ="t3://localhost:7001";
Environment env = new Environment();
Context ctx = env.getInitialContext();

interpreter = new WLSTInterpreter();

} catch (Exception e) {
System.out.println("Exception "+e);

Using the WLST is a dynamic method and can be used to enable debugging while the server is running.

Changes to the config.xml File

Changes in debugging characteristics, through console, or WLST, or command line are persisted in the config.xml file.

This sample config.xml fragment shows a transaction debug scope (set of debug attributes) and a single SAF attribute.

Listing 5-1 Example Debugging Stanza for SAF

SAF Debugging Scopes

It is possible to see the tree view of the DebugScope definitions using java weblogic.diagnostics.debug.DebugScopeViewer.

You can enable the following registered debugging scopes can be enabled for the WebLogic SAF.

Request Dyeing

Another option for debugging is to trace the flow of an individual (typically "dyed") application request through the SAF service. For more information, see Configuring the Dye Vector via the DyeInjection Monitor in Configuring and Using the WebLogic Diagnostic Framework.


SAF Message Life Cycle Logging for JMS Messages

The message life cycle is an external view of the events that a JMS message traverses through once it has been stored by a SAF agent and eventually forwarded to a remote JMS destination. Message life cycle logging provides an administrator with easy access to information about the existence and status of stored and forwarded JMS messages from a SAF agent's viewpoint. In particular, each message log contains information about basic life cycle events such as message storing, forwarding, and expiration. For more information on overall WebLogic logging services, see " Understanding WebLogic Logging Services" in Configuring Log Files and Filtering Log Messages.

SAF message logging is enabled by default when you create a SAF agent, however, you must specifically enable it on the imported SAF destinations in JMS modules targeted to the SAF agent. Logging can occur on a continuous basis and over a long period of time. It can be also be used in real-time mode while the SAF agent is deployed, or in an off-line fashion when the SAF agent is down. For information about configuring SAF message logging, see:

Events in the SAF Message Life Cycle

When SAF message life cycle logging is enabled for an imported SAF queue or topic, a record is added to the SAF agent's message log file each time a message meets the conditions that correspond to a basic message life cycle event. The life cycle events that trigger a SAF message log entry are as follows:

Message Log Location

The message log is stored under your domain directory, as follows:


where USER_DOMAIN is the root directory of your domain, typically c:\bea\user_projects\domains\USER_DOMAIN, which is parallel to the directory in which WebLogic Server program files are stored, typically c:\bea\weblogic910.

Enabling SAF Message Logging

You can enable or disable SAF message logging for a SAF queue and SAF topic using the WebLogic Server Administration Console. For more information see the following sources.

WebLogic Java Management Extensions (JMX) enable you to access the SAFAgentRuntimeMBean and SAFAgentRuntimeMBean MBeans to manage SAF message logs. For more information see " Overview of WebLogic Server Subsystem MBeans" in Programming WebLogic Management Services with JMX

When you enable SAF message logging, you can specify whether the log entry will include all the message header fields or a subset of them; all system-defined message properties or a subset of them; all user-defined properties or a subset of them. You may also choose to include or to exclude the body of the message. For more information about message headers and properties see " MessageProducer and MessageConsumer" in Programming WebLogic JMS.

SAF Message Log Content

Each record added to the log includes basic information such as the message ID and correlation ID for the subject message. You can also configure the SAF agent to include additional information such as the message type and user properties.

SAF Message Log Record Format

Except where noted, all records added to the SAF Message Life Cycle Log for JMS messages contain the following pieces of information in the order in which they are listed:

Sample Log File Records

The sample log file records that follow show the type of information that is provided in the log file for each of the message life cycle events. Each record is a fixed length, but the information included will vary depending upon relevance to the event and on whether a valid value exists for each field in the record. The log file records use the following syntax:

####<date_and_time_stamp> <transaction_id> <WLS_diagnostic_context> <date_in_milliseconds> <date_in_nanoseconds> <JMS_message_id> <JMS_correlation_id> <JMS_destination_name> <life_cycle_event_name> <JMS_user_name> <consumer_identifier> <JMS_message_content> <JMS_message_selector>
Note: If you choose to include the JMS message content in the log file, note that any occurrences of the left-pointing angle bracket (<) and the right-pointing angle bracket (>) within the contents of the message will be escaped. In place of a left-pointing angle bracket you will see the string "&lt;" and in place of the right-pointing angle bracket you will see "&gt;" in the log file.

Message Stored Event

####<Nov 3, 2005 11:11:02 AM EST> <> <> <1131034262637> <391826> <ID:<864239.1131034262172.0>> <> <SenderSafmodule!safDestinations2!xsafQ2@SendingAgent@D1S1> <Stored> <system> <> <&lt;?xml version="1.0" encoding="UTF-8"?&gt;

&lt;mes:WLJMSMessage xmlns:mes=""&gt;&lt;mes:Header&gt;&lt;mes:JMSDeliveryMode&gt;NON_PERSISTENT&lt;/mes:JMSDeliveryMode&gt;&lt;mes:JMSExpiration&gt;0&lt;/mes:JMSExpiration&gt;&lt;mes:JMSPriority&gt;5&lt;/mes:JMSPriority&gt;&lt;mes:JMSRedelivered&gt;false&lt;/mes:JMSRedelivered&gt;&lt;mes:JMSTimestamp&gt;1131034262172&lt;/mes:JMSTimestamp&gt;&lt;mes:Properties/&gt;&lt;/mes:Header&gt;&lt;mes:Body&gt;&lt;mes:Text&gt;EndPointExpireAllWithMessageForward(NP):0&lt;/mes:Text&gt;&lt;/mes:Body&gt;&lt;/mes:WLJMSMessage&gt;> <>

Message Forwarded Event

####<Nov 3, 2005 11:11:30 AM EST> <> <> <1131034290391> <277193> <ID:<864239.1131034288312.0>> <> <SenderSafmodule!safDestinations3!safErrorQueue@SendingAgent@D1S1> <Forwarded> <<WLS Kernel>> <> <&lt;?xml version="1.0" encoding="UTF-8"?&gt;

&lt;mes:WLJMSMessage xmlns:mes=""&gt;&lt;mes:Header&gt;&lt;mes:JMSDeliveryMode&gt;NON_PERSISTENT&lt;/mes:JMSDeliveryMode&gt;&lt;mes:JMSExpiration&gt;0&lt;/mes:JMSExpiration&gt;&lt;mes:JMSPriority&gt;5&lt;/mes:JMSPriority&gt;&lt;mes:JMSRedelivered&gt;false&lt;/mes:JMSRedelivered&gt;&lt;mes:JMSTimestamp&gt;1131034288312&lt;/mes:JMSTimestamp&gt;&lt;mes:Properties/&gt;&lt;/mes:Header&gt;&lt;mes:Body&gt;&lt;mes:Text&gt;EndPointExpireAllWithMessageRedirect(NP):0&lt;/mes:Text&gt;&lt;/mes:Body&gt;&lt;/mes:WLJMSMessage&gt;> <>

Message Expired Event

####<Nov 3, 2005 1:04:25 PM EST> <> <> <1131041065929> <42424> <ID:<130865.1131041065828.0>> <> <udd-saf!safDestinations!safRemoteQueue@SendingAgent@D1C1S1> <Expired> <<WLS Kernel>> <> <&lt;?xml version="1.0" encoding="UTF-8"?&gt;

&lt;mes:WLJMSMessage xmlns:mes=""&gt;&lt;mes:Header&gt;&lt;mes:JMSDeliveryMode&gt;PERSISTENT&lt;/mes:JMSDeliveryMode&gt;&lt;mes:JMSExpiration&gt;1131041065848&lt;/mes:JMSExpiration&gt;&lt;mes:JMSPriority&gt;7&lt;/mes:JMSPriority&gt;&lt;mes:JMSRedelivered&gt;false&lt;/mes:JMSRedelivered&gt;&lt;mes:JMSTimestamp&gt;1131041065828&lt;/mes:JMSTimestamp&gt;&lt;mes:Properties/&gt;&lt;/mes:Header&gt;&lt;mes:Body&gt;&lt;mes:Text&gt;testSAFLogging_Expired&lt;/mes:Text&gt;&lt;/mes:Body&gt;&lt;/mes:WLJMSMessage&gt;> <>

Message Removed Event

####<Nov 3, 2005 11:11:08 AM EST> <> <> <1131034268206> <803337> <ID:<864239.1131034262172.0>> <> <SenderSafmodule!safDestinations2!xsafQ2@SendingAgent@D1S1> <Removed> <<WLS Kernel>> <> <&lt;?xml version="1.0" encoding="UTF-8"?&gt;

&lt;mes:WLJMSMessage xmlns:mes=""&gt;&lt;mes:Header&gt;&lt;mes:JMSDeliveryMode&gt;NON_PERSISTENT&lt;/mes:JMSDeliveryMode&gt;&lt;mes:JMSExpiration&gt;0&lt;/mes:JMSExpiration&gt;&lt;mes:JMSPriority&gt;5&lt;/mes:JMSPriority&gt;&lt;mes:JMSRedelivered&gt;false&lt;/mes:JMSRedelivered&gt;&lt;mes:JMSTimestamp&gt;1131034262172&lt;/mes:JMSTimestamp&gt;&lt;mes:Properties/&gt;&lt;/mes:Header&gt;&lt;mes:Body&gt;&lt;mes:Text&gt;EndPointExpireAllWithMessageForward(NP):0&lt;/mes:Text&gt;&lt;/mes:Body&gt;&lt;/mes:WLJMSMessage&gt;> <>

Managing SAF Agent Log Files

After you create a SAF agent, you can configure criteria for moving (rotating) old log messages to a separate file. You can also change the default name of the log file.

Rotating Message Log Files

You can choose to rotate old log messages to a new file based on a specific file size or at specified intervals of time. Alternately, you can choose not to rotate old log messages; in this case, all messages will accumulate in a single file and you will have to erase the contents of the file when it becomes too large.

If you choose to rotate old messages whenever the log file reaches a particular size you must specify a minimum file size. After the log file reaches the specified minimum size, the next time the server checks the file size it will rename the current log file and create a new one for storing subsequent messages.

If you choose to rotate old messages at a regular interval, you must specify the time at which the first new message log file is to be created, and then specify the time interval that should pass before that file is renamed and replaced.

For more information about setting up log file rotation for SAF agents, see " Configure SAF agent JMS message log rotation" in the Administration Console Online Help.

Renaming Message Log Files

Rotated log files are numbered in order of creation. For example, the seventh rotated file would be named myserver.log00007. For troubleshooting purposes, it may be useful to change the name of the log file or to include the time and date when the log file is rotated. To do this, you add java.text.SimpleDateFormat variables to the file name. Surround each variable with percentage (%) characters. If you specify a relative pathname when you change the name of the log file, it is interpreted as relative to the server's root directory.

For more information about renaming message log files for SAF agents, see " Configure SAF agent JMS message log rotation" in the Administration Console Online Help.

Limiting the Number of Retained Message Log Files

If you choose to rotate old message log files based on either file size or time interval, you may also wish to limit the number of log files this SAF agent creates for storing old messages. After the server reaches this limit, it deletes the oldest log file and creates a new log file with the latest suffix. If you do not enable this option, the server will create new files indefinitely and you will have to manually clean up these files.

For more information about limiting the number of message log files for SAF agents, see " Configure SAF agent JMS message log rotation" in the Administration Console Online Help.


Frequently Asked Questions About JMS SAF

This section answers commonly asked questions about how JMS SAF operates in a WebLogic domain.

Q. Which sending agent is picked by SAF for a JMS producer?

A. WebLogic Sever's cluster load balancing is used to pick a sending agent for a given JMS producer. Once a SAF agent is picked, it is used for the life of the JMS producer.

Q. How do JMS clients find a SAF destination?

A. A SAF destination can be found the same way a non-SAF JMS destination is found.

Q. Can a JMS producer sending messages to a JMS SAF imported destination be associated with a JMS Unit-of-Order?

A. Yes. For information about the Unit-of-Order feature, see For more information about configuring a Path service, see " Using Message Unit-of-Order" in Programming WebLogic JMS.

Q. Why does my JMS producer associated with a JMS Unit-of-Order fail to send messages if the sending-side is a cluster?

A. In order to use JMS Unit-of-Order with SAF, you must configure the Path Service for the sending-side cluster. For information about the Path Service feature, see " Using the WebLogic Path Service" in Configuring and Managing WebLogic JMS.

Q. Do different JMS producers in the same Unit-of-Order pick up the same Sending Agent?

A. Yes. JMS SAF uses the Path Service to route to the same Sending Agent.

Q. Can a consumer be attached to a JMS SAF Imported Destination?

A. No. JMS consumers can only be attached to actual JMS destinations.

Q. Can a distributed destination be imported?

A. Yes, it can be imported using its JNDI name.

Q. Where do I configure Server Affinity, Load Balancing Enabled, and Forward Delay for a distributed destination that is imported in a sending cluster?

A. Server Affinity and Load Balancing Enabled are configured in the JMS connection factory on which the JMS producer was created. A JMS connection factory creates a JMS connection; a JMS connection creates a JMS session; a JMS session creates a JMS producer. Forward Delay is configured on the JMS distributed destination.

Q. Are the Server Affinity and Load Balancing parameters configured on a JMS connection factory in the sending cluster or server honored on the receiving cluster or server where the JMS Destination resides?

A. Yes. These attributes on the sending cluster or server are honored on the receiving cluster or server. For information about Server Affinity and Load Balancing for distributed destinations, see " Configuring Distributed Destinations" in Configuring and Managing WebLogic JMS

Q. Do XA transactions on the sending-side of a cluster ever cross the JMS SAF boundary? In other words, can the receiving-side participate with a transaction from the sending- side?

A. No. Messages are not forwarded until the transaction is committed.

Q. Does JMS SAF preserve the order of messages sent in a JMS session from a sending-side to a JMS destination?

A. Yes.

Q. In the SAF Remote Context, should I configure a Principal Name or a Username/Password?

A. You can configure the Remote SAF Context anyway you want. Username and Passwords are stored in the JMS module, and the principal name is stored in a Credential Mapper configured in the sending-side domain.

  Back to Top       Previous  Next