Managing Diagnostic and Message Logs

7 Managing Diagnostic and Message Logs

This chapter describes how to manage and configure diagnostic and message logs. Oracle Fusion Middleware components, including web services, generate log files containing messages that record all types of events, including startup and shutdown information, errors, warning messages, access information on HTTP requests, and so on. Each log message includes specific information such as time, component ID, and user to assist you in pinpointing and diagnosing problems that arise.

This chapter includes the following sections:

7.1 Introduction to Diagnosing Problems Using Logs

You can review log messages to diagnose problems with specific components, such as web services.

There are two categories of log files that you can reference to assist in diagnosing problems with web services:

Diagnostic logs—Enable you to access diagnostic data about specific feature components in Oracle Fusion Middleware. For more information, see "Overview of Diagnostic Logs for Web Services".

There is a set of predefined diagnostic loggers. You can configure your own diagnostic logger, as described in "Configuring Log Files for a Web Service".
Message logs—Enable you to view elements of the SOAP message request. You control message log creation using policies. For more information, see "Overview of Message Logs for Web Services".

For more information about logging in Oracle Fusion Middleware, see "Managing Log Files and Diagnostic Data" in Administering Oracle Fusion Middleware.

The following sections describe how to use diagnostic and message logs to diagnose problems. A set of sample logs is provided at the end of this section.

7.1.1 Overview of Diagnostic Logs for Web Services

Diagnostic logs enable you to access diagnostic data about specific feature components in Oracle Fusion Middleware.

The following sections describe how to view and manage diagnostic log files:

7.1.1.1 Setting the Log Level for Diagnostic Logs

You set the logging level for web service and OWSM components at the WebLogic Server level, using the Log Configuration page.

In addition, you can override the log levels set at the server level for a specific web service endpoint from the Web Service Endpoint page. The logging level set at the web service endpoint level must be "finer grained" than the level set at the WebLogic Server level. Otherwise, the logging level set at the WebLogic Server level will be used.

The following procedures describe how to set the log level for diagnostic logs at the WebLogic Server and web service endpoint levels. For more information, see "Setting the Level of Information Written to Log Files" in Administering Oracle Fusion Middleware.

To set the log level for diagnostics logs at the WebLogic Server level:

Navigate to the WebLogic Server for which you want to configure a logger.
1. In the navigation pane, expand WebLogic Domain.
2. Expand the domain.
3. Select the desired server from the list.
  
  The WebLogic Server home page is displayed.
From the WebLogic Sever menu, select Logs, then Log Configuration.

The Log Configuration page is displayed.
Select the Log Levels tab.

The list of loggers is displayed, as shown in Figure 7-1.

The Log Levels page shows the name of the logger, the current logging level, which you can edit, and the associated log file (for example, olh-handler). For information about configuring the log files, see "Configuring Log Files for a Web Service".

Figure 7-1 Log Levels Page

Description of "Figure 7-1 Log Levels Page"
Expand Root Logger.
Expand oracle.
Set the logging level for one or more of the following components:
- oracle.webservices—web service components.
- oracle.wsm—OWSM components.
You can fine tune the logging level by expanding either of the above components and specifying the logging level at the subcomponent level.

To change the logging level for a logger, navigate to the logger in the Logger Name column and select the desired logging level from the Oracle Diagnostic Logging Level (Java Level) drop-down menu.

For example, select TRACE:32 from the drop-down menu associated with the oracle.wsm logger.

By default, the logging levels are inherited from the parent and set to NOTIFICATION: 1 (INFO) for the web service and OWSM components and subcomponents.
Click Apply to store the new logging level.

To set the log level for diagnostic logs at the web service endpoint level:

Navigate to the Web Service Endpoint page, as described in "Introduction to Viewing Details for a Web Service Endpoint Using Fusion Middleware Control".
Click the Configuration tab.
Set the Logging Level field to one of the following settings: Severe, Warning, Information, Configuration, Fine, Finer, Finest or NULL.

7.1.1.2 Viewing Diagnostic Logs

You can view the diagnostic log files for an ADF web service endpoint from the Log Messages page.

To view diagnostic logs for a web service endpoint:

Navigate to the Web Service Endpoint page, as described in "Introduction to Viewing Details for a Web Service Endpoint Using Fusion Middleware Control", and in the Quick Links section of the Web Services Endpoint page (top right), click Diagnostic Log.

Note:

You can view a summary of all faults incurred by the web services in your application. For more information, see "Overview of Monitoring Web Services".

The Log Messages page is displayed, as shown in the following figure.

Figure 7-2 Log Messages Page

Description of "Figure 7-2 Log Messages Page"

Click on a message in the message area to view more details at the bottom of the page. If desired, you can export a message to a text, XML, or CSV file by selecting the messages on the list and clicking Export Messages to File.

You can control the message content displayed using the following controls:

Search—Modify the search criteria. For more information, see "Filtering Diagnostic Logs".
View menu—Select the columns to display in the table. Click on a particular column to sort contents up or down.
Show menu—Group messages by type or ID, or view them in chronological order.
View Related Messages—View messages related to those selected on the list.
—Broaden the scope of messages displayed. You can broaden the scope to include all messages for the domain, cluster, application deployment, or WebLogic Server.
Refresh menu—Specify an automatic or manual refresh rate.

To view the contents of a generated log file:

Click the log file icon associated with a message to view the contents of that log file.
Click Target Log Files... to display the Log Files page and view or download the contents of all generated log files.

For more information, see "Viewing and Searching Log Files" in Administering Oracle Fusion Middleware.

7.1.1.3 Filtering Diagnostic Logs

By default, the Log Messages page displays a summary of diagnostic messages logged over the last hour.

To filter diagnostic logs:

Filter the messages that are displayed by updating the search criteria using the following fields:
- Date Range—Set the date range to one of the following:
  - Most Recent—Set the amount of time to define the duration.
  - Time Interval—Set the start and end dates to define the interval.
- Message Types—Select the message types that you want to display.
- Add Fields—Add other message fields to your search criteria, such as Message ID, Component, and so on.
Click Search once you have set the fields, as desired.

The messages area is updated with the filtered results.

For more information, see "Viewing and Searching Log Files" in Administering Oracle Fusion Middleware.

7.1.1.4 Logging OWSM Debug Messages

To debug OWSM, pass one of the following properties when starting WebLogic Server, as required. For more information, see "Starting and Stopping Servers" in Administering Server Startup and Shutdown for Oracle WebLogic Server.

Note:

Enabling one or more of these properties may negatively impact performance for very large messages. When enabled, OWSM creates temporary buffers which will result in additional load on the Java garbage collector.

Table 7-1 Startup Properties for Logging OWSM Debug Messages

Startup Property	Description
`-Dxml.debug.verify=true`	Logs the sequence of bytes produced during a signature verification failure. Verification errors are output to `stderr` and the diagnostic log file when the log level is set to at least ERROR.
`-Dxml.debug.digest=true`	Verifies that the sequence of bytes produced during signature generation canonicalization and signature verification match. Verification errors are output to `stderr` and the diagnostic log file when the log level is set to at least FINE.
`-Dxml.debug.decrypt`	Logs the sequence of bytes produced following a decryption failure before XML parsing. Verification errors are output to `stderr` and the diagnostic log file.

7.1.2 Overview of Message Logs for Web Services

Message logs enable you to access the contents of the SOAP message requests and responses for ADF web services and clients. Messages logs are stored in a log file separate from the diagnostic messages, by default.

The following sections describe how to view and manage message log files:

7.1.2.1 Configuring Message Logs

You configure message logs for a web service or client in one of the following ways:

Attach a policy that contains a logging assertion to the web service or client.

There is one predefined logging assertion template: oracle/security_log_template, described in "oracle/security_log_template" in Securing Web Services and Managing Policies with Oracle Web Services Manager. This template is configured to log the entire SOAP message for the web service request and response. By default, all predefined web service security policies use this logging assertion to capture the entire SOAP message before and after the primary security assertion is executed. By default, the log assertion is not enforced. You must enable it in order for the SOAP message to be logged in message logs, as described in "Enabling or Disabling Assertions Within a Policy" in Securing Web Services and Managing Policies with Oracle Web Services Manager. It is recommended that the logging assertion be enabled for debugging and auditing purposes only.
Attach the oracle/log_policy policy to the web service or client. For more information, see "oracle/log_policy" in Securing Web Services and Managing Policies with Oracle Web Services Manager.
Create your own logging policy or assertion template to further refine the elements of the SOAP message that are logged for the web service request and response.

For example, you may wish to view only the SOAP body of the request message. To create a new policy, following the procedure described in "Creating a New Web Service Policy" in Securing Web Services and Managing Policies with Oracle Web Services Manager. You may wish to create a copy of the oracle/security_log_template assertion template and configure it for use in the new policy. For more information about creating a new assertion template, see "Cloning an Assertion Template" in Securing Web Services and Managing Policies with Oracle Web Services Manager.

7.1.2.2 Viewing Message Logs

You can view the message log files for an ADF web service endpoint from the Log Messages page.

To view message logs for a web service endpoint:

The Log Messages page is displayed, similar to Figure 7-2. For more details about the contents of the Log Messages page, see "Viewing Diagnostic Logs".

7.1.2.3 Filtering Message Logs

By default, the Log Messages page displays a summary of SOAP messages logged over the last hour. You can filter the messages that are displayed by updating the search criteria. The process is the same as for diagnostic logs; for more information, see "Filtering Diagnostic Logs".

By default, the Component and Module message fields are included as part of the Search criteria for message logs. The Component field is set to the WebLogic Server name; the Module field is set to oracle.wsm.msg.logging, which is the name of the message logging component.

7.1.3 Sample Logs

The following sections provide excerpts from sample logs, demonstrating how to diagnose specific problems using the log entries.

7.1.3.1 Sample Log: OWSM Policy Manager Not Available

The following sample log excerpt indicates that the OWSM Policy Manager is down. To resolve this issue, restart the wsm-pm application, as described in "Starting and Stopping Applications" in Administering Oracle Fusion Middleware.

2009-02-16 16:21:28,029 [[ACTIVE] ExecuteThread: '4' for queue:
 'weblogic.kernel.Default (self-tuning)'] 
 ERROR policymgr.PolicyManagerModelBean logp.251 - 
 Service lookup failed with URL:t3://host.example.com:7001/wsm-pm
 oracle.wsm.policymanager.PolicyManagerException: WSM-02118 : 
 The query service cannot be created.
...

7.1.3.2 Sample Log: Security Keystore Not Configured

The following sample log excerpt indicates that an OWSM security policy with message protection was applied, but the keystore was not configured. To resolve this security fault, configure the keystore, as described in "Configuring Keystores for Message Protection" in Securing Web Services and Managing Policies with Oracle Web Services Manager.

Feb 16, 2009 5:29:56 PM oracle.wsm.common.logging.WsmMessageLogger logSevere 
 SEVERE: The specified Keystore file /scratch/sbollapa/stage131/user_projects/domains/sai131_domain/config/fmwconfig/default-keystore.jks 
 cannot be found; it either does not exist or its path is not included in the 
 application classpath.
 Feb 16, 2009 5:29:56 PM oracle.wsm.common.logging.WsmMessageLogger logSevere
 SEVERE: Keystore is not properly configured in jps config.
 Feb 16, 2009 5:29:56 PM oracle.wsm.common.logging.WsmLogUtil log
 SEVERE: failure in OWSM Agent processRequest, category=security,
 function=agent.function.client, application=default, composite=pe3test3,
 modelObj=Service1, + policy=null, policyVersion=null, assertionName=null
 oracle.wsm.common.sdk.WSMException: WSM-00101 : The specified Keystore file 
 /scratch/sbollapa/stage131/user_projects/domains/sai131_domain/config/fmwconfig/default-keystore.jks cannot be found; 
 it either does not exist or its path is not included in the application classpath.
...

7.1.3.3 Sample Log: Certificate Not Available

The following sample log excerpt indicates that an OWSM security policy with message protection was applied that required a security certificate that was not available in the keystore. To resolve this security fault, configure the keystore with a certificate, as described in "Obtaining a Trusted Certificate and Importing it into the Keystore" in Securing Web Services and Managing Policies with Oracle Web Services Manager.

[2009-04-15T04:07:02.821-07:00] [jrfServer] [ERROR] [WSM-000062]
 [oracle.wsm.resources.security] [tid: [ACTIVE].ExecuteThread: '0' for 
queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] 
 [ecid: 0000I2dTFG7DScT6uBe9UH19tRyv000000,0:1] [WEBSERVICE_PORT.name:
 NonCAAsCAMessageProtectionPolicyPort] [APP: jaxwsservices] 
 [J2EE_MODULE.name: NonCAAsCAMessageProtectionPolicy] [WEBSERVICE.name:
 NonCAAsCAMessageProtectionPolicyService] [J2EE_APP.name: jaxwsservices] 
 [arg: oracle.wsm.security.SecurityException: WSM-00062 : 
 The path to the certificate used for the signature is invalid.] 

[2009-04-15T04:07:02.810-07:00] [jrfServer] [NOTIFICATION] []
 [oracle.wsm.security.policy.scenario.processor.Wss11X509TokenProcessor] 
 [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default
 (self-tuning)'] [userId: <anonymous>] 
[ecid: 0000I2dTFG7DScT6uBe9UH19tRyv000000,0:1] 
[WEBSERVICE_PORT.name: NonCAAsCAMessageProtectionPolicyPort] 
[APP: jaxwsservices] [J2EE_MODULE.name: NonCAAsCAMessageProtectionPolicy]
 [WEBSERVICE.name: NonCAAsCAMessageProtectionPolicyService] [J2EE_APP.name:
 jaxwsservices] Certificate path validation failed for signing certificate

[2009-04-15T04:07:02.821-07:00] [jrfServer] [ERROR] [WSM-00006]
 [oracle.wsm.resources.security] [tid: [ACTIVE].ExecuteThread: '0' for queue:
 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] 
[ecid: 0000I2dTFG7DScT6uBe9UH19tRyv000000,0:1] [WEBSERVICE_PORT.name:
 NonCAAsCAMessageProtectionPolicyPort] [APP: jaxwsservices] 
[J2EE_MODULE.name: NonCAAsCAMessageProtectionPolicy] [WEBSERVICE.name:
 NonCAAsCAMessageProtectionPolicyService] [J2EE_APP.name: jaxwsservices] 
[arg: oracle.wsm.security.SecurityException: WSM-00062 : The path to the
 certificate used for the signature is invalid.] Error in receiving the request:
 oracle.wsm.security.SecurityException: WSM-00062 : The path to the certificate
 used for the signature is invalid.

7.2 Configuring Log Files for a Web Service

To further organize your logging data, you can configure the log files for a web service.

You can configure log files for SOA and ADF services.

The following table defines the default log files that are relevant to OWSM.

Table 7-2 Default Log Files for OWSM

Default Log File	Description
odl-handler	Logs general diagnostic data for the Java EE components in the server.
owsm-message-handler	Logs SOAP messages as per OWSM logging policies.

The following procedure describes how to set the log level for diagnostic logs at the WebLogic Server and web service endpoint levels.

For more information about using Fusion Middleware Control or WLST to set the log levels, see "Setting the Level of Information Written to Log Files" in Administering Oracle Fusion Middleware.

To configure the log files for a web service:

Navigate to the WebLogic Server for which you want to configure a logger.
1. In the navigation pane, expand WebLogic Domain to view the domain name.
2. Expand the domain to see the list of servers.
3. Select the desired server from the list.
  
  The WebLogic Server home page is displayed.
From the WebLogic Server menu, select Logs > Log Configuration.

The Log Configuration page is displayed.
Select the Log Files tab.

The current list of log files is displayed. The Log Configuration page shows the currently configured log path, file format, and rotation policy.
If you wish to edit the log policy configuration, select the log file in the list and click Edit Configuration . . ..

The Edit Log File page is displayed.

Figure 7-3 Edit Log File Page

Description of "Figure 7-3 Edit Log File Page"

Edit the log file information, as required.

Table 7-3 Fields in Edit Log File Page

Field	Description
Log Path	Path to the log file. This field is required.
Log File Format	Format of the log file. Valid values are text or XML.
Log Level	Default log level for the logger. Select a log level from the list. Valid values include: INCIDENT_ERROR:1 (SEVERE+100) ERROR:1 (SEVERE) WARNING:1 (WARNING) NOTIFICATION:1 (INFO) NOTIFICATION:16 (CONFIG) TRACE:1 (FINE) TRACE:16 (FINER) TRACE:32 (FINEST)
Use Default Attributes	Flag that specifies whether to use default attributes for the logger.
Supplemental Attributes	Supplemental attributes required.
Loggers to Associate	Components to associate with the logger.
Rotation Policy	Specify whether you wish to rotate log files based on file size of length of time. For more information, see "Configuring Log File Rotation" in Administering Oracle Fusion Middleware. If Size Based is selected as the rotational policy, Maximum Log Files Size is a required field. If Time Based is selected as the rotational policy, Frequency is a required field.

Click OK to edit the log file configuration.