Diagnosing Oracle Fusion Middleware Problems
To help diagnose Oracle Fusion Middleware problems, you can use its log files and the Diagnostic Framework for critical errors.
Oracle Fusion Middleware components generate log files containing messages that record all types of events, including startup and shutdown information, errors, warning messages, access information on HTTP requests, and additional information. The log files can be used to identify and diagnose problems. See Managing Log Files and Diagnostic Data for more information about using and reading log files.
Oracle Fusion Middleware includes a Diagnostic Framework which aids in detecting, diagnosing, and resolving problems. The problems that are targeted in particular are critical errors such as those caused by code bugs, metadata corruption, and customer data corruption, deadlocked threads, and inconsistent state.
When a critical error occurs, it is assigned an incident number, and diagnostic data for the error (such as log files) are immediately captured and tagged with this number. The data is then stored in the Automatic Diagnostic Repository (ADR), where it can later be retrieved by incident number and analyzed. See Diagnosing Problems for more information about the Diagnostic Framework.
You can view an aggregated list of problems using the Support Workbench page of Fusion Middleware Control:
From the WebLogic Domain menu, select Diagnostics, then Support Workbench.
The summary page aggregates the Support Workbench diagnostic information across all WebLogic Servers that are members of this WebLogic Domain and shows the number of problems for each server. Click the number in the Problems column to see information about the problems for a particular server.
Troubleshooting Common Problems and Solutions
There are some common problems and solutions for Oracle Fusion Middleware.
This section describes some common problems and solutions. It contains the following topics:
Running Out of Data Source Connections
If the database performance has slowed or you receive the following message in the Oracle WebLogic Server log files, you may have leaks in the data source connections:
No resources currently available in pool datasource name Any product functionality that depend on the datasource will not function as it can't connect database to get required data.
If you receive this message, monitor the connection usage from the Administration Console data source monitoring page:
- From Domain Structure, expand Services, then Data Sources.
- Click the data source that you want to monitor.
- Select the Monitoring tab, then the Statistics tab.
- If the table does not display Active Connection Current Count, click Customize this table.
- In Column Display, select Active Connection Current Count and move it from the Available to the Chosen box. Click Apply.
- In the table, note the number in the Active Connection Current Count column.
If the active current count for a data source keeps increasing and does not go down, this data source is leaking connections. Contact Oracle Support.
Using a Different Version of Spring
When you configure a Managed Server with JRF, Spring 2.0.6 is installed and is placed in the Oracle WebLogic Server system classpath. If a custom application running in a JRF environment requires a different version of Spring, you must use the Filtering ClassLoader mechanism to specify the version of Spring.
Oracle WebLogic Server provides the FilteringClassLoader mechanism so that you can configure deployment descriptors to specify explicitly that certain packages should always be loaded from the application, rather than being loaded by the system classloader. This allows you to use alternate versions of applications such as Spring or Ant.
For more information about using the FilteringClassLoader mechanism, see Using a Filtering ClassLoader in Developing Applications for Oracle WebLogic Server.
ClassNotFound Errors When Starting Managed Servers
If a Managed Server is started by Node Manager (as is the case when the servers are started by the Oracle WebLogic Server Administration Console or Fusion Middleware Control), you may receive a ClassNotFound error if Node Manager has not been configured to use the start scripts when starting Managed Servers. See Configuring Node Manager to Start Managed Servers for information about resolving this problem.
This section describes common problems and solutions when working with SSL configuration. It contains the following topics:
Components May Enable All Supported Ciphers
You should be aware that when no cipher is explicitly configured, some 12c (220.127.116.11) components enable all supported SSL ciphers including
DH_Anon (Diffie-Hellman Anonymous) ciphers.
At this time, Oracle HTTP Server is the only component known to set ciphers like this.
Configure the components with the desired cipher(s) if
DH_Anon is not wanted.
SSL Certificate Chain Required on Certain Browsers
When you configure SSL for Oracle HTTP Server, you may need to import the entire certificate chain (rootCA, Intermediate CAs and so on).
Certain browsers, for example Internet Explorer, require that the entire certificate chain be imported to the browsers for SSL handshake to work. If your certificate was issued by an intermediate CA, you will need to ensure that the complete chain of certificates is available on the browser or the handshake will fail. If an intermediate certificate in the chain expires, it must be renewed along with all the certificates in the chain ((such as the OHS server certificate).
keyUsage Extension Required for Certificates in JDK7
In JDK6, a self-signed certificate can contain the
keyUsage extension without enabling the
keyCertSign bit. This is rejected in JDK7.
Under JDK7, if using self-signed CA certificates, ensure that the
keyCertSign bit of the
keyUsage extension is set. Otherwise connections fail with an exception such as:
weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
The key usage extension defines the purpose (for example enciphering, signature, certificate signing) of the key contained in the certificate.
Conforming CAs must include this extension in certificates that contain public keys that are used to validate digital signatures on other public key certificates or CRLs.
keyCertSign bit is asserted when the subject public key is used for verifying signatures on public key certificates. When generating self-signed CA certificates in JDK7, therefore, you must ensure that the
keyCertSign bit of
keyUsage is on.
You can achieve this, for example, by:
Creating a self-signed JKS keystore with option ku:c=keyCertSign, and
migrating the certificate from the keystore to the root wallet which will be used by the SSL DB connection
orapki wallet jks_to_pkcs12 -wallet ./ -pwd password -keystore ./ewallet.jks-jkspwd password
Troubleshooting FIPS Configuration
For details about this topic, see Troubleshooting FIPS 140 Issues.
Need More Help?
You can find more solutions on My Oracle Support or use can use the Remote Diagnostic Agent.
You can find more solutions on My Oracle Support,
http://support.oracle.com. If you do not find a solution for your problem, log a service request.
You can also use the Remote Diagnostic Agent, as described in:
Using Remote Diagnostic Agent
Remote Diagnostic Agent (RDA) is a command-line diagnostic tool that provides a comprehensive picture of your environment. Additionally, RDA can provide recommendations on various topics, for example configuration and security. This aids you and Oracle Support in resolving issues.
Remote Diagnostic Agent is designed to reduce returns trips for additional data by collecting, through a single scripted execution, frequently used diagnostic data such as:
Operating System and Environment
Memory, CPU and Disk data
OS Patches and Packages
Network configuration and statistics
Targeted Oracle Products
Install Version and Patch information
All configuration and log files
Metrics (for example, Status Info, DMS Dump, MBean values)
RDA also offers pre and post install health checks. Use a pre install check to discover if your host environment complies with Fusion Middleware system requirements. Run a post install check to bring to light configuration settings which may cause issues if not changed. Don't wait for a problem to arise. Take advantage of the health checks today.
The tool has a small footprint in terms of disk space, CPU, and memory usage. RDA is designed to be as unobtrusive as possible; it collects data passively and does not modify the state of the software or environment to do so. A security filter can be enabled in order to remove potentially sensitive information, such as hostnames and IP addresses, from the collection.
For more information about RDA, see:
My Oracle Support document ID 1498376.1
The readme file, which is located at:
(UNIX) ORACLE_HOME/oracle_common/rda/README_Unix.txt (Windows) ORACLE_HOME\oracle_common\rda\README_Windows.txt