Oracle® Fusion Applications Administrator's Troubleshooting Guide 11g Release 7 Refresh 2 (11.1.7) Part Number E25450-08 |
|
|
PDF · Mobi · ePub |
This chapter describes common problems that you might encounter when using Oracle SOA Suite and explains how to solve them.
This chapter contains the following topics:
Section 11.1, "Introduction to Troubleshooting Oracle SOA Suite"
Section 11.2, "Getting Started with Troubleshooting and Logging Basics for Oracle SOA Suite"
Section 11.4, "Security and Oracle WSM Policy Manager Configuration"
Some procedures in this chapter reference content in the Oracle Fusion Middleware guides. These guides describe using Fusion Middleware Control. These procedures also apply to Fusion Applications Control.
To gain insight into the log details generated by Oracle Fusion applications, see Chapter 2, "Troubleshooting Oracle Fusion Applications Using Incidents, Logs, QuickTrace, and Diagnostic Tests." Also, review the Oracle Fusion Middleware Error Messages Reference for information about the error messages you may encounter.
Note:
Some sections of this chapter describe how to set properties in the System MBean Browser of Fusion Applications Control. Some MBean properties are applicable to Oracle WebLogic Server and others are applicable to the SOA Infrastructure. Oracle WebLogic Server Mbeans are properties that impact the Java Virtual Machine (JVM) server level process that runs on the operating system. Example properties include the port on which the server is listening, and so on. These properties are agnostic to the type of application running on that server. SOA Infrastructure level Mbeans are properties that impact the SOA Infrastructure application running on the same Oracle WebLogic Server. Example properties include audit levels, transaction retries, and so on.
This section provides guidelines and a process for using the information in this chapter. Using the following guidelines and process will focus and minimize the time you spend resolving problems.
Guidelines
When using the information in this chapter, Oracle recommends:
After performing any of the solution procedures in this chapter, immediately retrying the failed task that led you to this troubleshooting information. If the task still fails when you retry it, perform a different solution procedure in this chapter and then try the failed task again. Repeat this process until you resolve the problem.
Making notes about the solution procedures you perform, symptoms you see, and data you collect while troubleshooting. If you cannot resolve the problem using the information in this chapter and you must log a service request, the notes you make will expedite the process of solving the problem.
Process
Follow the process outlined in Table 11-1 when using the information in this chapter. If the information in a particular section does not resolve your problem, proceed to the next step in this process.
Table 11-1 Process for Using the Information in this Chapter
Step | Section to Use | Purpose |
---|---|---|
1 |
Get started troubleshooting Oracle SOA Suite. The procedure in this section quickly addresses a wide variety of problems. |
|
2 |
Section 11.3 through Section 11.9 |
Perform problem-specific troubleshooting procedures. These sections describe:
|
3 |
Use My Oracle Support to get additional troubleshooting information about Oracle Fusion Applications or Oracle SOA Suite. My Oracle Support provides access to several useful troubleshooting resources, including Knowledge Base articles and Community Forums and Discussions. |
|
4 |
Log a service request if the information in this chapter and My Oracle Support does not resolve your problem. You can log a service request using My Oracle Support at |
SOA troubleshooting must be performed in the following scenarios:
The Oracle Fusion application transaction flow has an error and it has been identified as an error in SOA (or you want to check if it is a SOA-related error).
The Oracle Fusion application transaction flow has not completed and is taking longer than it should and it has been identified as stuck in SOA (or you want to check if it is stuck in SOA).
The Oracle Fusion application transaction flow is not working as expected, and the wrong data is passed.
The recommended approach to troubleshooting SOA issues is as follows:
Check the server and composite status log in Oracle WebLogic Server Administration Console and check if the SOA server is up and running successfully (that is, not in a failed state).
Log in to Oracle WebLogic Server Administration Console.
In the Domain Structure, click Deployments.
In the Name column of the Deployments section, find soa-infra.
Check that the State column is set to Active and the Health column is set to OK.
Click soa-infra > Monitoring tab.
Expand to ensure that all components have Health set to OK.
Note the server name on which SOA is deployed.
Go to the Domain Structure and click Environment > Servers to verify that the State column is set to RUNNING and the Health column is set to OK.
Note the Listen Port column value, as this is the SOA runtime port.
Check if any Oracle Fusion application incident was created for the problem encountered.
For information about Oracle Fusion application incident processing, see Section 2.2, "Investigating, Reporting, and Solving a Problem."
Check whether the event was delivered to the SOA cluster by checking the log messages in Fusion Applications Control.
This requires the oracle.integration.platform.blocks.event logger to be set to TRACE:32 (FINEST). There are potentially many types of exceptions that can appear in the log messages on the client side that raised the event. For example:
If a Java class is not in the class path, then a ClassNotFoundException
may appear.
If the Java Naming and Directory Interface (JNDI) for the context or connection factory is not configured properly, then a NamingException
or FabricException
may appear.
If the event payload XML that you create is invalid or contains invalid characters, you may receive a FabricException
indicating that enqueueing of the event fails at the PL/SQL API with invalid characters, and so on.
For information about enabling this log and troubleshooting business events, see Section 11.3.2.
Check the composite instance flow. If the event was fired, then find your composite instance based on the application data.
See Section 11.3.1 for primary key mapping to the composite instance ID.
Review the composite instance flow and fault details. If the instance is in error and it is recoverable, attempt recovery using the Recovery tab of the BPEL process service engine in Fusion Applications Control.
View the logs.
If the problem still cannot be solved, increase the log level of the system to debug the transactions. See the list of loggers and log levels in the "Troubleshooting Oracle SOA Suite and Oracle BPM Suite" appendix of Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite. To simplify troubleshooting, it is recommended that you enable the following parent loggers at the TRACE32 (FINEST) level in Fusion Applications Control:
oracle.soa
oracle.fabric
oracle.integration
The oracle.wsm logger can remain set to the ERROR level where it logs the required error messages. The oracle.apps logger should be set to ALL. To change logger levels, perform the following steps:
Go to Fusion Applications Control.
In the navigation pane, select WebLogic Domain.
Right-click a Managed Server from within the domain (each server's log levels can be independently set).
Choose Logs > Log Configuration.
In the Logger Name column, expand the oracle runtime loggers to display loggers such as oracle.soa.
Change the logging level to TRACE:32. The change should take effect within a few minutes.
Note that in a production system, setting the trace at a fine-grained level can result in a large amount of output that must be diagnosed. You can alternately use selective tracing that provides a way to get a detailed, on-disk trace selectively (for example, by user name, thereby eliminating trace output for other users).
To activate selective tracing, right-click the domain under WebLogic Domain and choose Logs > Selective Tracing.
Note that Selecting Tracing does not display as an option when you right-click an Administration Server or Managed Server and choose Logs.
From the Option Name list, choose the type of selective trace (for example, based on user name), and start the trace.
When the problem has been reproduced, disable the trace and view the output to narrow down the issue.
For more information on selective tracing, see the "Configuring and Using Selective Tracing" section of Oracle Fusion Middleware Administrator's Guide.
Review the error logs (from Fusion Applications Control) for more information on the error.
Cross layer, server, and family functionality can be correlated through the execution context ID (ECID) (for example, you can look up the composite instance for a given expense report by correlating all the log entries with the ECID associated with that expense report transaction). For more information, see the "Correlating Messages Across Log Files and Components" section of Oracle Fusion Middleware Administrator's Guide.
Check for incorrect configurations and any networking issues, especially around potentially incorrect settings of the following:
External load balancers
Oracle HTTP Server
Virtual host/IP address
Oracle WebLogic Server front-end URL
SSL host name verification settings
For more details, see the Oracle Fusion Middleware Enterprise Deployment Guide for Oracle SOA Suite.
Verify that custom applications have been deployed on the correct servers using the correct configuration plan during deployment.
For more details, see the "How to Use Configuration Plans to Customize SOA Composite Applications for the Target Environment" section of Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite.
Contact Oracle Support Services.
If the error still cannot be resolved, file a ticket with Oracle Support Services and provide the logs and information shown in Table 11-2.
Table 11-2 SOA Log Information for Oracle Support Services
Log | Description |
---|---|
Application diagnostic log |
View the following log: $DOMAIN_HOME/servers/domain_name/logs/apps/server_name-diagnostic.log |
Server diagnostic log |
View the following log: $DOMAIN_HOME/servers/server_name/logs/server_name-diagnostic.log For example, This is where the log output is available. By default, only the last 100 MB of the diagnostic logs are retained. |
Server log |
View the following log: server_name.log
For example, |
Server console output |
|
Server thread dump |
Enter the following at the operating system command prompt: kill -3 managed_server_process_ID
You can also use Oracle WebLogic Server Administration Console.
The output is in the console logs. |
OPatch thread dump |
kill -3 opatch_client_process_ID
|
OWSM message log |
The following log captures all SOAP messages on the wire. $DOMAIN_HOME/servers/server_name/logs/owsm/msglogging/diagnostic.log
This log is not enabled by default. To enable this log:
|
For more information about setting logging levels for SOA components, see the "Setting Logging Levels for Troubleshooting" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Send Oracle Support Services information about the error packaged with Automatic Diagnostic Repository-based incident processing.
For information about how to investigate, report, and, in some cases, resolve a problem, see the "Investigating, Reporting, and Solving a Problem" section of Oracle Fusion Middleware Administrator's Guide.
This section contains the following topics that describe common problems and solutions for Oracle SOA Suite runtime:
Section 11.3.1, "Correlating Application Issues to the SOA Composite IDs"
Section 11.3.2, "Business Event Subscriptions Are Not Firing"
Section 11.3.3, "Long Delays After an Event is Published and Before a Subscriber is Triggered"
Section 11.3.4, "Events Are Consumed by Multiple Revisions of the Same Composites"
Section 11.3.5, "Rolled Back OAOO Event Delivery Messages Are Displayed in the Log Files"
Section 11.3.7, "BPEL Process Received No Response from an ADF Asynchronous Service"
Section 11.3.8, "Business Event Is Picked Up Twice (Or More) By SOA Server"
Section 11.3.10, "Some Messages Are Lost Between EDN and Composites or Composites Across Clusters"
Section 11.3.11, "Some Composites Are Being Retried Multiple Times on Failure"
Section 11.3.13, "Automatic Recovery of BPEL Instances is Not Recovering A Specific Instance"
Section 11.3.14, "SOA Runtime Fails with a "Cannot read WSDL" Error"
Section 11.3.15, "Uploading a Composite for Oracle Support Services"
Section 11.3.16, "Confirming SOA Component Configuration Properties for Oracle Support Services"
Problem
How does an administrator translate an issue faced in the application to a composite instance in Fusion Applications Control for debugging purposes? For example, a purchase order transaction is stuck in processing for days and the administrator wants to see what the composite instance is doing. How does the administrator map the purchase order ID to the composite instance ID?
Solution
Oracle Fusion Applications modules write context-specific and business user-identifiable keys to the log files. This includes primary keys for any of the logical entities processed by the composite. After the primary key and other context information is included in the log messages, the administrator can search for that context (for example, purchase order ID) in the log file and derive the composite instance ID and ECID. From there, they can diagnose the issue in Fusion Applications Control. Usually, the primary key information is written to the logs in case of any error/incident/message level.
Cross layer, server, and family functionality are all designed to be correlated through the ECID. For example, you can look up the composite instance for a given expense report by correlating all the log entries with the ECID associated with that expense report transaction.
During a specific time interval, you may create several hundred instances., This is a fairly common use case. If there is no error, or if the application has not implemented a BPEL process with sensor logging (should not be common), then instead of searching for the primary key, perform one of the following tasks:
Try using the approximate timestamp of the transaction.
Inspect the input payload of the audit trail flow (until you find the right one) to identify the right composite.
Any extensions/customizations are also expected to log the same context-specific keys. You can set AF_LOG
and AF_LOG_MODULE
properly (such as setting log channel levels) to specify finer-grained logging from a particular family/logical business area (LBA)/composite.
Problem
When a business event is published, the business event subscription defined in the composite does not fire and a composite instance is not created.
This can occur, for example, with an ADF application invoking a SOA composite.
Solution
To resolve this problem.
Enable EDN logging in Fusion Applications Control. The following server loggers specific to EDN are available for selection:
oracle.integration.platform.blocks.event
oracle.integration.platform.blocks.event.saq
oracle.integration.platform.blocks.event.jms
Note that oracle.integration.platform.blocks.event.jms only appears if EDN is running in EDN-JMS mode instead of the default EDN-DB mode.
You can set the server loggers to one of the following levels:
TRACE:1 (FINE)
TRACE:16 (FINER)
TRACE:32 (FINEST)
You can alternately use selective tracing to get detailed, on-disk trace selectively (for example, by user name, thereby eliminating trace output for other users). See Section 11.2 for details.
Detailed logging goes into SOA server's diagnostic.log
file configured in Fusion Applications Control. To set the log level for the loggers:
Go to the navigation pane.
Right-click soa-infra.
Choose Logs > Log Configuration.
Expand oracle.integration > oracle.integration.platform > oracle.integration.platform.blocks > oracle.integration.platform.blocks.event.
Set the loggers described in step 1 to an appropriate logging level.
The following sample shows a portion of a server log file:
[2011-03-22T11:52:37.038-07:00] [soa_server1] [TRACE] [SOA-31010] [oracle.integration.platform.blocks.event.saq] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@96bab0] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1:100000140] [SRC_CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [APP: soa-infra] [composite_name: MediatorPubSub] [component_name: PublishEvent] [component_instance_id: 8E0411C054B511E0AF455DABE1395E7B] [SRC_ METHOD: fineDequeuedEvent] [composite_instance_id: 90087] Dequeued event, Subject: null [source type "J"]:[[ <business-event xmlns:ns="http://schemas.oracle.com/events/edl/MyEventDefn" xmlns="http://oracle.com/fabric/businessEvent"> <name>ns:MyEvent</name> <id>a7ae9d28-9530-4049-8385-e0ebfb0eea50</id> <source>default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/PublishEvent</source> <tracking> <ecid>5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83</ecid> <compositeInstanceId>90087</compositeInstanceId> <parentComponentInstanceId>mediator:8E0411C054B511E0AF455DABE1395E7B</parentCom ponentInstanceId> </tracking> <content> <inp1:singleString xmlns:inp1="http://xmlns.oracle.com/singleString">PO123</inp1:singleString> </content> </business-event> ]] [2011-03-22T11:52:37.042-07:00] [soa_server1] [TRACE] [SOA-31011] [oracle.integration.platform.blocks.event.saq] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@96bab0] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1:100000140] [SRC_CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [APP: soa-infra] [composite_name: MediatorPubSub] [component_name: PublishEvent] [component_instance_id: 8E0411C054B511E0AF455DABE1395E7B] [SRC_ METHOD: fineFilterResults] [composite_instance_id: 90087] Filter [XPath Filter: /be:business-event/be:content/ns1:singleString = 'PO123'] for subscriber "default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/SubsEvent" returned true . . . . . .
For more information about setting logging levels for Oracle SOA Suite, see the "Setting Logging Levels for Troubleshooting" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Check whether bad composites exist in the SOA domain that slow down the overall EDN event delivery.
EDN internally uses two advanced queuing (AQ) queues in a serial processing fashion for all business events. One or more bad SOA composite applications that fail to consume certain types of events can slow down EDN delivery of all other event types and cause an event backlog in the AQ queues.
Check if there are repeated errors from bad SOA composite applications in the SOA server log with EDN logging enabled. Even if the errors are in completely separate SOA composite applications that are unrelated to the event types of interest, it can delay the overall delivery of events of interest to the target SOA composite applications.
Check the event backlog count as detailed in Step d (events are stuck in advanced queue). Obtain the counts by running the following SQL queries (substitute SOAINFRA-SCHEMA-USER
with the schema owner (for example, HCM_FUSION_SOAINFRA
).
SELECT COUNT(*) FROM SOAINFRA-SCHEMA-USER.EDN_OAOO_DELIVERY_TABLE; SELECT COUNT(*) FROM SOAINFRA-SCHEMA-USER.EDN_EVENT_QUEUE_TABLE;
If bad composites are found in Step a and large numbers of backlogged events occur in Step b, then proceed to c.
Stop bad composites identified in Step a. This can be achieved from Oracle Enterprise Manager Fusion Middleware Control or the WLST command line utility.
Temporarily increase the number of EDN threads to a larger number, (for example, from 3
to 5
or 7
) to clear up the backlogged events faster.
To change the number of EDN threads through the configuration of the EDNConfig MBean in Oracle Enterprise Manager Fusion Middleware Control:
Go to the navigation pane.
Right-click soa-infra.
Select SOA Infrastructure > Administration > System MBean Browser.
Expand Application Defined MBeans > oracle.as.soainfra.config > Server: soa_server-x > EDNConfig > edn.
Change the value for the NumberOfThreads attribute, and click Apply.
If you do not care about backlog event processing, you can purge the backed up events directly in AQ. The purge script is as follows:
DECLARE purge_options dbms_aqadm.aq$_purge_options_t; BEGIN purge_options.block := FALSE; DBMS_AQADM.PURGE_QUEUE_TABLE( queue_table => '&edn_user..edn_event_queue_table', purge_condition => NULL, purge_options => purge_options); DBMS_AQADM.PURGE_QUEUE_TABLE( queue_table => '&edn_user..edn_oaoo_delivery_table', purge_condition => NULL, purge_options => purge_options); END; / commit;
Periodically monitor the EDN backlog count (see Step b) to confirm that it is being reduced. If the clear-up rate is very slow, you can further increase the EDN thread number (see Step d).
Continue to monitor the EDN backlog count over a period of time. Also monitor the recent instances and faults in Oracle Enterprise Manager Fusion Middleware Control to see if composite instances are being created or new faults are showing up both across the board (that is, at SOA Infrastructure level) and for the composite of interest.
Once the backlog is cleared up, revert the number of EDN threads to the previous setting or an appropriate number.
Publish a business event of interest and verify that the target composite is triggered with expected instance creations.
Check whether the event publication is being delivered to the SOA cluster.
One way to do this is to use the log messages in Fusion Applications Control. The log messages include information about the incoming events and the event subject and payload. Before you can view the event information in the log messages, you must first set the oracle.integration.platform.blocks.event logger to TRACE:32 (FINEST), as described in step 1, and then raise the event again. To search for the event-related messages:
Right-click soa-infra, and choose Logs > View Log Messages.
Specify the date range.
In the Message Types section, select the Notification and Trace checkboxes.
To search for all events delivered to the SOA cluster, specify Message contains Dequeued event
as the search criteria, and click Search.
The subject associated with the event is displayed in the Message field and the business event namespace and local name, payload, ECID, and message details are displayed in the Supplemental Detail field.
To search for a particular event by event name, click Add Fields.
Select Supplemental Detail, and click Add.
Specify that Supplemental Detail contains either the event namespace or event local name as the search criteria.
Click Search.
Use additional information such as the time interval or data in the payload to identify the specific event message of interest.
The log message is generally a good way to identify whether the issue is with the publisher or subscriber. For example, if the subject is missing or the fmw-context
is missing, it typically means there is an issue with the publisher. If the information reported in the log message is correct, then it typically means there is a problem with the subscriber.
If you see the Dequeued
event
log message for your business event, then go to step 10.
If you do not see the Dequeued
event
log message, there may be issues with the following:
For This Issue... | See... |
---|---|
EDN is in paused mode and temporarily stopped delivering events to its subscribers. |
Step 4 |
Events are stuck in the advanced queue. |
Step 5 |
More than one SOA cluster is pointing to the same advanced queue. |
Step 6 |
There are internal SOA issues in the database layer. |
Step 7 |
The data source is configured to point to the wrong |
Step 8 |
There are issues with the Java, SOA, or PL/SQL code raising the event. |
Step 9 |
When patches are applied to the SOA cluster, EDN is automatically placed in paused mode to prevent delivery of events during patching. After patching is complete, EDN exits paused mode and resumes event processing. When a patch fails, EDN may remain in paused mode to prevent event subscriptions from firing after the failure. Follow these steps to identify if EDN is in paused mode, and restart it:
From the navigation pane, expand SOA and right-click soa-infra.
Choose Administration > System MBean Browser.
Go to Application Defined MBeans > oracle.as.soainfra.config > Server: SOA_cluster_name > EDNConfig > edn.
In the Attributes column, locate the Paused property.
If Paused is set to true, the listener threads are decreased to zero, which temporarily stops event delivery.
To restart event delivery, select false.
Click Apply.
At times, the message may be stuck in the advanced queue in different stages of delivery. Check the count of (potentially stuck) events currently in the following queue tables:
EDN_EVENT_QUEUE_TABLE
: Every event published is temporarily enqueued into this table (for edn_event_queue
).
EDN_OAOO_DELIVERY_TABLE
: Only events with a one-and-only-one (OAOO) delivery target(s) are temporarily enqueued into this table (for edn_oaoo_queue
).
You can also check the count in the edn-db-log
. The total number of messages in the EDN_EVENT_QUEUE_TABLE
and EDN_OAOO_DELIVERY_TABLE
are displayed at the top of the page.
Navigate to the following URL to see the total number of messages at the top of the page:
http://host_name:port/soa-infra/events/edn-db-log
You do not need to enable the edn-db-log
to view these messages.
There may be more than one SOA cluster pointing to the same SOAINFRA
schema. This is highly unlikely in a provisioned environment, but for environments that are created by developers, it may be an issue. In this case, the expected SOA runtime environment may not be receiving the events properly. It is expected that only one SOA runtime environment listens for business events on a SOAINFRA
schema. Use the following query to identify the list of SOA runtime environments subscribing to business events from a SOAINFRA
schema. Have only the required SOA runtime environment up, which points to the SOAINFRA
schema, and shut down the others.
Navigate to the edn-db-log
to see log messages that display each SOA cluster that is connecting to the EDN advanced queue:
http://host_name:port/soa-infra/events/edn-db-log
You do not need to enable the edn-db-log
to view these messages. The messages continue to be displayed in the edn-db-log
until the next time you click the Clear link. Here is a sample message that you see in the edn-db-log
:
Starting EDN bus. Timestamp=Wed Feb 23 10:11:26 PST 2011. Parameters: platform="weblogic", cluster="", server="soa_server1", domain="fusion_domain", admin server="AdminServer", host="adc2180440".
Alternatively, you can run the following select
statement against the SOAINFRA
schema. For example, use FIN_FUSION_SOAINFRA
for the SOA cluster in the Financials domain.
select * from V$SESSION where username like 'family_FUSION_SOAINFRA';
Verify that the event is properly being enqueued and dequeued by the underlying EDN PL/SQL procedures. You can do this using the edn-db-log
that displays all events, including their namespace, names, payload, and subject, which are enqueued and dequeued by the PL/SQL EDN procedures.
Navigate to the following URL:
http://host_name:port/soa-infra/events/edn-db-log
Click the Enable link.
Raise the business event again.
Click the Refresh link.
Note:
You must have the administrator privilege to enable/disable the edn-db-log
.
You must always disable the edn-db-log
after debugging to disable logging. This prevents excessive database growth in the EDN database log table. If the edn-db-log
remains enabled, then debugging messages related to events that are published/enqueued into the database and subscribed to/dequeued from the database continue to be persisted into certain EDN database log tables. This causes the table to grow indefinitely.
The EDNSource
and EDNLocalTxSource
data sources may be pointing to a different database than the EDNDataSource
and EDNLocalTxDataSource
data source connections of the SOA server. Ensure that these four data source connections on ADF and the SOA server are pointing to the same database schema.
Check or set the data sources in the Oracle WebLogic Server Administration Console:
In the Domain Structure, select Services > Data Sources.
In the Name column of the table, select data_source_name.
The connection pool and driver name are only available for generic data sources, and not for multidata sources.
Select the Connection Pool tab.
In the Driver Class Name field, set the data source.
For more information, see the "Configuring the SOA Infrastructure" chapter in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Check the log files in Fusion Applications Control for the code that raises the event to see if there were failures in raising the event or failures in the code that prevented the event from being raised.
For ADF, check the log files. Look for the ECID corresponding to the business key (for example, purchase order).
To enable event-related log messages in the server running the ADF application that raises the event, set the log level for the oracle.integration.platform.blocks.event logger to TRACE (32) on that server.
By the time you reach this step, you have confirmed that the business event is properly delivered to the SOA cluster. To identify whether any composite instances were created, perform the following steps:
Because the ECID is different for a subscription with a guaranteed consistency level and a subscription with an OAOO level defined for the same event, you must first identify the consistency level for the event subscription. To do this, right click soa-infra, choose Business Events, and click the Subscriptions tab. Find the composite that subscribes to the business event in the Components Subscriptions table and get the value from the Consistency Level column.
If the value is set to Guaranteed, then get the ECID from the Dequeued
event
log message.
If the value is set to One And Only One, then search for the log message with Message contains Dequeued OAOO event
as criteria and use the ECID from that message.
Locate the composite instance by right-clicking soa-infra, selecting the Instances tab, entering the ECID in the search criteria, and clicking Search.
If you see composite instances, then continue to step 13.
If you do not see any composite instances, this may be due to the following issues:
If a filter is being used, there may be an issue with the filter logic. Use Fusion Applications Control to review the event subscriptions and filters defined for the composite and the payload content.
Log in to Fusion Applications Control in the domain where the composite with the subscription is deployed.
Navigate to domain_name > SOA.
Right-click soa-infra (SOA_cluster_name), and choose Business Events.
Click the Subscriptions tab.
Review the information in the Event Name, Namespace, and XPath Filter columns of the Component Subscriptions table.
Alternatively, you can also check the SOA server diagnostic.log
file for filter-related logging. You must set the log level to at least TRACE:1 (FINE) for the oracle.integration.platform.blocks.event.saq logger. The following is an example log message that has the following statement pattern: Filter
[XPath Filter: ….]
for
subscriber
"…"
returned
true/false
.
[2011-03-22T11:52:36.976-07:00] [soa_server1] [TRACE] [SOA-31011] [oracle.integration.platform.blocks.event.saq] [tid: [ACTIVE].ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1] [SRC_ CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [WEBSERVICE_PORT.name: execute_pt] [APP: soa-infra] [composite_name: MediatorPubSub] [component_name: PublishEvent] [component_instance_id: 8E0411C054B511E0AF455DABE1395E7B] [J2EE_MODULE.name: fabric] [SRC_METHOD: fineFilterResults] [WEBSERVICE.name: PublishEvent_ep] [J2EE_APP.name: soa-infra] [composite_instance_id: 90087] Filter [XPath Filter: /be:business-event/be:content/ns1:singleString = 'PO123'] for subscriber "default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/SubsEvent" returned true
Check the SOA diagnostic log for stack traces related to the issue. For example, the SOA instance may have been rolled back due to an error.
By the time you reach this step, you have confirmed that the business event has been delivered to the SOA cluster and a composite instance has been created. However, you may be encountering issues with the composite components:
If you see a composite instance, but not a BPEL instance, check if the BPEL process can be executed independently by a client, as opposed to the event subscription.
If you see a composite instance, but not a BPEL instance, this symptom may indicate a stuck thread on a service call. In this scenario, the BPEL process service engine is hung on a service call and has not created the audit trail. To verify the behavior, take three thread dumps approximately 30 seconds apart on all servers in the cluster. See Table 11-2 for more information on taking thread dumps. If the same thread shows it is stuck after the three thread dumps (that is, after 1.5 minutes), the thread is stuck. In this case, wait for the call to time out, which then enables you to recover the instance.
If the audit trail shows the following message:
Invoked 1-way operation "initiate" on target service service_name
There may be an issue with an incorrect driver type for the data sources. Note that the driver XA configuration should already be created by the SOA/provisioning template. If the driver type is changed or is not set properly, routing from Oracle Mediator does not occur. Instead, it tries to route and fails several times. This can be confirmed by instances in Fusion Applications Control. As per Oracle Fusion Applications standards, the SOADataSource
and EDNDataSource
data sources should use the XA driver type; that is:
EDNDataSource
: oracle.jdbc.xa.client.OracleXADataSource
SOADataSource
: oracle.jdbc.xa.client.OracleXADataSource
Problem
After a business event is published, there is a long delay (minutes, hours, or even days) before the event subscriber is triggered and its composite instance is created.
This can occur, for example, with an ADF application invoking a SOA composite.
Solution
To resolve this problem, perform the following steps.
Enable EDN logging in Fusion Applications Control. The following server loggers specific to EDN are available for selection:
oracle.integration.platform.blocks.event
oracle.integration.platform.blocks.event.saq
oracle.integration.platform.blocks.event.jms
Note that oracle.integration.platform.blocks.event.jms only appears if EDN is running in EDN-JMS mode, instead of the default EDN-DB mode.
Set the server loggers to one of the following levels:
TRACE:1 (FINE)
TRACE:16 (FINER)
TRACE:32 (FINEST)
You can alternately use selective tracing to selectively get a detailed, on-disk trace (for example, by user name, thereby eliminating trace output for other users). See Section 11.2 for details.
Detailed logging goes into the server's diagnostic.log
file configured in Fusion Applications Control.
Set the log level for the loggers.
Go to the navigation pane.
Right-click soa-infra.
Select Logs > Log Configuration.
Expand oracle.integration > oracle.integration.platform > oracle.integration.platform.blocks > oracle.integration.platform.blocks.event.
Set the loggers described in step 1 to an appropriate logging level.
The following sample shows a portion of a server log file:
[2011-03-22T11:52:37.038-07:00] [soa_server1] [TRACE] [SOA-31010] [oracle.integration.platform.blocks.event.saq] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@96bab0] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1:100000140] [SRC_CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [APP: soa-infra] [composite_name: MediatorPubSub] [component_name: PublishEvent] [component_instance_id: 8E0411C054B511E0AF455DABE1395E7B] [SRC_ METHOD: fineDequeuedEvent] [composite_instance_id: 90087] Dequeued event, Subject: null [source type "J"]:[[ <business-event xmlns:ns="http://schemas.oracle.com/events/edl/MyEventDefn" xmlns="http://oracle.com/fabric/businessEvent"> <name>ns:MyEvent</name> <id>a7ae9d28-9530-4049-8385-e0ebfb0eea50</id> <source>default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/PublishEvent</source> <tracking> <ecid>5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83</ecid> <compositeInstanceId>90087</compositeInstanceId> <parentComponentInstanceId>mediator:8E0411C054B511E0AF455DABE1395E7B</paren tCom ponentInstanceId> </tracking> <content> <inp1:singleString xmlns:inp1="http://xmlns.oracle.com/singleString">PO123</inp1:singleString> </content> </business-event> ]] [2011-03-22T11:52:37.042-07:00] [soa_server1] [TRACE] [SOA-31011] [oracle.integration.platform.blocks.event.saq] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@96bab0] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1:100000140] [SRC_CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [APP: soa-infra] [composite_name: MediatorPubSub] [component_name: PublishEvent] [component_instance_id: 8E0411C054B511E0AF455DABE1395E7B] [SRC_ METHOD: fineFilterResults] [composite_instance_id: 90087] Filter [XPath Filter: /be:business-event/be:content/ns1:singleString = 'PO123'] for subscriber "default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/SubsEvent" returned true . . . . . .
For more information about setting logging levels for Oracle SOA Suite, see the "Setting Logging Levels for Troubleshooting" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Find out when the event is published, dequeued, and sent to the subscriber by EDN.
Use the Java logging in step 1 to identify the timestamps of event publishing, dequeuing, and sending to the subscriber. From these timestamps, you can identify if the delay is on the publishing side or the EDN delivery side.
To search for the event-related messages in the log messages.
Right-click soa-infra.
Select Logs > View Log Messages.
Specify the date range.
To search for all events delivered to the SOA cluster, specify Message contains Dequeued
event
as the search criteria, and click Search.
The subject associated with the event is displayed in the Message field and the business event namespace and local name, payload, ECID, and message details are displayed in the Supplemental Detail field.
To search for a particular event by event name, click Add Fields.
Select Supplemental Detail, and click Add.
Specify that Supplemental Detail contains either the event namespace or the event local name as the search criteria.
Click Search.
Use additional information such as the time interval or data in the payload to identify the specific event message of interest.
The following sample shows a portion of a server log file containing information about event publishing with a timestamp of 2011-03-22T11:52:36.968-07:00
.
[2011-03-22T11:52:36.968-07:00] [soa_server1] [TRACE] [SOA-31000] [oracle.integration.platform.blocks.event.saq] [tid: [ACTIVE].ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1] [SRC_ CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [WEBSERVICE_PORT.name: execute_pt] [APP: soa-infra] [composite_name: MediatorPubSub] [component_name: PublishEvent] [component_instance_id: 8E0411C054B511E0AF455DABE1395E7B] [J2EE_MODULE.name: fabric] [SRC_METHOD: fineEventPublished] [WEBSERVICE.name: PublishEvent_ep] [J2EE_APP.name: soa-infra] [composite_instance_id: 90087] Received event: Subject: null Sender: default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/PublishEvent Event:[[ <business-event xmlns:ns="http://schemas.oracle.com/events/edl/MyEventDefn" xmlns="http://oracle.com/fabric/businessEvent"> <name>ns:MyEvent</name> <id>a7ae9d28-9530-4049-8385-e0ebfb0eea50</id> <tracking> <ecid>5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83</ecid> <compositeInstanceId>90087</compositeInstanceId> <parentComponentInstanceId>mediator:8E0411C054B511E0AF455DABE1395E7B</paren tComponentInstanceId> </tracking> <content> <inp1:singleString xmlns:inp1="http://xmlns.oracle.com/singleString">PO123</inp1:singleString> </content> </business-event> ]]
The following sample shows a portion of a server log file containing information about an event dequeued by EDN with a timestamp of 2011-03-22T11:52:37.038-07:00
.
[2011-03-22T11:52:37.038-07:00] [soa_server1] [TRACE] [SOA-31010] [oracle.integration.platform.blocks.event.saq] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@96bab0] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1:100000140] [SRC_CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [APP: soa-infra] [composite_name: MediatorPubSub] [component_name: PublishEvent] [component_instance_id: 8E0411C054B511E0AF455DABE1395E7B] [SRC_METHOD: fineDequeuedEvent] [composite_instance_id: 90087] Dequeued event, Subject: null [source type "J"]:[[ <business-event xmlns:ns="http://schemas.oracle.com/events/edl/MyEventDefn" xmlns="http://oracle.com/fabric/businessEvent"> <name>ns:MyEvent</name> <id>a7ae9d28-9530-4049-8385-e0ebfb0eea50</id> <source>default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/PublishEvent</source> <tracking> <ecid>5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83</ecid> <compositeInstanceId>90087</compositeInstanceId> <parentComponentInstanceId>mediator:8E0411C054B511E0AF455DABE1395E7B </parentComponentInstanceId> </tracking> <content> <inp1:singleString xmlns:inp1="http://xmlns.oracle.com/singleString">PO123</inp1:singleString> </content> </business-event> ]]
The following sample shows a portion of a server log file containing information of an event sent to a subscriber by EDN with a timestamp of 2011-03-22T11:52:37.150-07:00
.
[2011-03-22T11:52:37.150-07:00] [soa_server1] [TRACE] [SOA-31033] [oracle.integration.platform.blocks.event.saq] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@b402db] [userId: <anonymous>] [ecid: 5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83,0:1:100000141] [SRC_CLASS: oracle.integration.platform.blocks.event.saq.SAQBusinessEventBusMessages] [APP: soa-infra] [SRC_METHOD: fineSentOAOOEvent] Sent OAOO event [QName: "{http://schemas.oracle.com/events/edl/MyEventDefn}MyEvent" to target: "default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/SubsEvent"]:[[ <business-event xmlns:ns="http://schemas.oracle.com/events/edl/MyEventDefn" xmlns="http://oracle.com/fabric/businessEvent"> <name>ns:MyEvent</name> <id>a7ae9d28-9530-4049-8385-e0ebfb0eea50</id> <source>default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/default/MediatorPubSub!2.0*soa_ 58454bff-3b36-4e29-a883-949085d85719/PublishEvent</source> <tracking> <ecid>5fc0ca821d51e919:3d214296:12ebaa2c996:-8000-00000000000c2b83</ecid> <compositeInstanceId>90087</compositeInstanceId> <parentComponentInstanceId>mediator:8E0411C054B511E0AF455DABE1395E7B</paren tComponentInstanceId> </tracking> <content> <inp1:singleString xmlns:inp1="http://xmlns.oracle.com/singleString">PO123</inp1:singleString> </content> </business-event> ]]
If the delay comes from the event publishing side, then check the following:
If you use the remote EDN API to publish events from, for example, an ADF component to the SOA server, and you use the following event connection factory lookup code:
BusinessEventConnectionFactory BusinessEventConnectionFactorySupport. findRelevantBusinessEventConnectionFactory(boolean forceRetry)
You may need to set the forceRetry
parameter to true
to cache the BusinessEventConnectionFactory
object after it is found, and avoid forced retries of JNDI lookups that can cause a significant delay.
If you publish multiple events with a loop statement in a batch fashion, you may need to move the above event connection factory lookup outside the loop to avoid a lookup per event.
If the delay comes from event dequeuing after the event is published, then check the following:
Ensure That... | Description |
---|---|
The SOA server is up running, and EDN has been configured with the correct amount of dequeuer threads through an |
To examine the configuration of the
Set the number of EDN dequeuer threads (the default is |
The backing store of EDN is functioning properly. |
For example, if the backing store of EDN is Oracle AQ, and there are a significant amount of dead events stuck in the AQ that are not cleared, then event dequeuing by EDN may slow down significantly. |
If the delay comes from event consumption by its subscriber after the event is dequeued, then check the following:
Ensure that all the event subscribing components such as Oracle Mediator or a BPEL process can consume and process the event within the expected time frame. If not, investigate why there is a delay.
If the event delivery is configured with an OAOO delivery guarantee, then EDN delivers the event to its subscriber in a global transaction context with built-in retry logics, should the transaction fail to commit. Retries may delay overall event processing. If the transaction fails to commit, ensure that you research the root cause of transaction failure.
Problem
Events are consumed by multiple revisions of the same composites.
Solution
By design, different composites can subscribe to the same event. However, it is an error if multiple revisions of the same composite subscribe to the same event. This occurs when you have multiple active versions of the composite that may be a result of a patch failure.
Perform the following steps to identify whether there are multiple, active revisions of the same composite and to retire the composite revision that should not be active:
Go to Fusion Applications Control.
Click SOA > soa-infra.
In the Deployed Composites section of the Dashboard tab, you see a list of deployed composite names, revisions, and modes (for example, active).
Identify composites with the same name and with an active mode, but with different revisions.
Click the composite revision that should not be active.
Click the Retire button.
Problem
If the Java debug level is set to TRACE:16 (FINER) or a lower value, you may see log messages such as the following:
Began XA for OAOO Rolled back XA for OAOO
Solution
These are normal messages of OAOO event delivery when there are no events waiting to be delivered. These are not error conditions. You can turn off these messages by setting the Java logging level to TRACE:1 (FINE) or a higher value.
Problem
An application transaction is not completing. For example, a purchase order status may remain processing. Checking the composite instance shows that the composite is stuck in the running state. In this case, the component is probably not running. Instead, it has likely faulted and may need recovery.
Faults may occur for various reasons:
A BPEL activity faulted with an error (for example, a business error, security authorization error, or some other error).
A BPEL activity invoked an external web service that was unavailable.
A BPEL activity has already been terminated by the administrator using Fusion Applications Control.
A BPEL activity invoked an asynchronous ADF service and the message is stuck in the AQ/JMS queue.
A BPEL activity invoked an asynchronous ADF service, but because SOA was unavailable, the callback message did not arrive.
A BPEL activity invoked a synchronous ADF service, which is taking a long time (or is hanging).
A network error occurred.
Solution
Log in to Fusion Applications Control.
In the navigator pane, go to domain_name > SOA.
Click soa-infra (SOA_cluster_name).
Click the Instances tab.
Search for the composite instance, and click the instance ID.
The Flow Trace page appears.
If the instance is not visible (and the Audit Level is not set to Off in the SOA Infrastructure Common Properties page), this implies that the message is stuck outside of SOA. Check the ADF log (if invoked through an ADF service) to find out if it is stuck in the AQ/JMS queue.
If the message has reached Oracle Mediator, but not instantiated the BPEL flow, the BPEL instance may have been rolled back from the start due to an error.
If the BPEL flow exists, the Faults section of the Flow Trace page typically shows the faulted service that can trace the root cause.
In the Trace section, click the BPEL process.
Expand the BPEL audit trail to see the exact point at which the service faulted.
This information is also available in the Faults tab of the BPEL flow trace. It also indicates whether the fault can be recovered.
Click the View Raw XML link.
The same information is also available through this link, where you can see the error. For example:
... <message>Faulted while invoking operation "modifyUserRoles" on provider "UserService". </message><details> ... <tns:message>JBO-27023: Failed to validate all rows in a transaction.</tns:message> <tns:severity>SEVERITY_ERROR</tns:severity> ... <message>The transaction was rolled back. The work performed for bpel instance "451042" was rolled back to the previous dehydration point, but the audit trail has been saved. You can recover the instance from the recovery console by resubmitting the callback message or activity for execution.</message
Since the instance was rolled back to its previous dehydration point, the status remains as Running.
In the Audit Trail and Faults tabs, make a note of the following:
Composite name (for example, HcmUsersSpmlComposite)
Component (for example, UpdateGuid BPEL process)
BPEL instance ID (for example, bpel:451042)
This is all used in the recovery of the instance, if it is recoverable. Note that the audit trail window may mark the error as a nonrecoverable business fault, but the recoverability of the message can be found in the Recovery tab of the BPEL process service engine.
To attempt to recover the instance, right-click soa-infra (SOA_cluster_name), and choose Service Engines > BPEL.
Click the Recovery tab.
From the Type list, select Activity.
Specify the composite and component names captured in step 9, and click Search.
Find the specific BPEL instance ID. You can recover faults marked as Recoverable.
Check the other recovery options in the Type list (for example, Invoke and Callback), if they exist.
Note:
You can also search for recoverable messages from the Faults and Rejected Messages tab of the SOA Infrastructure by clicking the message and selecting the appropriate action from the Recovery Actions list.
If the instance is not marked as recoverable, then reinvoking the service is not allowed (most probably because it is not idempotent). In some cases, you may need to provide diagnostic information to Oracle Support Services to resolve issues with nonrecoverable, nonidempotent transactions.
If the BPEL activity has invoked an asynchronous ADF service and the message is stuck in the AQ JMS queue, you can view the server
.log
and server
-diagnostic.log
files to see the logging of the message metadata logged by the JRF web services infrastructure.
In addition, the ADF diagnostic logs are also available to debug, if needed.
Use the ECID field to correlate and track ADF service logging corresponding with the SOA composite that invoked it.
When viewing the log in Fusion Applications Control, click the Broaden Target Scope dropdown list and select the farm_name/domain_name (Oracle WebLogic Domain) to view messages across the domain.
In the Log Messages page for the Oracle WebLogic Server domain, in the Selected Targets section, ensure that the search includes the ECID field with the value noted in step a and the Component Name field is set to adf-bc.
Search and view log records for the Oracle Application Development Framework (Oracle ADF) Business Component (BC) and the ECID and note any issues.
For a specific ECID, you find several root instances (top level clients). You must drill down to the appropriate instance to find a specific fault.
Observe if the Oracle ADF Business Components completed successfully or completed with an error. See the "Viewing and Searching Log Files" section in the Oracle Fusion Middleware Administrator's Guide.
For more details on diagnosing Oracle ADF-BC asynchronous web service calls, see Section 11.3.7.
See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite for the various ways to recover transactions and messages:
"Managing SOA Composite Application Instances" chapter to recover from the SOA composite application page in Fusion Applications Control.
"Managing BPEL Process Service Components and Engines" chapter to recover from the BPEL process service component and BPEL process service engine message recovery pages.
Problem
A BPEL activity has invoked an asynchronous ADF service and not received a response. The message may be stuck in the request or response queues.
Solution
Check if the reply address is valid, the server is running, and the name of the server and port match with the server on which the BPEL process is running.
Verify that the policy on the callback receive activity matches with the policy advertised in the WSDL for the response port type/binding.
Look at the server-name
_diagnostic.log
file and check for all status messages of the request.
Go to Fusion Applications Control.
Right-click soa-infra.
Choose Logs > View Log Messages.
Click Target Log Files.
Select the server diagnostic file to view, and click View Log File.
For more information about the different types of loggers to set, see the "Setting Logging Levels for Troubleshooting" section of Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Note that every log contains the message ID and other details, such as ECID, that can help isolate it from other messages. After you find the message ID (that is, MessageID: urn:uuid:ac1a4a81-39df-45b1-a741-e16e752d5d33
), search for the same ID in the log file. You should see messages such as the following for each successful asynchronous operation.
If the following log is not available, this means the asynchronous operation was never called on this server:
[2010-12-17T12:27:13.537-08:00] [AdminServer] [NOTIFICATION] [] [oracle.j2ee.ws.common.jaxws.JAXWSMessages] [tid: [ACTIVE].ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 0000Inq0wfa76EWLHyo2yf1D2wUW000000,0:1] [WEBSERVICE_PORT.name: AsyncEjbPort] [APP: AsyncEjb] [J2EE_MODULE.name: AsyncEjb-ejb] [WEBSERVICE.name: AsyncEjbService] [J2EE_APP.name: AsyncEjb] *[MessageID: urn:uuid:ac1a4a81-39df-45b1-a741-e16e752d5d33] Sending message to JMS queue* "oracle.j2ee.ws.server.async.NonDefaultRequestQueue" for async processing of service "1d7551aa-73d9-4624-ad52-c15e35b5b25dRequest"
If the following log is not available, this means the asynchronous operation was never called on this server:
[2010-12-17T12:27:13.783-08:00] [AdminServer] [NOTIFICATION] [] [oracle.j2ee.ws.common.jaxws.JAXWSMessages] [tid: [ACTIVE].ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 0000Inq0wfa76EWLHyo2yf1D2wUW000000,0:1] [WEBSERVICE_PORT.name: AsyncEjbPort] [APP: AsyncEjb] [J2EE_MODULE.name: AsyncEjb-ejb] [WEBSERVICE.name: AsyncEjbService] [J2EE_APP.name: AsyncEjb] *[MessageID: urn:uuid:ac1a4a81-39df-45b1-a741-e16e752d5d33] An asynchronous request message is received* and successfully recorded for service "AsyncEjbService" with a replyTo address as "http://adc2180314:7001/AsyncEjbCallback/AsyncEjbResponseImplService" [2010-12-17T12:27:13.986-08:00] [AdminServer] [NOTIFICATION] [] [oracle.j2ee.ws.common.jaxws.JAXWSMessages] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid: 0000Inq0wfa76EWLHyo2yf1D2wUW000000,0] [APP: AsyncEjb] *[MessageID: urn:uuid:ac1a4a81-39df-45b1-a741-e16e752d5d33] Startedasynchronous request processing for the service* "AsyncEjbService" with the message selector "1d7551aa-73d9-4624-ad52-c15e35b5b25dRequest". Transaction enabled: "true" [2010-12-17T12:27:14.296-08:00] [AdminServer] [NOTIFICATION] [] [oracle.j2ee.ws.common.jaxws.JAXWSMessages] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid: 0000Inq0wfa76EWLHyo2yf1D2wUW000000,0] [APP: AsyncEjb] *[MessageID: urn:uuid:ac1a4a81-39df-45b1-a741-e16e752d5d33] Completed asynchronous request processing. A response will be sent to the client.* [2010-12-17T12:27:14.312-08:00] [AdminServer] [NOTIFICATION] [] [oracle.j2ee.ws.common.jaxws.JAXWSMessages] [tid: [ACTIVE].ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid: 0000Inq0wfa76EWLHyo2yf1D2wUW000000,0] [APP: AsyncEjb] *[MessageID: urn:uuid:ac1a4a81-39df-45b1-a741-e16e752d5d33] Startedasynchronous response processing for the service* "AsyncEjbService" with the message selector "1d7551aa-73d9-4624-ad52-c15e35b5b25dResponse". [2010-12-17T12:27:15.018-08:00] [AdminServer] [NOTIFICATION] [] [oracle.j2ee.ws.common.jaxws.JAXWSMessages] [tid: [ACTIVE].ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid: 77e271e3-afd5-48e8-a83d-9b326860f3f2-0000000000000018,0] [APP: AsyncEjb] *[MessageID: urn:uuid:ac1a4a81-39df-45b1-a741-e16e752d5d33] Completed asynchronous response processing successfully. Client must have received the response by now.*
To monitor the traffic to the asynchronous ADF web services, monitor the Oracle AQ queues that are used for each family. The queue names for each family are shown in the following table:
Family AQ | Name |
---|---|
Oracle Fusion Customer Relationship Management Product |
|
Oracle Fusion Human Capital Management Product |
|
Oracle Fusion Financials Product |
|
Oracle Fusion Procurement Product |
|
Oracle Fusion Project Product |
|
Oracle Fusion Supply Chain Management Product |
|
Oracle Fusion Setup Product |
|
IC |
|
Problem
Business events are raised from Fusion J2EE applications and are picked up more than once. Expected behavior is that they should be picked up only once.
Solution
If a patch fails, it may be possible for two versions of a given composite to be active (that is, the older version has not been retired). When multiple versions of a composite are active, they all become subscribers and the event is picked up more than once. This should not happen under normal scenarios. Reapply the patch and ensure that the deployed composite only has one active version.
Problem
Long running synchronous calls to remote web services end with JTA transaction rolled-back errors. When executing a transaction making an asynchronous call (for example, to the SOA server), the application returns with an error. Server logs show JTA transaction timeouts, which can cause this behavior.
Solution
Check the JTA transaction timeout in Oracle WebLogic Administration Console.
Log in to Oracle WebLogic Server Administration Console.
In the Domain Structure, select Services > JTA to check the timeout value.
If the transaction is always timing out beyond 30 seconds and is a custom composite synchronous client invocation, then you may need to revisit the design approach. It may be best for the external web service to be invoked as an asynchronous transaction.
Increasing the JTA for supporting long running synchronous transactions is simply an interim mechanism.
For information on changing the JTA transaction timeout setting, see the "Resolving Connection Timeouts" section of Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
If synchronous client invocations to Oracle Fusion Applications code take a long time, check for any performance issues with the system and try to resolve them.
Check for appropriate values for the syncMaxWaitTime property and BPEL's EJB transaction timeout settings in relation to the JTA timeout settings and only then increase the value of the JTA timeout, if needed.
For information on viewing and changing the syncMaxWaitTime property, see the "Configuring BPEL Process Service Engine Properties" section of Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
To view and change the BPEL EJB transaction timeout settings, perform the following steps:
Log in to Oracle WebLogic Server Administration Console.
In the Domain Structure, click Deployments.
Expand soa-infra > EJBs.
Update the following EJBs:
- BPELActivityManagerBean
- BPELDeliveryBean
- BPELDispatcherBean
- BPELEngineBean
- BPELFinderBean
- BPELInstanceManagerBean
- BPELProcessManagerBean
- BPELSensorValuesBean
- BPELServerManagerBean
Click Save.
Restart Oracle WebLogic Server.
For asynchronous transactions, check the values for both the BPEL EJB transaction timeout and the JTA transaction timeout and adjust as needed.
Problem
Messages primarily get lost for the following reasons:
The EDN message was not delivered.
The target asynchronous service did not respond.
The message was roll backed (though it was not lost; you still see it in Fusion Applications Control).
Solution
This issue typically requires you to identify where the messages are supposed to be and to diagnose the path.
If the calling component is an ADF web service invoking a SOA composite, then check for any client errors in the ADF logs.
Because ADF invokes SOA composites using EDN, check the JMS/AQ and XA data source configurations to ensure that events are being fired. For more information about EDN issues, see Section 11.3.2.
Check if the Oracle WSM Policy Manager security configurations allow the client to invoke the SOA composite.
Verify that the transaction is not transient (nonpersistent).
Oracle BPEL Process Manager uses the dehydration store database to maintain long-running, asynchronous processes and their current state information in a database while they wait for asynchronous callbacks. Storing the process in a database preserves the process and prevents any loss of state or reliability if a system shuts down or a network problem occurs. There are two types of processes in Oracle BPEL Process Manager. These processes impact the dehydration store database in different ways.
Transient processes: This process type does not incur any intermediate dehydration points during process execution. If there are unhandled faults or there is system downtime during process execution, the instances of a transient process do not leave a trace in the system. Instances of transient processes cannot be saved in-flight (whether they complete normally or abnormally). Transient processes are typically short-lived, request-response style processes. The synchronous process you design in JDeveloper is an example of a transient process.
Durable processes: This process type incurs one or more dehydration points in the database during execution because of the following activities:
Receive activity
OnMessage branch of a pick activity
OnAlarm branch of a pick activity
Wait activity
Instances of durable processes can be saved in-flight (whether they complete normally or abnormally). These processes are typically long-living and initiated through a one-way invocation. Because of out-of-memory and system downtime issues, durable processes cannot be memory-optimized.
If a composite instance is not visible in Fusion Applications Control and the SOA Infrastructure is running, check that the Audit Level is not set to Off in the SOA Infrastructure Common Properties page. This can be checked in Fusion Applications Control:
Right-click soa-infra (SOA_cluster_name).
Choose SOA Administration > Common Properties.
Check the ADF logs to see if there was a bad/malformed message or incorrect SOAP headers (if invoking from an external web service). The transaction may have rolled back before being dehydrated due to an internal error. Again, the SOA logs can identify the cause of this issue.
If the composite instance is available, check the bpel.config.oneWayDeliveryPolicy BPEL property value. You can check the value in Fusion Applications Control:
In the navigation pane, expand soa-infra (SOA_cluster_name).
Expand the partition, and select the composite.
The Dashboard page for the composite is displayed.
In the upper right corner, click the Show XML Definition icon.
The contents of composite.xml
for that composite are displayed.
If this is set to async.cache, you may lose messages. Set it to async.persist for reliable messages. This is typically specified in the BPEL process service component section of the composite.xml
file, so this can be set for custom composites. If the value is not set in composite.xml
, the value for oneWayDeliveryPolicy in the System MBean Browser in Fusion Applications Control is used. The following values are possible:
async.persist: Messages are persisted in the database hash map.
async.cache: Messages are stored in memory.
sync: Direct invocation occurs on the same thread.
For more information about these settings, see the "Deployment Descriptor Properties" appendix and the "Transaction and Fault Propagation Semantics in BPEL Processes" chapter in the Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite.
For information about setting the oneWayDeliveryPolicy property in the System MBean Browser, see the "Configuring BPEL Process Service Engine Properties" section of Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
It is also possible that the transaction has invoked a target asynchronous service that has not responded back. In this case, the composite instance flow shows the call to the target asynchronous service.
Check if the instance has rolled back and the message is in recovery.
Log in to Fusion Applications Control.
Right-click soa-infra (SOA_cluster_name), and choose Service Engines > BPEL.
Click the Recovery tab.
This may occur if any external references receiving the message are not reachable (for example, an external web service, enterprise applications such as Siebel, and so on).
Problem
When a BPEL process flow errors out, it is retried with all its invocations. This is undesirable in some cases.
Solution
The property GlobalTxMaxRetry (default value is 3
) specifies how many retries are performed if an error is identified as a retriable one. For example, after several web service invocations, if dehydration fails due to a data source error, then this is identified as a retriable error and all activities from the prior dehydration state are retried. If the activities being retried are not idempotent (that is, their state can change with each retry and is not guaranteed to give the same behavior), then multiple retries can be problematic.
To rectify this, customize the composite by specifically marking the nonidempotent activities with idempotentset
set to false
in the partner link settings section of the composite.xml
file to prevent retries.
<property name="bpel.partnerLink.partner_link_name.idempotent">false</property>
You cannot set the idempotentset
property in Fusion Applications Control.
For more information about Oracle SOA Suite customizations in JDeveloper, see the "Customizing and Extending SOA Components" chapter in the Oracle Fusion Applications Extensibility Guide.
You can also set GlobalTxMaxRetry to 0
in the Systems MBean Browser.
Right-click soa-infra (SOA_cluster_name).
Choose SOA Administration > Common Properties.
Click More SOA Infra Advanced Configuration Properties.
Click GlobalTxMaxRetry.
In the Value field, enter an appropriate value.
Click Apply.
For information about the idempotent
property, see the "Tuning BPEL Properties Set Inside a Composite" section of Oracle Fusion Middleware Performance and Tuning Guide.
Problem
Fusion Applications Control is missing features needed for diagnostics and recovery. For example, the Recovery tab for the BPEL process service engine shows a BPEL instance as recoverable along with instructions to click the Recover button. However, there is no Recover button available. In addition, the ability to export a composite is not available.
Solution
Check that you are logged in as a user with administrator privileges and are not logged in as a read-only user. Read-only users do not see administrative features such as the Recover button or the ability to export composites.
See the "Oracle Enterprise Manager Roles" appendix of Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Problem
BPEL processes have an automatic recovery feature that attempts to automatically recover activities that are recoverable such as unresolved invoke/callback messages, activities not completed over a provided threshold time, and so on. However, the automatic recovery feature only tries to recover a few instances and only retries a fixed number of times. If some instances are not being automatically recovered, they are likely not being picked up because of the configuration of the automatic recovery parameters.
Solution
Set the maximum number of messages to automatically recover.
By default, the automatic recovery feature of Oracle BPEL Process Manager processes fifty messages to submit for each recovery attempt. This is controlled by the maxMessageRaiseSize property.
In the navigation pane, right-click soa-infra (SOA_cluster_name).
Choose SOA Administration > BPEL Properties > More BPEL Configuration Properties > RecoveryConfig.
Expand both RecurringScheduleConfig > maxMessageRaiseSize and StartupScheduleConfig > maxMessageRaiseSize.
The default value is 50
for each. A negative value causes all messages selected from the database to be submitted for recovery. A value of 0
causes no messages to be selected from the database (effectively disabling recovery). To recover more than fifty messages, set the property value to that value. Use this property to limit the impact of recovery on the server.
Set the maximum number of automatic recovery attempts on a given message.
You can also configure the number of automatic recovery attempts to submit in the same recoverable instance. The value you provide specifies the maximum number of times that invoke and callback messages are recovered. If the value is 0
(the default value), it recovers all messages. After the number of recovery attempts on a message exceeds the specified value, a message is marked as nonrecoverable.
Follow these steps to configure automatic recovery attempts for invoke and callback messages in Fusion Applications Control.
In the navigation pane, right-click soa-infra (SOA_cluster_name).
Choose SOA Administration > BPEL Properties > More BPEL Configuration Properties.
Select MaxRecoverAttempt, and enter a value in the Value field.
Click Apply.
It may not be desirable in all cases to use automatic recovery. If services are not idempotent, then corruption can occur. Moreover, the automatic recovery restores the composite to the last save point that can be immediately after an asynchronous invoke, wait, and so on. Therefore, it is important to understand the process behavior and what it does next before performing mass recoveries. Attempt mass automatic recovery only after the root cause of the composite failures has been fixed (for example, a service that was unavailable is now available, a database running out of space was fixed, and so on). Automatic recovery can also trigger an unexpected load during failure scenarios. This causes more threads to block on a remote server that can induce hangs in the SOA server in a cascading fashion.
For more information, see Section 11.3.11 and the "Configuring Automatic Recovery for Oracle BPEL Process Manager" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Problem
The following error is displayed if either the endpoint is not available for a reference or a composite is deployed with an incorrect deployment plan file.
oracle.fabric.common.FabricException: Cannot read WSDL
Solution
Ensure that the endpoint for the reference is up and running.
The WSDL or endpoint is stated in the error. The service can then be looked up from Fusion Applications Control to check if it is active. After the service is active and the endpoint is reachable, search for the SOA instance and retry through the Recovery tab for the BPEL process service engine in Fusion Applications Control.
For extensions/customizations, ensure that the correct URL is updated in the deployment configuration plan.
For information about deployment plans, including examples of using sca_extractPlan
to extract plans, see the "Customizing Your Application for the Target Environment Before Deployment" section of Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite and the "Customizing SOA Composite Applications with JDeveloper" section of Oracle Fusion Applications Extensibility Guide.
Problem
Oracle Support Services has requested you to upload your composite to review the flow and associated configuration files.
Solution
Log in to Fusion Applications Control as a user with administrator privileges, and not as a read-only user.
In the navigation pane, expand soa-infra (SOA_cluster_name).
Expand the partition in which the composite is located (for example, default).
Right-click the composite and choose Export.
Accept the default selections, and save the resulting JAR file.
Provide the JAR file to Oracle Support Services so that it can be imported for review in JDeveloper by selecting File > Import.
Problem
Oracle Support Services has requested you to confirm configuration properties of various SOA components.
Solution
In the navigation pane, right-click soa-infra (SOA_cluster_name).
Choose Administration > MDS Configuration > Export.
Upload the resulting file to Oracle Support Services for review.
For information about troubleshooting Oracle WSM Policy Manager, see the "Diagnosing Problems" chapter in the Oracle Fusion Middleware Security and Administrator's Guide for Web Services.
This section contains the following topics that describe common problems and solutions for human workflow:
Section 11.5.1, "Recovering Human Task Instances Stuck in the Alerted State"
Section 11.5.2, "Worklist: Notifications and Approvals Region Contains No Data"
Section 11.5.6, "Task History: Cannot Remove or Move a Participant After Yourself"
Section 11.5.8, "HCM Services are Timing Out or Getting Stuck"
Section 11.5.9, "Error/Warning Reported in Retrieving the Position for a User"
Section 11.5.10, "Users See No Approvals in the Inbox when Using Position Hierarchy"
Section 11.5.11, "Task Escalation Does Not Follow Job Level, Supervisory, or Position Hierarchies"
Section 11.5.12, "User Is Not Allowed to Perform an Action on a Human Workflow Task"
Section 11.5.15, "Actionable Notifications Are Sent But No Action is Taken After Responding"
Section 11.5.17, "Only a Subset of Users in LDAP Can Log In to Oracle BPM Worklist"
Section 11.5.19, "Add Participant Button in Oracle BPM Worklist Is Disabled"
Section 11.5.20, "Task Modifications Made in a Runtime Tool Are Not Appearing for the Task"
Section 11.5.21, "How Can I Set Commonly Used Human Workflow Configuration Parameters?"
Section 11.5.26, "How to Test the Health of the Installed Server"
Section 11.5.28, "Error During Import of Task Payload Mapped Attribute Mappings"
Problem
If a human task instance fails due to a business error, the task status stays in the alerted state.
Solution
This can happen if, for example, the following occurs:
The assigned user does not exist.
The manager is not defined.
A rule does not return assignees.
In these cases, the task is set to alerted and is assigned to the process owner or the error assignee (if defined in the task metadata).
The administrator or process owner must go to Oracle BPM Worklist and reassign these tasks to the correct users for the workflow to continue forward.
Problem
If the Worklist: Notifications and Approvals region in the Welcome dashboard on the Oracle Fusion Applications home page does not contain data, it is likely the connection from Oracle BPM Worklist to the SOA Managed Server is timing out.
Solution
Ensure the SOA Managed Servers are running in the CommonDomain
domain with Fusion Applications Control:
From the navigation pane, expand the farm, WebLogic Domain, CommonDomain, FS_SOACluster.
Click FS_SOACluster to display the Oracle WebLogic Server Cluster Home page.
In the Servers section, view the Status column of the SOA Managed Server, for example, soa_server1.
If the Managed Server is not running, restart it. See the "Starting the Administration Servers and Managed Servers" section in the Oracle Fusion Applications Administrator's Guide.
Locate the HomePageServer_
n
-Diagnostic.log
diagnostic log files on the CommonDomain
domain. This log file is typically stored in the following locations:
(UNIX) DOMAIN_HOME/servers/server_name/logs/HomePageServer_n-Diagnostic.log (Windows) DOMAIN_HOME\servers\server_name\logs\HomePageServer_n-Diagnostic.log
These log files may be stored in a different location if you reconfigured the log file.
Review the HomePageServer_
n
-Diagnostic.log
diagnostic log for the following exception:
[HomePageServer_1] [ERROR] [] [oracle.soa.services.workflow.worklist] [tid: [ACTIVE].ExecuteThread: '24' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: username] [ecid: ecidvalue] [APP: HomePageApp#V2.0] [URI: /homePage/faces/AtkHomePageWelcome] TimeoutException occurred for server='null'.[[ oracle.bpel.services.common.concurrent.TimeoutException: Execution timeout service : FederetedWorklistService
Modify default timeout interval in the profile:
Log in to the Setup UI application.
From the Administration menu in the work area, choose Setup and Maintenance.
The Overview page displays.
Click the All Tasks tab.
In the Name field, enter Profiles
and click Search.
In the Search Results table, in the Manage Administrator Profile Values table, click the Go to Task.
From the Manage Administrator Profile Values page, in the Profile Display Name field, enter Welcome Dashboard Worklist Timeout Interval
, and then click Search.
In the Search Results table, select ATK_HOME_PAGE_WORKLIST_TIMEOUT.
In the ATK_HOME_PAGE_WORKLIST_TIMEOUT: Profile Values table, modify the Profile Value field.
Increase the default timeout of 5 seconds to a larger value such as 10 seconds. A larger timeout value increases the amount of time that home page server waits for response from background SOA web services for all users and it might adversely impact home page performance when there are many concurrent users. Increasing this value should be a temporary change until you can find the root cause for the delay in SOA federated worklist response time.
Click Save.
Run a session, and check that the Worklist: Notifications and Approvals region in the Welcome dashboard on the Oracle Fusion Applications home page contain data.
Problem
When you click the human task in Oracle BPM Worklist, the following message is displayed:
Task detail is not available
Solution
This problem typically occurs during the deployment of the ADF J2EE application containing the task details page definition. As a result, the URI for the details page is not registered with the SOA cluster.
To confirm whether the task details page was registered in Fusion Applications Control:
Log in to the domain where the composite with the subscription is deployed.
In the navigation pane, right-click SOA > soa-infra (SOA_cluster_name).
Choose Service Engines > Human Workflow.
In the Components section, click the appropriate task.
Click the Administration tab to see a URI for the task details page.
If there is no URI displayed, then check the log files for any errors during deployment.
A possible problem may be incorrect entries in the wf_client_config.xml
file. The wf_client_config.xml
file resides in the exploded EAR file's APP-INF/classes
directory. For example:
APPLICATIONS_BASE/fusionapps/applications/fin/deploy/EarFinPayables.ear/
APP-INF/classes/wf_client_config.xml
The file contains the cluster URL information for the local SOA runtime of that particular Oracle WebLogic Server domain. If the cluster name is wrong or that cluster does not exist in the domain, then a problem exists with the deployment and configuration of the environment.
As a workaround, you can directly enter the task details page URI in Fusion Applications Control. See the "Managing the URI of the Human Task Service Component Task Details Application" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite. For the values, enter the following:
Field | Value |
---|---|
Application Name |
|
Host Name |
Your server host name for SOA |
HTTP Port |
Your HTTP server port used for SOA if SSL is disabled. |
HTTPS Port |
Your HTTPS server port used for SOA if SSL is enabled. |
URI |
For example, for financials: /payables/faces/adf.task-flow?_id=FinApInvoiceApprovalTaskFlow&_ document=WEB-INF/oracle/apps/financials/payables/invoices/transact ions/ui/invoiceApprovalTask/flow/FinApInvoiceApprovalTaskFlow.xml |
Problem
At runtime, when a human workflow task tries to fetch the list of users, it errors with the following message:
Ruleset returned lists with different list builder
This can be seen in the Task Detail Comments field. Alternately, it may select a user or approver, which may not seem to be the correct or expected one. This is primarily caused by having overlapping rules with different list builders. When participants of a task are specified using business rules, it is expected that business rules return list builders of the same type.
Moreover, only one rule from a ruleset should be applicable for a transaction. In case a number of rules are true, the actions associated with the applicable rule with the highest priority get executed. If multiple applicable rules have the same priority, then the first rule in the list is picked and its actions are executed.
Solution
To resolve this issue, avoid writing overlapping rules. Constraints from different list builders are different and cannot be mixed. If more than one rule gets triggered with a different list builder, this error occurs. Moreover, only one set of constraints is honored.
Check that all rules in the ruleset have priorities defined so that multiple rules with the same priority are not applicable for the same transaction. For more details, see the "Using Approval Management" chapter in the Oracle Fusion Middleware Modeling and Implementation Guide for Oracle Business Process Management.
Problem
The document is stuck pending approval and the logs show the following error:
"Unable to walk hierarchy for user username'
This occurs when the hierarchy for the given user has not been set up correctly. For example, assume you have approval rules set up so that the rule getting fired uses the supervisory hierarchy and you expect it to route to USER01 > USER02 > USER03 > . . . > USER10 (ten level hierarchy). If the supervisory/management hierarchy of USER01 has not been set up, then the approval task throws an error stating the inability to go through the hierarchy for USER01.
Solution
Since the user level hierarchy does not exist for the user_name
, verify the Human Capital Management (HCM) setup by manually running the HCM service to which workflow-identity-config.xml
is pointing. If it does not return any result, then configure the correct hierarchy for the user in HCM.
Run the HCM service, select the fetchManager
(s) API (for supervisory), and provide the following payload, where <ns2:Id>DD7A1614BBFAA0F0A4511ACD96D2C88D</ns2:Id>
is the GUID of the user whose manager you are trying to find. The supervisory hierarchy and position hierarchy services can be found under HcmCore
. The name is HierarchyProviderService
. The corresponding WSDLs are as follows:
Supervisory hierarchy:
http://host:port/hcmEmploymentCoreApprovalHierarchy/HierarchyProviderService?wsdl
Position hierarchy:
http://host:port/hcmTreesModel/HierarchyProviderService?wsdl
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Header> <wsse:Security soap:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-sec ext-1.0.xsd"> <wsse:UsernameToken> <wsse:Username>FUSION_APPS_AMX_APPID</wsse:Username> <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profi le-1.0#PasswordText">Welcome1</wsse:Password> </wsse:UsernameToken> </wsse:Security> </soap:Header> <soap:Body xmlns:ns1="http://xmlns.oracle.com/apps/hcm/employment/core/approvalHierarchyServi ce/types/"> <ns1:fetchManagers> <ns1:principal xmlns:ns2="http://xmlns.oracle.com/apps/hcm/employment/core/approvalHierarchyServi ce/"> <ns2:Id>DD7A1614BBFAA0F0A4511ACD96D2C88D</ns2:Id> </ns1:principal> <ns1:level>2</ns1:level> </ns1:fetchManagers> </soap:Body> </soap:Envelope>
The following example shows the workflow-identity-config.xml
file used in a development environment:
<?xml version='1.0' encoding='UTF-8'?> <ISConfiguration xmlns="http://www.oracle.com/pcbpel/identityservice/isconfig"> <configurations> <configuration realmName="myrealm"> <provider name="JpsProvider" providerType="JPS" service="Identity"> <property name="jpsContextName" value="default"/> <property name="IdentityServiceExtension" value="HCMIdentityServiceExtension"/> </provider> </configuration> </configurations> <property name="caseSensitive" value="false"/> <property name="caseSensitiveGroups" value="true"/> <serviceExtensions> <serviceExtension name="HCMIdentityServiceExtension"> <serviceProvider classname="oracle.bpel.services.identity.hierarchy.providers.hcm.HCMHierarchyProvi der" type="supervisoryHierarchyProvider"> <initializationParameter name="wsdlUrl" value="http://example.com:20619/hcmEmploymentCoreApprovalHierarchy/Hie rarchyProviderService?WSDL"/> <initializationParameter name="csf-key-name" value="FUSION_APPS_AMX_APPID-KEY"/> <initializationParameter name="http-read-timeout" value="30000"/> <initializationParameter name="securityPolicyName" value="oracle/wss_username_ token_client_policy"/> </serviceProvider> <serviceProvider classname="oracle.bpel.services.identity.hierarchy.providers.hcm.HCMHierarchyProvi der" type="jobLevelHierarchyProvider"> <initializationParameter name="wsdlUrl" value="http://example.com:20619/hcmEmploymentCoreApprovalHierarchy/Hie rarchyProviderService?WSDL"/> <initializationParameter name="csf-key-name" value="FUSION_APPS_AMX_APPID-KEY"/> <initializationParameter name="http-read-timeout" value="30000"/> <initializationParameter name="securityPolicyName" value="oracle/wss_username_ token_client_policy"/> </serviceProvider> <serviceProvider classname="oracle.bpel.services.identity.hierarchy.providers.hcm.HCMPositionHierar chyProvider" type="positionHierarchyProvider"> <initializationParameter name="wsdlUrl" value="http://example.com:20619/hcmTreesModel/HierarchyProviderService ?WSDL"/> <initializationParameter name="csf-key-name" value="FUSION_APPS_AMX_APPID-KEY"/> <initializationParameter name="http-read-timeout" value="30000"/> <initializationParameter name="securityPolicyName" value="oracle/wss_username_ token_client_policy"/> </serviceProvider> <serviceProvider classname="oracle.bpel.services.identity.position.provider.hcm.PositionLookupServi ceProvider" type="positionLookupProvider"> <initializationParameter name="wsdlUrl" value="http://example.com:20619/hcmEmploymentCore/positionLookupServic e?WSDL"/> <initializationParameter name="csf-key-name" value="FUSION_APPS_AMX_APPID-KEY"/> <initializationParameter name="http-read-timeout" value="30000"/> <initializationParameter name="securityPolicyName" value="oracle/wss_username_ token_client_policy"/> </serviceProvider> <serviceProvider classname="oracle.bpel.services.identity.position.provider.hcm.PositionDisplayName Provider" type="positionDisplayNameProvider"> <initializationParameter name="wsdlUrl" value="http://example.com:20619/hcmTreesModel/HierarchyProviderService ?WSDL"/> <initializationParameter name="csf-key-name" value="FUSION_APPS_AMX_APPID-KEY"/> <initializationParameter name="http-read-timeout" value="30000"/> <initializationParameter name="securityPolicyName" value="oracle/wss_username_ token_client_policy"/> </serviceProvider> </serviceExtension> </serviceExtensions> </ISConfiguration>
The workflow-identity-config.xml
file is in the MDS repository. For information about how to export the workflow-identity-config.xml
file from MDS, see Section 11.5.7.
Problem
You cannot remove or move a participant that was added through future participant editing. This is because the participant was anchored to you and you are the current assignee.
Solution
This feature is not supported.
Problem
Hierarchy list builders (for example, supervisory, job-level, and position) used in rules are not working as expected and are throwing security configuration-related errors. For example:
SOAPFaultException: FailedAuthorization
Hierarchy services are protected services using Oracle Web Services Manager policies (for example, supervisory, job-level, and position services). Elevated privileges are used for authentication. These privileges must be configured in Fusion Applications Control and in configuration files.
Solution
To resolve this problem:
Check the MDS store to see if workflow-identity-config.xml
and workflow-config.xml
are set up with the correct information (pointing to HCM services and the JAZN name).
Run the following script to connect to the WLST
shell on Linux operating systems. There is no equivalent script on Windows operating systems.
$MW_HOME/oracle_common/common/bin/wlst.sh
Run connect()
. This takes you to the prompt as shown in the following example:
wls:/offline> connect() Please enter your username [weblogic] :weblogic Please enter your password [weblogic] : Please enter your server URL [t3://localhost:7001] :t3://example.com:9401
Run the following command after connecting:
exportMetadata(application='soa-infra',server='soa_server1',toLocation='any Location on server',docs='/soa/configuration/default/workflow-identity-config.xml')
The file is stored in the soa/configuration/default
directory in the location given in the toLocation
attribute. After verifying, you can upload the file with the following command.
importMetadata(application='soa-infra',server='soa_ server1',fromLocation='any Location on server',docs='/soa/configuration/default/workflow-identity-config.xml')
Verify that the csf-keys
are specified as initialization parameters.
Ensure that the csf-key-name
value is FUSION_APPS_AMX_APPID
-KEY
in workflow-identity-config.xml
, as described in Section 11.5.5.
Check in Fusion Applications Control that the keys defined for the services have the correct values for authentication.
Retrieve a credential with a given map and key by using scripting to invoke the MBean operation JpsCredentialMXBean.getPortableCredential(map, key)
.
(map="oracle.wsm.security", key="FUSION_APPS_AMX_APPID-KEY")
For more information about this MBean, see the "Managing Credentials with OPSS Scripts" section of Oracle Fusion Middleware Application Security Guide.
Problem
Hierarchy services sometimes get overloaded and do not respond (for example, supervisory, job-level, and position services).
Solution
Check that the workflow-identity-config.xml
file for the problematic service has correct timeout settings defined as initialization parameters. The timeout parameter is http-read-timeout
. For an example of the configuration file, see Section 11.5.5. Note that increasing the timeout may cause errors, such as other timeouts and only puts more load on an already overloaded server.
Problem
Not all LDAP users are defined as people within Oracle Fusion HCM. For example, users may be defined in LDAP for Oracle Fusion Supply Chain Management that are not defined as people with positions in Oracle Fusion HCM. Therefore, the position lookup can generate a message that the user does not have positions.
Solution
This is a tolerated condition and the warning can be ignored if the user name specified in the warning is not an HCM user. If this is an HCM user, the problem is on the HCM side. Add a position to these users in the HCM application.
Problem
This issue may be noticed by organizations using position hierarchy such as governments and universities. After users log in, they do not see any approvals in their inbox. This is caused by a failure to look up the position. This can be caused by network issues when accessing the HCM services. Sometimes due to load balancing or network setup, HCM services may not be accessible and Oracle BPM Worklist has errors such as the following:
WSDL port is not accessible
or
Could not find the host
Solution
Check the network setup and ensure that the HCM services are accessible and WSDL ports can be accessed in the browser. Tools like ping
and traceroute
can also diagnose the problem.
If the services are not available, bring up the services and retry.
Restart the HCM server and SOA server. For information, see the following documentation:
"Starting Managed Servers with the java weblogic.Server Command" section in Oracle Fusion Middleware Managing Server Startup and Shutdown for Oracle WebLogic Server
"Starting and Stopping Managed Servers Using Fusion Middleware Control" section in Oracle Fusion Middleware Administrator's Guide
"Start and Stop Servers" and various startup and shutdown procedures of the Cluster section in the Administration Console Online Help
Problem
This feature is not supported. Escalation does not follow the job level, supervisory, or position hierarchies.
Solution
Build your own custom plug-in. For more details, see Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite.
For troubleshooting information about this issue, see the "Task Action Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Notification Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For additional details about configuring email driver properties, see the "Configuring Email Notifications for Oracle Fusion Applications" section in the Oracle Fusion Applications Post-Installation Guide.
For troubleshooting information about this issue, see the "Notification Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite. This section describes how to make the task actionable in the Human Task Editor during design time.
In addition, if notifications are sent, but are not actionable, the administrator may have configured the notification to be nonactionable through Oracle BPM Worklist. To make notifications actionable, select the Make notification actionable checkbox under the Event Driven subtab of the Task Configuration tab.
For additional details, see the "Configuring Email Notifications for Oracle Fusion Applications" section in the Oracle Fusion Applications Post-Installation Guide.
For troubleshooting information about this issue, see the "Notification Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Identity Service Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Identity Service Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Identity Service Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Task History Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Design Time at Runtime Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Human Workflow Service/System MBean Browser Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Human Workflow Service/System MBean Browser Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Human Workflow Service/System MBean Browser Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Human Workflow Service/System MBean Browser Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Human Workflow Service/System MBean Browser Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Test-to-Production Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Test-to-Production Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Test-to-Production Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Test-to-Production Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "AMX Extension Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about patching and deployment issues, see the "Troubleshooting Patching Sessions for SOA Composites" section in the Oracle Fusion Applications Patching Guide.
This section contains the following topics that describe common problems and solutions for Oracle SOA Suite performance:
Problem
SOA transactions are failing and logs indicate that the database is running out of space. The problem may occur with the SOA dehydration store or MDS store running out of space due to a high volume of transactions. In the latter example, you see errors such as the following:
java.sql.SQLException: ORA-01653: unable to extend table SH_MDS.CUBE_INSTANCE by 16 in tablespace FUSION_TS_TOOLS
This indicates that the tablespace is full and the database cannot extend it.
Solution
Purge the SOA dehydration store tables periodically, taking into account the appropriate record retention policies and ensuring that the applications have no dependencies on runtime data.
The purge should be followed by commands to coalesce the space. For the purging strategy to work, it is important to understand how long to retain the data in the database. Factors that drive the retention policy include the following:
Legal requirements
Line of business requirements
Overall company policy on retention of data
The longer the retention policy, the greater the volume of data that must be stored and, correspondingly, the higher the disk capacity requirements.
For details on creating a purging strategy, see the "Managing Database Growth" chapter in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Ensure that the database hardware has sufficient resources to handle the demands of Oracle Database partitioning prior to configuring your tables for partitioning.
For the SOA dehydration store, database partitioning using range partitioning and hash partitioning is an optimal solution. Partitioning by definition means storing data in multiple tables to reduce bigger data sets into smaller, more manageable data sets. Partitioning strategies play a large role in easing maintenance overheads (dropping and pruning the partition) and improving performance. Partitioning should at least be done for tables having high activity. This plays a large role in balancing disk I/O and preventing hot disks. One important requirement that you must meet prior to configuring your tables for partitioning is to ensure that the database hardware has sufficient resources to handle the demands of Oracle Database partitioning. If preproduction testing has indicated that the installation is large, Oracle expects that you have sized your environment (CPU, memory, and disk space) correctly to take advantage of the partitioning features.
Tune database parameters for memory, tablespace, and partitions to get maximum performance. For more information, see the "Tuning Database Parameters" section in the Oracle Fusion Middleware Performance and Tuning Guide.
For other tablespaces running out of space, use the following query to check for free tablespace:
SELECT Total.tablespace_name "Tablespace Name", Free_space, (total_space-Free_ space) Used_space, total_space, round((Free_space*100/total_space),2) "Free %" FROM (select tablespace_name, sum(bytes/1024/1024) Free_Space from sys.dba_ free_space group by tablespace_name) Free, (select tablespace_name, sum(bytes/1024/1024) TOTAL_SPACE from sys.dba_data_ files group by tablespace_name) Total WHERE Free.Tablespace_name = Total.tablespace_name AND Total.tablespace_name = '<tablespacename>' ORDER BY 5;
To increase tablespace settings, use the administrator account. For example:
alter tablespace tablespace_name add datafile 'datafile_name' size 500m autoextend on;
For more details, see the "Resolving Message Failure Caused by Too Many Open Files" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Problem
You observe slow application performance and/or memory trashing. For example, it may take longer to load and serve pages or to complete composite transactions. Response time may seem slower compared to normal behavior.
There are various reasons for slow performance. It may be due to a large number of servers running on the same host, or there may be a large number of records/sessions/locking at the database. Thread contention can also be a reason for poor performance.
Note:
All configuration changes in Fusion Applications Control may be unavailable due to this problem.
Solution
Check the CPU utilization to see if it is saturated due to a heavy load or an excessive number of processes in relation to CPU capacity.
If CPU utilization is at 100% during normal load hours (the target should be 70-80%), you have no capacity to handle a peak load and the hardware resources are insufficient. Add scale-out servers to handle the additional load.
Check applications using Fusion Applications Control to report on performance. Check the performance of ADF services and the invoke/response times in the BPEL process audit trail to identify if the issue is in Oracle Fusion Applications.
To maximize performance, it is recommended that you not set the logging level higher than the default INFO level.
For debugging purposes, you must set the logging level to the FINEST level. However, after issues are resolved, reset the logging level to the default level for best performance. It is also recommended that you set Audit Level to Production in the SOA Infrastructure Common Properties page. This can be set in Fusion Applications Control as follows:
Right-click soa-infra (SOA_cluster_name).
Choose SOA Administration > Common Properties.
Set Audit Level to Production.
Purge periodically based on retention requirements to maintain the service level agreements (SLAs).
To identify the tables where data growth may lead to performance issues, see the "Identifying Tables Impacted By Instance Data Growth" section in the Oracle Fusion Middleware Performance and Tuning Guide.
For information about the use of the purge scripts, see the "Managing Database Growth" chapter in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Check the memory/IO/paging/swapping/CPU usage load statistics using Top or Glance or another monitoring tool.
Optimize the JVM to avoid full garbage collection or out-of-memory errors.
Frequent garbage collection can be either due to higher memory usage or memory leaks.
Ensure that the sum of the maximum heap size of all the JVMs running on your system does not exceed the amount of available physical RAM to avoid operating system level paging activity.
Use the JRockit mission control memory profiling tools to get thread dumps and memory snapshots, which helps Oracle Support Services debug any code issues.
Optimize threads to avoid contention. Get a thread dump to investigate and submit it to Oracle Support Services.
Check the JVM and thread dumps for methods invoking the database to identify if database performance is a bottleneck.
Run database Automatic Workload Repository (AWR) snapshots to identify causes of database performance issues.
Ensure that database statistics are updated at regular intervals and other tunable parameters for memory, tablespace, and partitions are used effectively to obtain maximum performance.
Here are some common tuning recommendations. For more information, see the "Tuning Database Parameters" section in the Oracle Fusion Middleware Performance and Tuning Guide.
Put indexes and tables in as physically separate disk areas if possible.
Never put rollback segments with data or index segments.
Separate highly active tables and indexes into their own tablespaces.
Partition high activity tables and indexes to help balance disk I/O and prevent hot disks.
Have processes in place to generate database table statistics at regular intervals.
Tune database tables to control the high watermark (HWM) contention of large objects. Tune database advanced queues (AQ) to control high watermark (HWM) contention and ensure consistent performance of producing and consuming messages from AQ.
Specifically, the AQs to be aware of are as follows:
Family AQ | Queue Name |
---|---|
EDN |
|
CRM |
|
HCM |
|
FIN |
|
PRC |
|
PRJ |
|
SCM |
|
COMMON |
|
IC |
|
Cross Family Business Event Subscriptions |
|
Tune the BPEL and EDN thread counts to ensure optimal settings (for example, the Dispatcher Invoke Threads and Dispatcher Engine Threads properties on the BPEL Service Engine Properties page in Fusion Applications Control). If the thread configuration is too high, the servers run out of memory. If they are too low, the messages start backing up.
Tune the BPEL properties to reduce overhead (for example, disable the ValidateXML and StatsLastN (statistics gathering batch size) properties on the BPEL Service Engine Properties page in Fusion Applications Control) if they are not needed.
In case of integration with packaged applications (for example, Siebel), check if the issue lies with the legacy applications.
For more information about performance tuning the various components, see the "Top Performance Areas" chapter and the "SOA Suite Components" part in the Oracle Fusion Middleware Performance and Tuning Guide.
For information about setting properties on the BPEL Service Engine Properties page, see the "Configuring BPEL Process Service Engine Properties" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Problem
Fusion Applications Control pages are loading very slowly.
Solution
To improve Fusion Applications Control page load times on pages with instance and fault metrics, set the SOA Infrastructure Display Data Counts properties.
Right-click soa-infra (SOA_cluster_name).
Choose SOA Administration > Common Properties.
In the Display Data Counts section, deselect the Disable fetching of instance and fault count metrics checkbox.
Reduce the Duration Details value.
Click Apply.
In addition to setting this property, it is also useful to scope queries by time where available (for example, only show items occurring in the last hour).
For more information, see the "Optimizing the Loading of Pages with Instance and Fault Metrics" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
This section contains the following topics that describe common problems and solutions for Oracle SOA Suite maintenance:
Section 11.8.1, "JVM is Suspended If Any In-use Log Directories are Deleted"
Section 11.8.2, "Server Start Script Does Not Work for Scale-Out (Postprovisioning) Servers"
Section 11.8.4, "SOA Server Is Not Responding To Administrative Commands"
Section 11.8.5, "Undeploying a Composite Left in an Inconsistent State Due to a Failed Patch"
Section 11.8.6, "Increased Load Observed on RAC Instances when Using Multiple Oracle RAC Servers"
Section 11.8.8, "Log Files Have Been Deleted or Are Missing"
Problem
JRockit writes log information (called flight recorder files) under /tmp
when using JRockit Mission Control. When it rotates to a new flight recorder file, the JVM is suspended until it completes. If the old directories in use under /tmp
are deleted (for example, a cleanup or custom maintenance script to run to clear /tmp
when it is close to full to avoid server downtime), the domain may go into a suspended state because the JVM may wait for a directory that no longer exists.
Solution
Change any cleanup scripts/procedures to first shut down the domain cleanly.
Perform a cleanup of /tmp
.
Restart the domain.
To recover existing, suspended JVMs without restarting them, you must recreate the directories that the JVM was trying to use.
Problem
Scale-out servers, created after provisioning, cannot be started using back-end commands. The script start_fusion_Allservers.sh
only starts servers that are created during provisioning time.
Solution
To start scale-out servers, use Fusion Applications Control. See the "Starting and Stopping an Oracle WebLogic Server Cluster Using Fusion Applications Control" section in the Oracle Fusion Applications Administrator's Guide.
Problem
The SOA server (soa_infra
) does not start.
Solution
There can be various reasons for server startup issues. A discovery-based approach to find the root cause is required. Check the server and diagnostic logs as a first attempt to diagnose the issue. In addition, check the following:
Check if the database is not available or there are not enough connections available. Some failures with SOA runtime can result from database outage/connectivity issues. Perform the following steps:
Log in to Oracle WebLogic Server Administration Console.
In the Domain Structure, view the status by selecting Services > Data Sources > SOADatasource > Monitoring > Testing to test the data source. You can also check this from a SQL prompt.
Check the list of ports used for port conflicts.
Check if the MDS repository is unable to load shared documents (incorrect MDS configuration or the database that holds the MDS schema is not reachable).
Check the Coherence configuration if using a SOA cluster (use of unicast versus multicast).
This issue only applies after provisioning is complete. Clustering of SOA servers may fail if there are port conflicts (used by Coherence). For example, one SOA server may be picking up the deployment, but the other server is not. In some cases, this may only present the following error:
[soa_server1] [ERROR] [] [Coherence] [tid: Logger@352821903 3.6.0.4] [ecid: 46f620208907e045:63f295ec:12dd091ec2e:-8000-0000000000000003,1:27187] [APP: soa-infra] 2011-01-28 23:06:19.463/414.816 Oracle Coherence GE 3.6.0.4 <Error> (thread=[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', member=n/a): Error while starting cluster: com.tangosol.net.RequestTimeoutException: Timeout during service start: ServiceInfo(Id=0, Name=Cluster, Type=Cluster[[...
In this example, Coherence timeouts are prominently available. However, sometimes it presents itself with an unrelated error, such as the following:
Error creating bean with name 'SensorManager' defined in ServletContext resource
The root cause of this is still primarily related to Coherence configuration.
Check if both the Administration Server and Managed Server ports are open and accessible.
Check if Managed Server startup failed because the Administration Server is not reachable.
Check network issues (for example, IP routing filtering/rules that may be causing issues).
Check Oracle WebLogic Server LDAP security corruption.
The Managed Server may report that policies for the application System MBeans Browser already exist or do not exist.
For more information, see the "Setting the Frontend URL for the Administration Console and Setting Redirection Preferences" section and the "Setting the Frontend HTTP Host and Port" section in the Oracle Fusion Middleware Enterprise Deployment Guide for Oracle SOA Suite.
Problem
The SOA server (soa_infra
) is not responding to administrative commands. This problem can be due to your environment using a load balancer, and the default listener address or channel is local-only. Therefore, the connection cannot be made from the external client or server.
Solution
To resolve this problem, perform the following steps from the Oracle WebLogic Server Administration Console:
Ensure that the listener address is correctly defined and accessible.
From the Domain Structure, expand Environment and select Servers.
From the Servers table, click AdminServer(admin).
From the Settings for AdminServer page, click the Protocols tab.
Click the HTTP subtab.
Set the Frontend Host property to the load balancer address.
Similarly, set the Frontend HTTP Host and Frontend HTTP Port properties for the SOA cluster:
From the Domain Structure, expand Environment and select Clusters.
From the Clusters table, click domain_SOACluster
From the Settings for the domain_SOACluster page, click the HTTP subtab.
Set the Frontend Host property to the load balancer address.
Change the Frontend HTTP Host and Frontend HTTP Port properties.
For more information, see the "Setting the Frontend URL for the Administration Console and Setting Redirection Preferences" section and the "Setting the Frontend HTTP Host and Port" section in the Oracle Fusion Middleware Enterprise Deployment Guide for Oracle SOA Suite.
For troubleshooting information about this issue, see the "Troubleshooting SOA Composite Deployment Failures" section in the Oracle Fusion Applications Patching Guide.
Problem
An increased load is seen on Oracle Real Application Clusters (Oracle RAC) instances when using multiple Oracle RAC servers.
Solution
Use a multidata source against Oracle RAC instances using load balancing rather than failover. This ensures that the load is evenly distributed rather than all load staying on one node until it fails over, at which point all the connections are failed over to the other node.
Ensure that you set the Connection Test Frequency Seconds property at the multidata source level to a nonzero value. A value that is too low means more load on the listeners/RAC instances when multiple servers attempt to reconnect at the same time.
For more information about configuring JDBC multidata sources, see Oracle Fusion Middleware Configuring and Managing JDBC Data Sources for Oracle WebLogic Server.
Note:
The Oracle Fusion Applications default configuration for the these settings has been tuned for best performance. Only change these settings when the configuration has not been set out-of-the-box by Oracle Fusion Applications.
Problem
Get a list of all composite state values from the *_SOAINFRA.COMPOSITE_INSTANCE
state table (for diagnosis).
Solution
Table 11-3 shows the composite state values.
Table 11-3 Composite State Values
State | Description |
---|---|
|
Running |
|
Completed |
|
Running with faults |
|
Completed with faults |
|
Running with recovery required |
|
Completed with recovery required |
|
Running with faults and recovery required |
|
Completed with faults and recovery required |
|
Running with suspended |
|
Completed with suspended |
|
Running with faults and suspended |
|
Completed with faults and suspended |
|
Running with recovery required and suspended |
|
Completed with recovery required and suspended |
|
Running with faults, recovery required, and suspended |
|
Completed with faults, recovery required, and suspended |
|
Running with terminated |
|
Completed with terminated |
|
Running with faults and terminated |
|
Completed with faults and terminated |
|
Running with recovery required and terminated |
|
Completed with recovery required and terminated |
|
Running with faults, recovery required, and terminated |
|
Completed with faults, recovery required, and terminated |
|
Running with suspended and terminated |
|
Completed with suspended and terminated |
|
Running with faulted, suspended, and terminated |
|
Completed with faulted, suspended, and terminated |
|
Running with recovery required, suspended, and terminated |
|
Completed with recovery required, suspended, and terminated |
|
Running with faulted, recovery required, suspended, and terminated |
|
Completed with faulted, recovery required, suspended, and terminated |
|
Unknown |
|
State |
Problem
The SOA log files are getting deleted. This is typically due to the retention policy set in Fusion Applications Control.
Solution
Change the retention policy for the odl-handler
to the appropriate value in Fusion Applications Control:
Right-click soa-infra.
Choose Logs > Log Configuration.
This section contains the following topics that describe common problems and solutions for Oracle SOA Suite runtime:
Section 11.9.1, "Unable to Use Layered Customization on Specific Artifacts"
Section 11.9.2, "New or Deleted Artifacts Do Not Appear as a Layered Customization"
Section 11.9.3, "Warnings Regarding xml:id when Building or Deploying a Customized Composite"
Section 11.9.4, "Distributed Queue Topic Messages Are Retrieved Multiple Times by Subscribers"
Section 11.9.5, "Internal Key is Displayed Instead of a Translated String"
Section 11.9.9, "High-Volume Cross Reference (XREF_DATA) Table Impacts Performance and Maintenance"
Section 11.9.10, "Access Denied Error While Invoking an Oracle ADF BC Service from BPEL"
Section 11.9.12, "Synchronous Service Invocation Errors Due to WS-Addressing"
Section 11.9.14, ""Invalid Subject" Error Thrown with Human Workflow API or Notification"
Section 11.9.16, "Task Completes Without Any Assignment Occurring"
Section 11.9.18, "All Added Adhoc Participants Disappear After a Page Refresh"
Section 11.9.19, "Future Approvers Are Not Visible in the History Table"
Section 11.9.22, "oracle.jrf.UnknownPlatformException Error When Customizing an Oracle Mediator"
Section 11.9.23, "java.lang.NullPointerException When Customizing an Oracle Mediator"
Section 11.9.24, "JDeveloper Compilation Error in a SOA Project with SOA MDS Service Location"
Problem
Only certain artifacts allow layered customization in Oracle JDeveloper because it requires editors that use XMLEF to make delta changes to the underlying Document Object Model (DOM). XMLEF is the XML Editing Framework. XMLEF is an Oracle JDeveloper framework that supports flexible editing of XML documents and is used extensively in layered customizations.
Solution
The following SOA artifact types are customizable:
Composite (composite.xml
)
BPEL process (.bpel
)
Oracle Mediator (.mplan
)
The following SOA artifact types are not customizable:
XSL map (.xsl
)
Human task (.task
)
Decision service (.decs
)
Rules (.rules
)
Event definition (.edl
)
Domain value map (.dvm
)
The following common artifact types are not customizable because Oracle XDK/WSDL APIs do not support customization (although their editors do use XMLEF to make delta changes to the underlying DOM):
XSD schema (.xsd
)
WSDL document (.wsdl
)
Noncustomizable artifacts are not sensitive to a customization context. Any changes to those artifacts from any customization context are visible to other customization contexts.
For more information about customizations in Oracle JDeveloper and runtime tools (such as Oracle SOA Composer and Oracle BPM Worklist), see the "Customizing and Extending SOA Components" chapter in the Oracle Fusion Applications Extensibility Guide.
Problem
Because MDS customization does not capture artifact creations or deletions as deltas in the customization role, artifact creations and deletions in one customization context are visible to other customization contexts.
Solution
Artifact creations and deletions are not customization context sensitive. Therefore, only existing artifact customizations are supported in layers. Because new artifacts cannot be added as part of a layer, the artifacts are added to the project like any other base artifact. Any component artifacts that are added to the project are not incorporated into the runtime composite model unless the composite.xml
file has explicitly referred to it. Therefore, if a customization did not alter composite.xml
to include the newly added component, the new artifacts have no impact. Of course, the new artifacts are saved along with the base artifacts.
Additions of any new artifact introduced in the composite definition as a customization (for example, XSLT artifacts) must be named appropriately to prevent collisions with base XSLT artifacts. References to the new XSLT files must be made within the Oracle BPEL Process Manager/Oracle Mediator artifacts (through a layer).
Problem
When building or deploying the customized composite, you may get warnings similar to this because the xml:id
element is not supported:
[scac] [WARNING] Line [13] Column [29] Schema validation failed for C:\customizationdemo\project1\composite.xml <Line 13, Column 29>: XML-24535: (Error) Attribute 'http://www.w3.org/XML/1998/namespace:id' not expected.
Solution
This warning can be ignored and should not prevent you from deploying the composite successfully.
Problem
In a clustered environment, a queue topic is being retrieved multiple times by the subscribers. SOA clusters are typically homogenous (that is, each node has the same services running). Therefore, if a queue topic has multiple subscribers (for example, an error queue is subscribed to by a fallout notification service and a trouble ticketing service), then in a clustered environment, each service is running on every node. If it is a four-node cluster, then there are four instances of the fallout notification service and four instances of the trouble ticketing service, each of which retrieve the topic message. This is not desirable.
Solution
Set the consuming service singleton property to ensure that in a multi-node environment, only one of the subscriber instances can consume the topic message. In the example in the "Problem" section, the fallout notification service and the trouble ticketing service each should set their singleton property so that each of them consumes the message only once by any one of the four instances.
<service name="ConsumeFaultMessage" ui:wsdlLocation="ConsumeFaultMessage.wsdl"> <interface.wsdl interface="http://xmlns.oracle.com/pcbpel/adapter/jms/EH/AIAReadJMSNotificationPro cess/ConsumeFaultMessage/#wsdl.interface(Consume_Message_ptt)"/> <binding.jca config="ConsumeFaultMessage_jms.jca"> <property name="singleton">true</property> </binding.jca> </service>
Problem
An internal key is displayed instead of a translated string in the human task title, human task outcome, email subject and body, and human task mapped attributes displayed in Oracle BPM Worklist.
Solution
To resolve this problem:
Check the SOA server
-diagnostic.log
for errors for each server in the cluster. If a resource bundle or the key is not found, you see a similar error stack that contains the composite name and resource bundle name.
2010-09-29T23:45:46.702-07:00 soa_server1 ERROR \[\] oracle.soa.services.workflow.soa tid: orabpel.invoke.pool-4.thread-7 userId: weblogic ecid:59ab739a26595dc1:-38d33932:12b614ddb44,0:1:100000026 APP: soa-infra composite_name: RC3HumanTaskComposite component_name: Humantask1 component_instance_id: composite_instance_id: 310003 <WorkflowServiceEngine.request> Operation 'initiateTask' failed with exception 'EJB Exception: : java.util.MissingResourceException: Can't find bundleforbase name oracle.apps.hcm.people.soa.resource.HcmPeopleTopSoaBundleDummy, locale en\[\[ at java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1427 ) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1250) at java.util.ResourceBundle.getBundle(ResourceBundle.java:952)
Confirm that the resource bundle has been deployed to the correct location in SOA MDS by first exporting the contents of SOA MDS and confirming that the resource bundle class is defined under /apps/resource
. Follow these steps to export the metadata contents from Fusion Applications Control:
From the navigation pane, expand SOA and select soa-infra.
From the SOA Infrastructure menu, choose Administration > MDS Configuration.
Select Export metadata documents to an archive on the machine where the web browser is running.
Click Export.
If the resource bundle is not defined in SOA MDS or is in the wrong location in SOA MDS, then you may have encountered an error during patching or there is a problem with the resource bundle definition.
If the resource bundle is properly defined in SOA MDS and the string that is not translated is a mapped attribute that appears in the column heading of the worklist, then perform the followings steps to:
Set the System MBean Browser WorkflowCustomClasspathURL property to a value of "".
Apply the changes.
Set WorkflowCustomClasspathURL to "oramds:///apps/resource/"
.
Apply the changes in Fusion Applications Control.
Log in to Fusion Applications Control in the domain where the JAR file was deployed.
From the navigation pane, expand SOA and select soa-infra.
From the SOA Infrastructure menu, choose SOA Administration > System MBean Browser.
Go to Application Defined MBeans > oracle.as.soainfra.config > Server: SOA_cluster_name > WorkflowConfig > human-workflow.
Remove the value in the Value column for the WorkflowCustomClasspathURL attribute.
Click Apply.
Enter oramds:///apps/resource/
in the Value column of the WorkflowCustomClasspathURL attribute.
Click Apply.
If the resource bundle is properly defined in SOA MDS and the string that is not translated is not a mapped attribute that appears in the column heading of the worklist, then stop and restart the composite that references the resource bundle by following these steps:
Log in to Fusion Applications Control in the domain where the JAR file was deployed.
From the navigation pane, expand SOA and select soa-infra.
Click Shut Down.
Click Yes in the confirmation window.
Click Start Up.
Click Yes in the confirmation window.
Retest your transaction flow.
Problem
If entity variables are used in Oracle BPEL Process Manager, this error may be thrown at runtime:
com.oracle.bpel.entity.dataprovider.EntityVarMgrException:zero item
Solution
When the entity variable tries to bind the key provided to it at runtime in Oracle BPEL Process Manager and if it cannot find the key in the database, it throws this error. This may be due to human error. Either the developer has passed an incorrect primary key value to bind to the entity variable or the composite has been deployed to the wrong business component (BC) URL in the deployment plan. To resolve, ensure that the primary key value that gets bound to the entity variable at runtime exists in the database used by the BC service.
Problem
If the payload node used in an activity such as an assign activity in BPEL is empty/null, then it throws this error.
The XPath variable or expression /ns4:findDtsPurchasingDeliverable1FindByDelStatusResponse/ns4:resultns2:Deliverabl eId=$DeliverableIdVar/ns2:BatchId is empty at line 344. selectionFailure: com.oracle.bpel.client.BPELFault: faultName: {{http://schemas.xmlsoap.org/ws/2003/03/business-process/}selectionFailure}
Solution
This error is thrown when a selection operation performed either in a function or in an assignment encounters an error. If there is a possibility of getting a null/empty value for a payload element at runtime, then you can get around this error by enclosing this element in a switch case with a condition check of (string-length(
payload_element
) > 0
) or a bpelx:assert
element to test for null conditions in their code. If these errors are encountered at runtime, there is information in the log and the BPEL audit trail flow.
Problem
When a BPEL process has partner links targeted to external servers, during start up of the SOA server or activation of the BPEL composite, the BPEL service engine recognizes that the target server is down and marks the BPEL composite as unavailable. Even after the target server becomes available, there is no way to make the BPEL process available without restarting the complete SOA server or redeploying the BPEL process. Until then, the BPEL composite is not listed in the navigation pane of Fusion Applications Control and is only visible through the Deployed Composites tab. Even though the composite is displayed here (sometimes as active), if you click the process, you receive this error:
The composite <Process Name>[ <Version> ] is not available. This could happen because either the composite has been undeployed or EM has not yet discovered this composite.
Solution
The issue is caused by using a concrete WSDL file in the BPEL definition. Using an abstract WSDL prevents the BPEL composite/process from becoming inactive or its state from becoming retired when the target server or service is down.
Oracle Fusion Applications develop composites with oramds://
references to WSDL files at design time for both the binding and the WSDL location. At deployment, the binding's concrete location (which is only used at runtime) is fixed.
Follow these steps to use an abstract WSDL by making a local copy of the target WSDL file in your BPEL project.
When you create a web service partner link in a composite in the Create Web Service dialog, select the copy WSDL and its dependent artifacts into the project checkbox.
After you are done with web service creation and the partner link is wired (connected) with the BPEL process service component, you see two WSDL files in the Application Navigator of Oracle JDeveloper:
Webservice.WSDL (the local copy of your target WSDL)
WebserviceRef.WSDL
Redeploy the SOA composite.
Retest the scenario.
Problem
When the volume of data in the XREF Oracle SOA schemas grows very large, maintaining the database can become difficult. It may have an impact on performance and may be identified in AWR snapshots.
Solution
To address this maintenance challenge, Oracle SOA 11g has been instrumented with partition keys that enable database administrators (DBAs) to take advantage of the Oracle RDBMS partitioning features and capabilities. Even thought multiple types of partitioning are possible, the list partitioning strategy is the preferred one for partitioning the cross reference data table.
List partitioning allows greater flexibility in the mapping of rows to partitions than range or hash partitioning. Because the cross reference data (XREF_DATA
) is distributed based on discrete column values, unordered and unrelated sets of data can be grouped together with no relationship between the partitions. For this table, the list partitioning is based on the column XREF_TABLE_NAME
. Because you know the list of entities that have cross reference data, this strategy is ideal. The only caveat is that you must repeat this activity if a new entity gets added to the system.
Better understanding of data distribution can lead to further refinement and space management by employment of composite partitioning strategies such as list - list and list - range schemes. For information about partitioning, see the Oracle Database Performance Tuning Guide.
In addition to this partitioning strategy, you can also use Oracle Mediator's feature by which custom database tables can be created to store cross reference data for certain high volume entities. For more information, see Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite.
Problem
A common case in Oracle Fusion Applications is Oracle ADF BC service invocations from various SOA components. If the user context attached with the SOA composite instance or identity switching used in a SOA reference does not have proper permission to access the Oracle ADF BC service, then an access
denied
error is displayed.
Solution
Common causes for this error are as follows. These must be resolved.
The Oracle ADF BC service does not have required permissions in jazn-data.xml
.
The permissions provided in jazn-data.xml
are not properly reflected in system-jazn-data.xml
.
The ejb-jar.xml
file does not have the <interceptors>
tag added or it has an incorrect application name. An example of ejb-jar.xml
with the correct tags is as follows:
<ejb-jar xsi:schemaLocation="http://java.sun.com/xml/ns/javaee [http://java.sun.com/xml/ns/j2ee/ejb-jar_3_0.xsd]"version="3.0"> <enterprise-beans> <session> <ejb-name>oracle.apps.financials.payables.invoices.transactions.invoiceApproval Service.InvoiceApprovalBCServiceBean</ejb-name> <resource-ref><res-ref-name>jdbc/ApplicationDBDS</res-ref-name><res-type>javax. sql.DataSource</res-type><res-auth>Container</res-auth></resource-ref> <resource-ref><res-ref-name>jdbc/ApplicationServiceDBDS</res-ref-name><res-type >javax.sql.DataSource</res-type><res-auth>Container</res-auth></resource-ref> </session> </enterprise-beans> <interceptors> <interceptor> <interceptor-class>oracle.security.jps.ee.ejb.JpsInterceptor</interceptor-class > <env-entry> <env-entry-name>application.name</env-entry-name> <env-entry-type>java.lang.String</env-entry-type> <env-entry-value>fscm</env-entry-value> <injection-target> <injection-target-class>oracle.security.jps.ee.ejb.JpsInterceptor</injection-ta rget-class> <injection-target-name>application_name</injection-target-name> </injection-target> </env-entry> </interceptor> </interceptors> <assembly-descriptor> <interceptor-binding> <ejb-name>*</ejb-name><interceptor-class>oracle.security.jps.ee.ejb.JpsIntercep tor</interceptor-class> </interceptor-binding> </assembly-descriptor> </ejb-jar>
Problem
At runtime, when you click the Manage Approvals page (for both a requisition and purchase order), you may get a SOA
server
may
be
down
error among other errors. The page may also show a No
data
to
display
error.
Solution
Check to see if you have wf_client_config.xml
pointing to the correct SOA server. Oracle Fusion Applications environments are all multidomain and applications that must query their local SOA runtime must do so through the wf_client_config.xml
file configuration.
For more information about the wf_client_config.xml
file, see Section 11.5.3.
Problem
When integrating with external web services, you may have issues related to the WS-Addressing headers in the request and response XML documents used in synchronous request-response scenarios. This may even occur if there is a dynamic endpoint that resolves to an external web service that does not support WS-Addressing.
A SOA composite acting as a client fails to invoke a service provider that does not support WS-Addressing headers in the request message for synchronous interactions.
An external service consumer that does not support WS-Addressing headers in the response message fails to invoke a synchronous operation of a SOA composite.
A service provider throws a wsa:InvalidAddressingHeader
error when SOAPAction
is defined in the WSDL <binding>
element of the service provider interface.
Solution
The WS binding component of Oracle Fusion Middleware 11g generates WS-Addressing headers by default, which is as per the specification, but may be incompatible with some web services. This behavior can be overwritten using properties that allow you to control the generation of WS-Addressing header information for interoperability purposes.
A SOA composite acting as a client fails to invoke a service provider that does not support WS-Addressing headers in the request message for synchronous interactions.
When your Oracle SOA Suite components act as service clients (consumers) and invoke synchronous operations of external web services, the invocation fails. The invocation failure is due to the WS-Addressing headers included in the request message when invoking the service provider.
The stack trace displays the following information:
NSGetCustomer (faulted) Nov 10, 2009 4:26:40 PM Faulted while invoking operation "get" on provider "NetSuite2009_PL". <payload> Nov 10, 2009 4:26:40 PM "{urn:platform_2009_1.webservices.netsuite.com}UnexpectedErrorFault" has been thrown. <payload> Nov 10, 2009 4:26:40 PM There is a system exception while performing the BPEL instance, the reason is "Invalid SOAP Header: '<ns1:To xmlns:ns1="http://www.w3.org/2005/08/addressing">https://webservices.netsuite.c om/services/NetSuitePort_2009_1</ns1:To>'. Value is 'https://webservices.netsuite.com/services/NetSuitePort_2009_1'." <payload>
In Oracle SOA Suite 11g, the WS-Binding component includes the following WS-Addressing headers:
wsa:to
wsa:replyTo
wsa:action
These headers are included by default regardless of interacting with synchronous or asynchronous web service operations. Although the SOAP message generated by 11g is fully specification compliant, there are service provider implementations that do not understand these WS-Addressing headers when included in the request message for synchronous operations. This is why the invocation fails.
Oracle Fusion Middleware 11g uses new standards, WS-Addressing (wsa
) and WS-I Basic profile, which demand that WS-Addressing headers be added by default to the request message.
To increase interoperability, the property oracle.soa.ws.outbound.omitWSA
has been introduced for the <binding.ws>
element in the composite.xml
file. This property enables you to optionally suppress generation of WS-Addressing headers in the request message when set to true
. The default value is false
.
Manually add the following property to the <binding.ws>
element in the composite.xml
file.
[snippet] <binding.ws ...... <property name="oracle.soa.ws.outbound.omitWSA" type="xs:boolean" many="false" override="may">true</property> </binding.ws> [/snippet]
Redeploy the composite.
An external service consumer that does not support WS-Addressing headers in the response message fails to invoke a synchronous operation of a SOA composite.
When external services invoke the endpoints of Oracle SOA Suite composites, which expose synchronous operation(s) acting as a server (service provider), the external web service fails with invocation errors. This is commonly due to the WS-Addressing headers included in the response message, which are not supported by some web service implementations.
In SOA Suite 11g, the WS-Binding component includes the WS-Addressing headers by default in the response messages. Although the SOAP message generated by 11g is fully specification compliant, there are service consumer implementations that do not understand the WS-Addressing headers when included in the response message in synchronous request-response scenarios. This is why the invocation fails.
To increase interoperability with clients (consumers) that do not support WS-Addressing headers, a property named oracle.soa.addressing.response.enabled
has been introduced. This property can be used within the <binding.ws>
element in the composite.xml
file. This property enables you to optionally suppress generation of WS-Addressing headers in the response message when set to true
. The default value is false
.This means that in a synchronous request-response scenario, in which the Oracle SOA Suite composite acts as a service provider, WS-Addressing headers are included in the response message by default. If the property is set to false
, no WS-Addressing-related headers are returned in the response message.
Set the property in the composite.xml
file as follows:
[snippet]
<binding.ws>
......
<property name="oracle.soa.addressing.response.enabled" type="xs:boolean"
many="false" override="may">false</property>
</binding.ws>
[/snippet]
A service provider throws a wsa:InvalidAddressingHeader
error when SOAPAction
is defined in the WSDL <binding>
element of the service provider interface.
When invoking a web service provider with a synchronous interface from a composite deployed on 11.1.1.1, a wsa:InvalidAddressingHeader
message is returned. This happens when both of the following conditions are met:
The service interface (WSDL) of the service provider does not include the wsa:Action
element in the WSDL input message for the port operations
and
The soapAction
attribute in the WSDL binding section of the service provider interface is specified.
The difference with the first scenario is that the service provider interface may accept WS-Addressing headers for synchronous endpoint operations by using the WS-Addressing extensibility attribute wsa:UsingAddressing
in the WSDL binding section. A WSDL binding section sample is shown as follows:
<wsdl:binding name="mySoapBinding"
type="impl:myService">
<wsaw:UsingAddressing wsdl:required="false"
xmlns:wsaw="http://www.w3.org/2006/05/addressing/wsdl"/>
<wsdlsoap:binding style="document"
transport="http://schemas.xmlsoap.org/soap/http"/>
<wsdl:operation name="myOperation">
<wsdlsoap:operation soapAction="myAction"/>
<wsdl:input name="myRequestMsg">
<wsdlsoap:body use="literal"/>
</wsdl:input>
<wsdl:output name="myResponseMsg">
<wsdlsoap:body use="literal"/>
</wsdl:output>
</wsdl:operation>
</wsdl:binding>
As the following code sample shows, neither the input message nor the output message (as defined in the service provider WSDL) contain a <wsa:Action>
element:
.... <wsdl:message name="myRequestMsg"> <wsdl:part name="parameters" element="impl:myRequestElement"/> </wsdl:message> ..... <wsdl:message name="myresponseMsg"> <wsdl:part name="parameters" element="impl:myResponseElement"/> </wsdl:message>
The problem has been observed when integrating with synchronous web services deployed on the IBM WebSphere 6.1 Application Server.
The stack trace looks similar to the following code example:
...... oracle.fabric.common.FabricInvocationException: javax.xml.ws.soap.SOAPFaultException: A header representing a Message Addressing Property is not valid and the message cannot be processed at oracle.integration.platform.blocks.soap.WebServiceExternalBindingComponent.thro wFabricInvocationException(WebServiceExternalBindingComponent.java:415) at oracle.integration.platform.blocks.soap.WebServiceExternalBindingComponent.thro wFabricInvocationExceptionForSoapFault(WebServiceExternalBindingComponent.java: 411) at oracle.integration.platform.blocks.soap.WebServiceExternalBindingComponent.proc essSOAPFault(WebServiceExternalBindingComponent.java:394) at oracle.integration.platform.blocks.soap.WebServiceExternalBindingComponent.proc essOutboundMessage(WebServiceExternalBindingComponent.java:253) at oracle.integration.platform.blocks.soap.WebServiceExternalBindingComponent.send SOAPMessage(WebServiceExternalBindingComponent.java:644) at oracle.integration.platform.blocks.soap.WebServiceExternalBindingComponent.requ est(WebServiceExternalBindingComponent.java:526) at oracle.integration.platform.blocks.mesh.SynchronousMessageHandler.doRequest(Syn chronousMessageHandler.java:139) at oracle.integration.platform.blocks.mesh.MessageRouter.request(MessageRouter.jav a:179) at oracle.integration.platform.blocks.mesh.MeshImpl.request(MeshImpl.java:144) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at ......
If you use HTTP Analyzer or Apache TCP Monitor to trace the message exchange, you observe that the issue is related to the <wsa:action>
element in the request SOAP header, which is not the same as SOAPAction
in the HTTP header. The response message looks like the following code sample:
<soapenv:Envelope> ....... <soapenv:Header> <wsa:FaultDetail xmlns:wsa="http://www.w3.org/2005/08/addressing"> <wsa:ProblemHeaderQName>wsa:Action</wsa:ProblemHeaderQName> </wsa:FaultDetail> </soapenv:Header> <soapenv:Body> <soapenv:Fault> <faultcode>wsa:InvalidAddressingHeader</faultcode> <faultstring> <![CDATA[A header representing a Message Addressing Property is not valid and the message cannot be processed]]></faultstring> </soapenv:Fault> </soapenv:Body> </soapenv:Envelope>
As mentioned in the previous sections, the WS Binding component of Oracle Fusion Middleware 11g generates WS-Addressing headers by default. According to the W3C specification (Web Services Addressing 1.0 - WSDL Binding W3C Candidate Recommendation 29 May 2006), Section 4.4.1 Explicit Association:
"In the absence of a wsa:Action
attribute on a WSDL input element where a SOAPAction
value is specified, the value of the [action] property for the input message is the value of the SOAPAction
specified."
The generated value for the <wsa:Action>
element of the input message matches the value of the SOAPAction
element, if that is specified in the WSDL binding section. Otherwise, it generates a default value for the <wsa:Action>
element.
A sample of the generated request is as follows (provided WSDL <binding>
and <messages>
section, as shown previously).
POST /currencyratesapi/services/myAction HTTP/1.1 Host: <host>:<port> Connection: TE TE: trailers, deflate, gzip, compress User-Agent: Oracle HTTPClient Version 10h SOAPAction: "myAction" Accept-Encoding: gzip, x-gzip, compress, x-compress ECID-Context: 1.0000IUhXOGw3R9BpNWK6ye1BPi6A0002Wk;kYhgp8TLo0hgv0 Content-type: text/xml; charset=UTF-8 Content-Length: 1473 <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="http://www.w3.org/2005/08/addressing"> <env:Header> <wsa:To>http://<host>:<port>/currencyratesapi/services/myAction</wsa:To> <wsa:Action>myAction</wsa:Action> <wsa:MessageID>...</wsa:MessageID> <wsa:RelatesTo>...</wsa:RelatesTo> <wsa:ReplyTo> <wsa:Address>http://www.w3.org/2005/08/addressing/anonymous</wsa:Address> <wsa:ReferenceParameters> ...... </wsa:ReferenceParameters> </wsa:ReplyTo> </env:Header> <env:Body> ...... </env:Body> </env:Envelope>
Problem
Almost all Oracle Fusion Applications have the hwtaskflow.xml
file with the IntegrateTaskFlowWithTask
entry in the web.xml
file to register the notification projects. When deploying the J2EE EAR file, it throws the following error if you did not configure the foreign JNDI connection or wf_client_config.xml
with a valid SOA environment.
javax.naming.NameNotFoundException:Unable to resolve 'TaskQueryService'. Resolved ''; remaining name 'TaskQueryService'
Solution
If the SOA server is not up and running, then start the SOA server using Fusion Applications Control. If it is up and the problem exists, then check that the wf_client_config.xml
file has a valid SOA environment. If possible, avoid foreign JNDI references because the JNDI resources get bound into all servers in the domain that causes BindException
errors for servers in which the actual service exists when source and target JNDI names are the same. Oracle Fusion Applications do not use foreign JNDI references, instead bundling a correctly configured wf_client_config.xml
file in the EAR file or using APIs at runtime to generate the Java Architecture for XML Binding (JAXB) for the human workflow APIs.
Use wf_client_config.xml
(bundled at the application level) for all ADF worklist client applications communicating with Oracle SOA Suite. Ensure that you specify the correct URL in the file configuration.
<?xml version="1.0" encoding="UTF-8" ?> <workflowServicesClientConfiguration xmlns="http://xmlns.oracle.com/bpel/services/client"> <server name="default" default="true" clientType="REMOTE"> <localClient> <participateInClientTransaction>false</participateInClientTransaction> </localClient> <remoteClient> <serverURL>t3:cluster://CRMSoaCluster</serverURL> <initialContextFactory>weblogic.jndi.WLInitialContextFactory</initialContextFa ctory> <participateInClientTransaction>false</participateInClientTransaction> </remoteClient> <soapClient> <rootEndPointURL>http://adc60048fems.us.example.com:6361/soa-infra</rootEndPointURL > </soapClient> </server> </workflowServicesClientConfiguration>
Problem
When Oracle SOA Suite is installed on Domain1
and ADF on Domain2
and if any remote calls are performed between them for any human workflow API use or for notifications, errors similar to the following are thrown if the domain trust is not set.
javax.servlet.ServletException: javax.servlet.ServletException:java.lang.SecurityException: Security:090398Invalid Subject:principals=CVBUYER01, AR_MANAGER_VISION_OPERATIONS_DATA, AR_ACCOUNTS_ RECEIVABLE_MANAGER_JOB, FBI_TRANSACTIONAL_BI_WORKER
Solution
Set the domain trust password on both domains (global trust).
Log in to Oracle WebLogic Server Administration Console.
In the Domain Structure, click the domain name.
Click the Security tab, and then click Advanced.
Set the password in the Credential and Confirm Credential fields.
Note:
Oracle Fusion Applications do not support RMI across domains. That is, all remote calls between ADF and SOA are always within the same domain. When cross domain communication is required, SOAP is used.You may encounter the scenario described in this section in your development environment. For example, you may be running the ADF task flow for human tasks in the JDeveloper-integrated Oracle WebLogic Server and running the composite in an independent standalone SOA server. Both are in different domains.
For troubleshooting information about this issue, see the "Task Assignment/Routing/Escalation Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Task Assignment/Routing/Escalation Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Task Assignment/Routing/Escalation Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Task History Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Task History Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Task History Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
For troubleshooting information about this issue, see the "Task History Issues" section in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.
Problem
You can receive the following error when making customization modifications to an Oracle Mediator service component in a SOA application.
For example, assume you perform the following steps:
Log in to JDeveloper with the Fusion Application Developer role.
Create a SOA application and project with the Customizable option selected.
Add appropriate JAR files under the Libraries and Classpath section of the Project Properties dialog for the project.
Under Application Resources > Descriptors > ADF META-INF > adf-config.xml, add appropriate customization classes.
Add an Oracle Mediator service component to the application.
Save the application and restart JDeveloper.
Add appropriate routing rule and XSL transformations to the Oracle Mediator service component.
The following error is displayed in the command prompt window:
oracle.jrf.UnknownPlatformException: JRF is unable to determine the current application server platform.
Solution
This error can be ignored. After this error occurs, everything still works correctly.
Problem
You can receive the following error when making a customization modification to an Oracle Mediator service component in a SOA application.
For example, assume you perform the following steps:
Log in to JDeveloper with the Fusion Application Developer role.
Create a SOA application and project with the Customizable option selected.
Add appropriate JAR files under the Libraries and Classpath section of the Project Properties dialog for the project.
Under Application Resources > Descriptors > ADF META-INF > adf-config.xml, add customization classes.
Add an Oracle Mediator service component to the application.
Add a routing rule to the Oracle Mediator service component.
The following error is displayed in the command prompt window:
java.lang.NullPointerException
Solution
If you do not see any alerts indicating that a WSDL file cannot be found or parsed, then this exception in the log/console can be ignored.
Problem
Start JDeveloper with the Fusion Application Developer role.
Create a SOA application.
Create and deploy a SOA bundle to the server.
Create another SOA application.
Create an Oracle Mediator service component.
Create a reference web service and access the SOA MDS connection and browser for the service of the SOA application that you created in Step 2.
Wire the components.
Update the empty port
and location
values in the composite.xml
file. For the location
value, enter the oramds
path. For example:
<reference name="Service1" ui:wsdlLocation="oramds:/apps/CalleeProject/CalleeBPELProcess.wsdl"> <interface.wsdl interface="http://xmlns.oracle.com/Comp2Comp_ EDGApp/CalleeProject/CalleeBPELProcess#wsdl.interface(CalleeBPELProcess)"/> <binding.ws port="http://xmlns.oracle.com/Comp2Comp_ EDGApp/CalleeProject/CalleeBPELProcess#wsdl.endpoint(calleebpelprocess_client_ ep/CalleeBPELProcess_pt)" location="oramds:/apps/CalleeProject/CalleeBPELProcess.WSDL"/> </reference>
Compile the application.
You receive the following error message.
Warning(26,76): Failed to Find Binding
Solution
This is only a warning message thrown by the compiler. Runtime works fine. Having a reference from SOA MDS in a composite makes the port
and location
values empty in the bindings. To resolve this warning message:
Explicitly enter the corresponding details. For example, replace the binding.ws
location
value with the concrete WSDL file.
Deploy the composite using the configuration plan, which replaces the port and oramds
location with the concrete WSDL file.