Troubleshoot Integration Runtime
Learn about troubleshooting integration runtime in Oracle Integration.
Topics:
- An Integration Runs Slowly
- Fewer Integration Instances Ran Than Usual
- Using Basic Authentication to Call Services When Oracle Identity Cloud Service is Configured With Multi-Factor Authentication Causes a 401-Unauthorized Error
- Perform Remedial Actions When the Payload Exceeds the Size Limit
- Remove Unprintable Control Characters from the Incoming Payload
- Notification Action Returns a <from> value is invalid Error
- Synchronous Integration Timeout Failure
- Activity Stream Details
- Message Dequeuing Takes Three to Four Seconds in Asynchronous Integrations
An Integration Runs Slowly
Several common issues can result in performance issues for an integration. For example, when the target application takes longer than expected to respond, the integration also runs slower than expected.
How to Determine If Your Integration Is Running Slowly
See Determine an Integration's Typical Processing Times, and then take the appropriate action:
-
If the integration has always run slower than expected, determine whether it contains a common pitfall.
See Common Integration Pattern Pitfalls and Design Best Practices.
- If the integration has detectable performance outliers, follow the troubleshooting steps on this page.
Step 1. Investigate the Target Application
The target application for an invoke connection might be taking longer than usual to respond. A slow target application is the most common cause of slow-running integrations.
To determine whether the target application caused the slowness for your integration, check the logs of the application. The logs help you determine whether the application was experiencing an issue when Oracle Integration connected to it.
Step 2. Check the Number of Incoming Messages
When an integration receives a large number of messages to process, Oracle Integration adds the messages to a queue and processes them until it clears the queue. A long queue can impact the overall performance of your system. For example, a retailer might experience higher message volume during a promotion.
To see how many messages Oracle Integration processed in the last hour:
-
In the navigation pane, click Observability, then Integrations.
-
Click Filter
, and update the following filtering requirements:
- Time window: Last 1 hour.
- Status: All.
- Click Apply.
- Review the numbers in the Processed column.
Keep in mind that the number of messages is just one factor in performance. Smaller messages that contain very large incoming payloads can also impact performance.
Step 3. Determine Whether the Message Counts Are Too High
-
Work with an administrator to check whether your organization purchased enough message packs for the messages that you're using. See Edit the Edition, License Type, and Message Packs of an Instance in Provisioning and Administering Oracle Integration 3.
-
If the integration is synchronous, determine whether the integration is trying to process too many requests at a time. See Calculate Requests Per Second in Provisioning and Administering Oracle Integration 3.
Step 4. Check for a Clustered Schedule (Schedule Integrations Only)
Schedule integrations can run slower than expected when too many integrations start at the same time.
To determine whether your schedule integrations are clustered together, see View the Calendar of Schedule Integration Runs.
Step 5. Look For Inefficiencies
If the integration processes records, you might be able to fine-tune the efficiency of the processing.
Action for processing records | Recommendations |
---|---|
For-each action |
You typically can process 10,000 or fewer records using the for-each action without experiencing performance issues. To process 10,000 or more records, use a for-each action within a parallel action. The parallel action allows you to process the records in parallel threads, thereby improving the performance of the integration.
|
Stage file action |
You typically can read and write 10,000 or fewer records within a file using the stage file action without experiencing timeouts. To process a file with 10,000 or more records, use a stage file action within a parallel action. The parallel action allows you to read and write the records in chunks, thereby improving the performance of the integration.
|
Haven't Solved the Problem?
Get help from Oracle by entering a service request.
Fewer Integration Instances Ran Than Usual
Consider a scenario in which 100 integration instances typically run by a specific time of day, but only 80 integration instances have run by that time today. You need to quickly determine what happened with your missing integration instances.
Note:
The Observability pages show information about the integrations that are running and have already run, not any missing instances. To identify whether any integration instances are missing, you must know the integrations that usually run.Why Integrations Don't Run
An integration might not run for a number of reasons. For example:
- Someone deactivated an integration.
- The external application that triggers an integration is down.
- Someone changed the schedule for a schedule integration, such as changing it from running hourly to running daily.
To Troubleshoot Missing Integration Instances:
Using Basic Authentication to Call Services When Oracle Identity Cloud Service is Configured With Multi-Factor Authentication Causes a 401-Unauthorized Error
If Oracle Identity Cloud Service is configured with multi-factor
authentication (for example, user name / password + multi-factor authentication) and you use
basic authentication to invoke Oracle Integration services, you receive a
401-Unauthorized
error. None of the adapters or the connectivity agent
in Oracle Integration support multi-factor authentication.
- Disable multi-factor authentication from the account with which you are using basic authentication.
- Use OAuth instead.
Perform Remedial Actions When the Payload Exceeds the Size Limit
When the message payload in an integration exceeds the size limit, the message is rejected and an error message is displayed. The error message describes remedial actions to perform.
Issue | Error Message | Remedial Action to Perform |
---|---|---|
Payload size limit exceeded |
|
The preferred way to send large data sets is by using attachments. Check whether the target application exposes a SOAP API to return data sent as an attachment. If yes, the integration can be designed to configure an invoke connection using the SOAP Adapter to accept an MTOM attachment. The attachment can be an archive containing an XML/comma-separated value (CSV) document. |
Attachment size limit exceeded |
|
Check whether the attachment can be broken and sent through multiple APIs. For example, if the attachment is a ZIP archive containing a large CSV file and unstructured documents, have one API that returns the CSV file and another API that retrieves the unstructured documents. |
For additional details about 50 MB payload support, limits, and best practices, see Service Limits in Provisioning and Administering Oracle Integration 3.
Remove Unprintable Control Characters from the Incoming Payload
Oracle Integration does not support certain non-printable control characters in the following range.
['\u0000', '\u0001', '\u0002', '\u0003', '\u0004', '\u0005', '\u0006',
'\u0007', '\u0008', '\u000b', '\u000c', '\u000e', '\u000f', '\u0010', '
\u0011', '\u0012', '\u0013', '\u0014', '\u0015', '\u0016', '\u0017', '\
u0018', '\u0019']
{
"type" : "10.4.1",
"title" : "exception occurred while translating the request from native to
xml.",
"detail" : "oracle.cloud.connector.api.CloudInvocationException:
ORABPEL-15235\n\nTranslation Failure.\nFailed to translate JSON to XML.
org.codehaus.jackson.JsonParseException: Illegal unquoted character
((CTRL-CHAR, code 1)): has to be escaped using backslash to be included in
string value\n at [Source: java.io.BufferedReader@.#; line: 2, column:
22]\nThe incoming data does not conform to the NXSD schema. Please correct
the problem.\n",
"o:errorCode" : "TRANSLATION-ERROR-00",
"o:errorDetails" : [ {
"type" : "NA",
"instance" : "NA",
"title" : "NA",
"o:errorPath" : "NA",
"o:errorCode" : "NA"
} ]
}
If you receive this error, contact Oracle Support for information on how to preprocess incoming JSON payloads and filter unprintable control characters.
Notification Action Returns a <from> value is invalid Error
If the notification action in an integration validates successfully, but the following error occurs during integration runtime, the value specified in the Parameter Value field of the notification action may be incorrect.
:
summary=<summary>The <from> value is invalid.
Xpath expression associated with <from> in copy assign activity is invalid.
There is an user error that results in missing element value(s) in the xpath query.
Please review the payload and modeling to ensure that all elements defined in the <from> xpath query have valid non-null values .
</summary>
:
,query=<query>fn:current-dateTime()</query>
The error can be confirmed by viewing errors for the integration instance on
the Instances page or the response of a REST testing tool such as
Postman
.
string(fn:current-dateTime())
Synchronous Integration Timeout Failure
If a synchronous integration fails to complete within 300 seconds, the following error appears in the activity stream that is viewable from the Instances page or Errors page.
CompositeException in: io.reactivex.internal.operators.single.SingleOnErrorReturn$OnErrorReturn.onError2 exceptions occurred
No details are returned when the call is processed (for example, no
502
status error is returned).
Optimize your integration and/or the endpoint you invoke to complete instance processing in under 300 seconds.
See Service Limits in Provisioning and Administering Oracle Integration 3.
Activity Stream Details
You can enable tracing with the payload, view the activity stream logs on the Instances page, and download the activity stream logs with the REST APIs.
Enable Tracing with the Payload and View the Activity Stream on the Instances Page
Download the Activity Stream Logs with the REST APIs
- See Retrieve Activity Stream Messages
- See Download a Log File
(by specifying the
icsflowlog
parameter)
Note:
If you decide to use absolute values (10 MB, instead of a certain size), downloading the file downloads the latest 10 MB file only.Message Dequeuing Takes Three to Four Seconds in Asynchronous Integrations
You may observe that Oracle Integration sometimes takes three to four seconds to go from Message Received:Instance
created and enqueued for processing
to message dequeued and processed
by Trigger trigger_name
. This increases the end-to-end processing time
of a particular instance.
This is the expected behavior with asynchronous requests. Asynchronous requests are processed using a queue. The first two entries in the flow logs show the enqueue and dequeue times. Therefore, a few seconds lag is expected.
403 Error When Accessing an API Protected Using OAuth
OAuth enables Oracle Integration to access a user's resources on their
behalf. REST and other REST-based adapters (such as Google Calendar
Adapter, Microsoft Office 365
Outlook Adapter, and others) often access APIs that are protected using OAuth. If these
calls result in a 403
error (forbidden), it usually indicates a lack of
permissions to access the API.
403
errors are as follows:
-
The access token was procured for a scope that does not cover the API being accessed.
-
During the phase to provide consent, the user that provided the consent is not the resource owner or owner of the API being accessed.
-
The access token expired or is not valid to access the resource any longer.
-
The authorization server revoked access privileges to a particular resource.
Troubleshoot Certificate Import Failures
The certificate that you receive from a Certificate Authority (CA) such as Verisign, Entrust, or others can fail during import into Oracle Integration with the following error in the log file:
java.security.cert.CertificateException: Unable to initialize,
java.io.IOException: extra data given to DerValue constructor
-
Obtain a fixed certificate from the CA.
-
Convert the certificate to a Distinguished Encoding Rules (DER) binary certificate using tools such as
openssl
. For example:openssl x509 -outform der -in sfdc-client.crt -out sfdc-client.der