Common Integration Style Pitfalls and Design Best Practices
Note the following best practices and integration style pitfalls to avoid when designing an integration.
- Avoid Common Integration Style Pitfalls
- Avoid Creating Too Many Scheduled Integrations
- Synchronous Integration Best Practices
- Design Long-Running or Time-Consuming Integrations as Asynchronous Flows
- Time Outs in Service Calls During Synchronous Invocations
- Parallel Processing in Outbound Integrations
Avoid Common Integration Style Pitfalls
Designing integrations correctly from the start can save you a tremendous amount of rework. This section describes common integration style pitfalls (known as antipatterns) and best practices for avoiding these pitfalls.
- Chatty Integrations
- Scheduled Job that Never Stops Trying to Process
- lmport an Externally Updated IAR File
- Synchronous Integration Doing Too Much
- Too Many Connections in an Integration
- Read Files with Many Records'
- Integrations Running Unchanged Despite Changing Business Needs
Chatty Integrations
Use Case: Synchronize records in a file or large data set with an external system (for example, synchronizing journal transactions or uploading employee records into Oracle HCM Cloud).
Antipattern | Why an Antipattern? | Best Practice |
---|---|---|
Use an invoke activity within a looping construct to call external APIs for every record. |
|
|
Scheduled Job that Never Stops Trying to Process
Use Case: Process records within a set of files with a tight service level agreement (SLA) (for example, synchronize employee records into Active Directory from Oracle HCM Cloud or send benefits information).
Antipattern | Why an Antipattern? | Best Practice |
---|---|---|
The scheduled integration looks for all files to process and loops over all to sequentially process until no files remain. |
|
|
Import an Externally Updated IAR File
Use Case: Need to leverage advanced XSL constructs that may not be available in the mapper.
Antipattern | Why an Antipattern? | Best Practice |
---|---|---|
Updating the IAR file externally and then importing it into Oracle Integration. |
|
|
Synchronous Integration Doing Too Much
Use Case: A request triggers complex processing involving enrichment and updates across multiple systems.
Antipattern | Why an Antipattern? | Best Practice |
---|---|---|
|
|
|
Too Many Connections in an Integration
Use Case: As developers create integrations, they define their own connections pointing to the same application. This leads to many duplicate connections.
Antipattern | Why an Antipattern? | Best Practice |
---|---|---|
Every developer creates their own connection using a different set of configurations/credentials. |
|
|
Read Files with Many Records
Use Case: Read a file with a large number of records and process individual records.
Antipattern | Why an Antipattern? | Best Practice |
---|---|---|
Reading the entire file in memory using the read file option and processing record by record. |
|
|
Integrations Running Unchanged Despite Changing Business Needs
Use Case: Integrations/schedules created during the initial implementation continue to run even though your business requirements have changed over time.
Antipattern | Why an Antipattern? | Best Practice |
---|---|---|
|
|
|
Avoid Creating Too Many Scheduled Integrations
When there are too many scheduled integrations configured, instances can get backlogged waiting for resources to become available or previous integration runs to complete. This can cause processing delays where some instances are in a waiting state longer than they should be and schedules may not start at the scheduled time.
- Use an asynchronous REST Adapter trigger instead of a scheduled trigger when an active schedule is not absolutely required.
- Do not create any long-running scheduled integrations (a scheduled integration that takes longer than one hour, for example, to complete). This blocks scheduler resources impacting other scheduled runs.
- Spread schedules out over time to avoid schedule clusters.
You can convert a scheduled integration to an application-driven orchestrated, REST Adapter trigger-based integration. See Convert a Scheduled Integration to a REST Adapter-Triggered Orchestration Integration.
- For each scheduled integration, convert it to an application-driven orchestrated, REST Adapter trigger-based integration. See Convert a Scheduled Integration to a REST Adapter-Triggered Orchestration Integration.
- Create a new scheduled integration that only performs an
asynchronous invoke of the application-driven orchestrated integration you
converted in step 1 above.
This solution enables the scheduled integration to start at the scheduled time, invoke the REST Adapter trigger-based child integration asynchronously, and complete within milliseconds. This approach reduces the backlog and contention surrounding scheduler resources.
If you have a large number of scheduled integrations to convert, a staged approach starting with the following integrations is recommended.- Longest-running scheduled integrations.
- Scheduled integrations configured with the shortest frequency (for example, integrations that run every 10 minutes or less).
Design any new scheduled integrations with the design practices described above.
See Scheduled Integrations Are Not Executing on Time and When a Scheduled Integration Instance Gets Terminated.
Synchronous Integration Best Practices
Note the following best practices when designing a synchronous integration.
-
A synchronous integration calling any asynchronous request response service:
-
Calling asynchronous fire and forget (one-way) is acceptable.
-
Oracle Integration does not currently allow modeling an asynchronous request response service. However, all scheduled orchestration styles internally use an asynchronous request response. Therefore, a synchronous integration using a scheduled orchestration is an antipattern.
-
-
A synchronous integration calling multiple services that span more than 5 minutes is reported as a stuck thread in Oracle WebLogic Server.
Design Long-Running or Time-Consuming Integrations as Asynchronous Flows
Note the following best practice when designing long-running or time-consuming integrations.
Do not expose long-running or time-consuming integrations as synchronous flows. This action can lead to client applications (including other integrations) timing out. Synchronous integrations also have a server-side time out. Instead, model any synchronous integration taking more than two minutes as an asynchronous flow.
Time Outs in Service Calls During Synchronous Invocations
You may have scenarios in which synchronous invocations from Oracle Integration (including calls to other integrations) are blocking calls and must complete within 300 seconds.
Because the call may involve one or more proxies, each of the proxies may have a similar time out. For instance, the default proxy on Oracle Public Cloud has a time out value of 120 seconds. If the call is to an on-premises service behind a fire wall, the proxy configured may also have its own time out value.
In case time outs are defined at multiple layers, the service invocation fails at the first time out.
Parallel Processing in Outbound Integrations
While there is no specific integration design to automatically enable an outbound integration to send data to different third party systems in parallel, there are integration design approaches that allow this scenario.
-
Create a main parent integration that only receives/processes the data.
-
Create separate child integrations to perform the individual outbound REST invocations.
The interface between the main and separate child integrations can follow these approaches:
-
Consist of dummy REST calls, but it must be asynchronous. Essentially, the asynchronous calls are not blocked by the response and the fire-and-forget design enables available threads to work on child integration processing in parallel, within the available system resources. This type of design is recommended because if all synchronous REST calls are done in the same integration, a time out error may occur if the sum of time taken for each synchronous call exceeds five minutes.
-
Follow a publish/subscribe design approach (for example, putting the data events in a queue, having each child flow subscribe from the queue, and so on).