4 Integration
This chapter describes the various scenarios which involve configuring the Process Orchestration and Monitoring (POM) application and integrating it with external systems.
Setting Up a New Batch Schedule in POM
When POM is first installed for a specific customer, it does not include any application batch schedules out of the box such as Merchandising or Retail Intelligence, and so on. An Oracle administrator or a system integrator need to first configure those schedules before they get loaded with the scheduling data. Configuring a new schedule entails setting up schedule properties such as the schedule name and description, and customer environment information for callbacks. It also entails setting up the location of different components and services with which different POM components need to interact to function properly.
Refer to the section "Configure New Schedule" in the "System Configuration" section of the POM User Guide.
Invoking POM Entities
Different SaaS customers operate in different models for running their batch. Some may choose to use the POM Scheduler to schedule the different entities such as Nightly, Recurring or Standalone. Refer to the POM User Guide for documentation on the POM Scheduler.
Others may choose to control the time and frequency of batch executions by invoking the provided ReST APIs. See the Batch Execution API section of the Invoking POM Services chapter of this guide for more details.
External Status Update (Callbacks)
The External Status Update feature provides the ability for external systems to register with POM to receive the Job status notifications as a callback to their ReST interface.
Note:
While ReST service calls from external systems (customers) to POM are required to use the OAuth2 authentication standard, ReST service calls from POM to external systems such as the call for External Status Update are limited to Basic Auth at this time.Figure 4-1 External Status Updates
Schedule Configuration
This section details the steps to configure the External Status Updates feature at the schedule level:
-
Navigate to the System Configuration screen.
-
Click the Edit icon on the External Configuration Panel to open the External Configuration window.
Figure 4-2 External Configuration Window
-
Enter the configuration values:
-
External Status URL - External system's URL that needs to be called for status updates.
Note:
In addition to this configuration, you must work with Oracle support to get the External Status URL added to the allowlist. -
External Status Update Mode - Choose one of the options below:
-
ALL - POM will send a status update to the external system for each job's execution in the schedule regardless of success or failure. This option may be an overuse of this feature and may impact performance.
-
FAILED - POM will send a status update only for failed jobs.
-
NONE - No status updates will be sent by POM.
Note:
The External Status Update Mode defined on this screen applies to all the jobs in a schedule. If status update is desired only for specific jobs then set the mode on the above screen to NONE and follow the steps defined in the Job Configuration section below.
-
-
Click Update Credentials and provide the credentials for the external system.
-
Job Configuration
This section describes the steps to configure the External Status Update Mode at job level.
-
Navigate to the Batch Administration screen and select the desired schedule.
Figure 4-3 Batch Administration Screen
-
Select a Cycle - Nightly/Recurring/Standalone
-
If Standalone is selected, select the Flow / Process from the left pane, then select the Process/Job combination and click Edit from table action menu to open the popup below.
-
If Nightly or Recurring is selected, then select the Process/Job combination row and click Edit from the table action menu to open the popup below.
Figure 4-4 Edit Job Dialog
-
-
Set the External Status Update Mode to one of the following values:
-
ALL - POM will send a status update to the external system for this job's execution regardless of success or failure.
-
FAILED - POM will send a notification only when this job fails.
-
NONE - No status update will be sent by POM for this job.
-
Payload Specification
Attribute | Description |
---|---|
processName |
Name of the root process in a given cycle/flow Note: Process names in the callback
response are prefixed with the name of the schedule. For instance, a callback response sent for Process "P1" would have |
processExecutionId |
Unique identifier generated by POM to track the process executions. |
activityName |
Name of the job for which the callback/status update is sent. |
activityExecutionId |
Unique identifier generated by POM to track the job run instance. |
callerId |
Identifier provided by the caller to POM when submitting the invocation/execution request. POM returns the same ID back to the caller. |
correlationId |
Identifier provided by the caller to POM when submitting the invocation/execution request. POM returns the same ID back to the caller |
callBackServiceDataDetail.<KeyName> |
Key-value pairs supplied to POM when submitting the invocation/execution request. They are returned back to the caller |
failedActivity |
In the case where the callback is for a failed job, this field is populated with the details of the failed Job. |
status |
Status of the job execution:
|
activityStatus |
Status of the job, and the derived activity state:
|
Payload Examples
Below are sample external status update payloads for the MERCH
schedule.
Description | Payload |
---|---|
Hourly Job Callback |
|
Nightly Job Callback |
|
External Dependency
This feature allows customers to control the execution of a schedule running in POM by defining custom pre-dependencies on certain Jobs. POM pauses the schedule execution upon encountering these external pre-dependencies and resumes the execution once they are released by customer.
Note:
External Dependencies can only be created for Jobs on the Nightly Cycle. They cannot be created for Jobs that belong to the either the Hourly or Adhoc Cycles.Figure 4-5 External Dependency
Configuration
This section details the steps involved in setting up the external dependency.
-
Navigate to the Batch Administration screen and select the schedule for which the external dependency is to be added.
Figure 4-6 Batch Administration Screen
-
Select a Cycle - Nightly/Recurring/Standalone.
-
If Standalone is selected, select the Flow / Process from the left pane then select the Process/Job combination from the right pane to which the dependency needs to be added.
-
If Nightly or Recurring is selected, then select the Process/Job combination row from the table at the bottom of the screen.
-
-
Click on the Job name to open the Batch Job Details panel.
Figure 4-7 Batch Job Details
-
In the External Dependency section of the screen, click the Add button to create the external dependency.
-
Provide the external job name.
-
Click Ok to save and exit or Ok and add another to create another dependency.
Figure 4-8 Add External Dependency
Releasing Dependency
External systems need to invoke the corresponding ReST APIs to release/fulfill external dependencies. See the Releasing External Dependency section of the Invoking POM Services chapter for further details.
Bulk Data Integration Jobs
In POM, Jobs of type BDI, invoke BDI Processes on the BDI Process Flow component.
The specifications followed for the parameters to such Jobs is:
[bdi-process-name] [bdi-extractor-name] [callback-params] [filter-param]
For example, the parameters to the BDI_RPAS_Store_Fnd_PF_From_RMS_JOB
on the MERCH
schedule
are:
Store_Fnd_FileCreator_ProcessFlow_From_RMS Store_Fnd_Extractor #CallbackCtxt.CallerId___#CallbackCtxt.CorrelationId___#JobCtxt.rootProcessName___#JobCtxt.rootProcessExecId___#JobCtxt.jobRunId___#JobCtxt.processName filter=RmsDBDS:RMS_BATCH_STATUS_SQL.GET_EOW_RUN_SIGNAL
Parameter Name | Description | Example |
---|---|---|
|
Name of the BDI Process |
|
|
Name of the Extractor Activity on the BDI Process |
|
|
Parameters that need to be propagated to BDI, so that they can be set in the Callback, if configured. |
|
|
This is an optional parameter. The presence of this parameter indicates that this Job must run only if it’s the end of week. POM will invoke the MERCH “End-Of-Week” endpoint, and only if it returns true, will the Job actually run; otherwise the Job will be Skipped. |
|
The integration between POM and BDI is controlled by the parameters passed to the Job, and can be broadly classified into the following three modes:
Mode 1 : Fire-And-Wait
In this mode, the POM Job will trigger the BDI Flow and will proceed to wait for the main BDI Process and its sub-processes to complete. Any error in either the main BDI Process or its Sub-Processes will cause the POM Job to be marked Error.
Mode 2 : Fire-And-Forget
In this mode, POM Job will trigger the BDI Flow and will only wait for the specified extractor activities of the main BDI Process to complete. If the extractor activities are completed, the Job is assumed to be completed in POM.
Set Mode
This mode is set by ensuring that “extractor” names (second parameter) are passed as parameters to the POM Job that is triggering the BDI Flow.
Implications
The impact of running Jobs in this mode are
-
It is possible that the BDI Flow eventually fails (after the Extractors have completed), while the POM Job is marked Completed. This could possibly impact the integrity of the data.
-
It doesn’t cause the remaining batch schedule to wait for a BDI Flow to complete. Non-dependent Jobs can continue processing while the BDI Flow continues to run separately.
Mode 3 : Fire-And-Wait-Later
This is essentially an improved version of the previous mode. In this mode too, the POM BDI job waits only for the extractor activities to complete, to be marked Complete itself. However, a later Job can be configured to wait for the BDI subprocess to complete by configuring an external dependency and routing the callback from BDI to POM.
The external dependency must be configured as follows:
{Name of Subprocess} # {end}
Set Mode
To set this mode, the configuration needed is as follows:
-
Configure the POM BDI Callback URL on the BDI Process Flow. This ensures that BDI will call POM on the completion of the specified activity.
-
Configure an External Dependency in POM, based on the name of the BDI Subprocess. This external dependency doesn’t need to be manually released. On receiving a callback from the BDI Process Flow, POM will auto-release this external dependency. This ensures that the POM Flow is not moved to completion until the relevant BDI Flows have also been completed successfully.