3 Integration

This chapter describes the various scenarios which involve configuring the Process Orchestration and Monitoring (POM) application and integrating it with external systems.

Setting Up a New Batch Schedule in POM

When POM is first installed for a specific customer, it does not include any application batch schedules out of the box such as Merchandising or Retail Intelligence, and so on. An Oracle administrator or a system integrator need to first configure those schedules before they get loaded with the scheduling data. Configuring a new schedule entails setting up schedule properties such as the schedule name and description, and customer environment information for callbacks. It also entails setting up the location of different components and services with which different POM components need to interact to function properly.

Refer to the section "Configure New Schedule" in the "System Configuration" section of the POM User Guide.

Also refer to the Customizing Batch Schedules chapter of this guide for setting up a new custom batch schedule.

Invoking POM Entities

Different SaaS customers operate in different models for running their batch. Some may choose to use the POM Scheduler to schedule the different entities such as Nightly, Recurring or Standalone. Refer to the POM User Guide for documentation on the POM Scheduler.

Others may choose to control the time and frequency of batch executions by invoking the provided ReST APIs. See the Batch Execution API section of the Invoking POM Services chapter of this guide for more details.

External Status Update (Callbacks)

The External Status Update feature provides the ability for external systems to optionally register with POM to receive the Job status notifications as a callback to their ReST interface.

Note:

While ReST service calls from external systems (customers) to POM are required to use the OAuth2 authentication standard, ReST service calls from POM to external systems such as the call for External Status Update are limited to Basic Auth at this time.

POM has a health check feature that calls this ReST service to verify that the external status update (Callback) is up and running. This call uses the HTTP method OPTIONS or the HEAD method on the provided external status update URL. Customers need to make sure that their external status update API URL implements / allows the OPTIONS or HEAD method for the POM Health Check to work correctly.

Note:

Retailers are responsible for using valid, certificate authority (CA) signed certificates for TLS for the APIs, such as the one associated with External Status Update. For more information, see My Oracle Support Doc ID 2710163.1.

Figure 3-1 External Status Updates

Schedule Configuration

This section details the steps to configure the External Status Updates feature at the schedule level:

1. Navigate to the System Configuration screen.

2. Click the Edit icon on the External Configuration Panel to open the External Configuration window.

   Figure 3-2 External Configuration Window

   

3. Enter the configuration values:

   •  External Status URL - External system's URL that needs to be called for status updates and also for health-check.

     Note:

     In addition to this configuration, you must work with Oracle support to get the External Status URL added to the allowlist.

     The POM health check validates the external systems that must have the OPTIONS request method implemented. If the OPTIONS method is not implemented, POM will validate the external systems using the HEAD request method.

   • External Status Update Mode - Choose one of the options below:

     • ALL - POM will send a status update to the external system for each job's execution in the schedule regardless of success or failure. This option may be an overuse of this feature and may impact performance.

     • FAILED - POM will send a status update only for failed jobs.

     • NONE - No status updates will be sent by POM.

       Note:

       The External Status Update Mode defined on this screen applies to all the jobs in a schedule. If status update is desired only for specific jobs then set the mode on the above screen to NONE and follow the steps defined in the Job Configuration section below.

   • Click Update Credentials and provide the credentials for the external system.

Job Configuration

This section describes the steps to configure the External Status Update Mode at job level.

1. Navigate to the Batch Administration screen and select the desired schedule.

   Figure 3-3 Batch Administration Screen

   

2. Select a Cycle - Nightly/Recurring/Standalone

   • If Standalone is selected, select the Flow / Process from the left pane, then select the Process/Job combination and click Edit from table action menu to open the popup below.

   • If Nightly or Recurring is selected, then select the Process/Job combination row and click Edit from the table action menu to open the popup below.

   Figure 3-4 Edit Job Dialog

   

3. Set the External Status Update Mode to one of the following values:

   •  ALL - POM will send a status update to the external system for this job's execution regardless of success or failure.

   •  FAILED - POM will send a notification only when this job fails.

   •  NONE - No status update will be sent by POM for this job.

Payload Specification

Attribute	Description
processName	Name of the root process in a given cycle/flow Note: Process names in the callback response are prefixed with the name of the schedule. For instance, a callback response sent for Process "P1" would have processName attribute as MERCH_P1 on return.
processExecutionId	Unique identifier generated by POM to track the process executions.
activityName	Name of the job for which the callback/status update is sent.
activityExecutionId	Unique identifier generated by POM to track the job run instance.
callerId	Identifier provided by the caller to POM when submitting the invocation/execution request. POM returns the same ID back to the caller.
correlationId	Identifier provided by the caller to POM when submitting the invocation/execution request. POM returns the same ID back to the caller
callBackServiceDataDetail.<KeyName>	Key-value pairs supplied to POM when submitting the invocation/execution request. They are returned back to the caller
failedActivity	In the case where the callback is for a failed job, this field is populated with the details of the failed Job.
status	Status of the job execution: COMPLETED SKIPPED ERROR SKIPPED_ON_ERROR
activityStatus	Status of the job, and the derived activity state: ERROR - ACTIVITY_FAILED COMPLETED - ACTIVITY_COMPLETED SKIPPED - ACTIVITY_COMPLETED SKIPPED_ON_ERROR - ACTIVITY_COMPLETED

Payload Examples

Below are sample external status update payloads for the MERCH schedule.

Description	Payload
Hourly Job Callback	{ " callerId " : "XXX", " correlationId " : "37", " processName " : "MERCH_ BATCH_PROCESS _01", " processExecutionId " : "MERCH_ BATCH_PROCESS_01 ~asfasfdasdfas", " activityName " : " MERCH_ BATCH_JOB", " activityExecutionId " : "123456", " status " : "COMPLETED", " activityStatus " : "ACTIVITY_COMPLETED", " failedActivity " : null }
Nightly Job Callback	{ " callerId " : "XXX", " correlationId " : "37", " processName " : "MERCH_ START_NIGHT_BATCH_PROCESS ", " processExecutionId " : "MERCH_ START_NIGHT_BATCH_PROCESS ~asfasfdasdfas", " activityName " : " MERCH_ START_NIGHT_BATCH_JOB", " activityExecutionId " : "123456", " status " : "COMPLETED", " activityStatus " : "ACTIVITY_COMPLETED", " failedActivity " : null }

External Dependency

This feature allows customers to control the execution of a schedule running in POM by defining custom pre-dependencies on certain Jobs. POM pauses the schedule execution upon encountering these external pre-dependencies and resumes the execution once they are released by customer.

Note:

External Dependencies can only be created for Jobs on the Nightly Cycle. They cannot be created for Jobs that belong to the either the Hourly or Adhoc Cycles.

Figure 3-5 External Dependency

Configuration

This section details the steps involved in setting up the external dependency.

1. Navigate to the Batch Administration screen and select the schedule for which the external dependency is to be added.

   Figure 3-6 Batch Administration Screen

   

2. Select a Cycle - Nightly/Recurring/Standalone.

   • If Standalone is selected, select the Flow / Process from the left pane then select the Process/Job combination from the right pane to which the dependency needs to be added.

   • If Nightly or Recurring is selected, then select the Process/Job combination row from the table at the bottom of the screen.

3. Click on the Job name to open the Batch Job Details panel.

   Figure 3-7 Batch Job Details

   

4. In the External Dependency section of the screen, click the Add button to create the external dependency.

5. Provide the external job name.

6. Click Ok to save and exit or Ok and add another to create another dependency.

   Figure 3-8 Add External Dependency

   

Releasing Dependency

External systems need to invoke the corresponding ReST APIs to release/fulfill external dependencies. See the Releasing External Dependency section of the Invoking POM Services chapter for further details.

Bulk Data Integration Jobs

In POM, Jobs of type BDI, invoke BDI Processes on the BDI Process Flow component.

The specifications followed for the parameters to such Jobs is:

[bdi-process-name] [bdi-extractor-name] [callback-params] [filter-param]

For example, the parameters to the BDI_RPAS_Store_Fnd_PF_From_RMS_JOB on the MERCH schedule are:

Store_Fnd_FileCreator_ProcessFlow_From_RMS Store_Fnd_Extractor #CallbackCtxt.CallerId___#CallbackCtxt.CorrelationId___#JobCtxt.rootProcessName___#JobCtxt.rootProcessExecId___#JobCtxt.jobRunId___#JobCtxt.processName filter=RmsDBDS:RMS_BATCH_STATUS_SQL.GET_EOW_RUN_SIGNAL

Parameter Name	Description	Example
bdi-process-name	Name of the BDI Process	Store_Fnd_FileCreator_ProcessFlow
bdi-extractor-name	Name of the Extractor Activity on the BDI Process	Store_Fnd_Extractor
callback-params	Parameters that need to be propagated to BDI, so that they can be set in the Callback, if configured.	#CallbackCtxt.CallerId___#CallbackCtxt.CorrelationId___#JobCtxt.rootProcessName___#JobCtxt.rootProcessExecId___#JobCtxt.jobRunId___#JobCtxt.processName
filter-param	This is an optional parameter. The presence of this parameter indicates that this Job must run only if it’s the end of week. POM will invoke the MERCH “End-Of-Week” endpoint, and only if it returns true, will the Job actually run; otherwise the Job will be Skipped.	filter=RmsDBDS:RMS_BATCH_STATUS_SQL.GET_EOW_RUN_SIGNAL

The integration between POM and BDI is controlled by the parameters passed to the Job, and can be broadly classified into the following three modes:

Mode 1 : Fire-And-Wait

In this mode, the POM Job will trigger the BDI Flow and will proceed to wait for the main BDI Process and its sub-processes to complete. Any error in either the main BDI Process or its Sub-Processes will cause the POM Job to be marked Error.

Set Mode

This mode is set by ensuring that no “extractor” names (second parameter) are passed as parameters to the POM Job that is triggering the BDI Flow.

Implications

The impact of running Jobs in this mode are:

• The status of the POM Job reflects the status of the BDI Flow. Guarantees the completion of the BDI Flow before moving on to the next Job in POM.

• Target Batch Duration can be impacted if the BDI Process takes a long time to complete.

Mode 2 : Fire-And-Forget

In this mode, POM Job will trigger the BDI Flow and will only wait for the specified extractor activities of the main BDI Process to complete. If the extractor activities are completed, the Job is assumed to be completed in POM.

Set Mode

This mode is set by ensuring that “extractor” names (second parameter) are passed as parameters to the POM Job that is triggering the BDI Flow.

Implications

The impact of running Jobs in this mode are

• It is possible that the BDI Flow eventually fails (after the Extractors have completed), while the POM Job is marked Completed. This could possibly impact the integrity of the data.

• It doesn’t cause the remaining batch schedule to wait for a BDI Flow to complete. Non-dependent Jobs can continue processing while the BDI Flow continues to run separately.

Mode 3 : Fire-And-Wait-Later

This is essentially an improved version of the previous mode. In this mode too, the POM BDI job waits only for the extractor activities to complete, to be marked Complete itself. However, a later Job can be configured to wait for the BDI subprocess to complete by configuring an external dependency and routing the callback from BDI to POM.

The external dependency must be configured as follows:

 {Name of Subprocess} # {end}

Set Mode

To set this mode, the configuration needed is as follows:

1. Configure the POM BDI Callback URL on the BDI Process Flow. This ensures that BDI will call POM on the completion of the specified activity.

2. Configure an External Dependency in POM, based on the name of the BDI Subprocess. This external dependency doesn’t need to be manually released. On receiving a callback from the BDI Process Flow, POM will auto-release this external dependency. This ensures that the POM Flow is not moved to completion until the relevant BDI Flows have also been completed successfully.

Retail Data Store Integration

POM provides support for RDS jobs out-of-the-box. Consuming Applications are not needed to configure the RDS Job Type, as it would be pre-defined within POM and ready to use.

For every environment, there is a single instance of RDS that can be configured to work with an instance of POM. This information is configured in the System Settings panel of the System Configuration screen.

Important Notes

• The RDS Job Type currently supports synchronous jobs. This means that POM will invoke the RDS service and wait for the response. A successful response indicates that the Job has completed and POM marks the job completed.

   

• Because a job of RDS job type runs synchronously, there is a limit on how long it can run. The limit is configurable on the System tab of the System Configuration screen. The limit is also bound by the Load Balancer max configuration for ReST endpoints, which at this time is five minutes. Therefore, the maximum runtime for a job of RDS job type is five minutes.

• If an asynchronous approach is needed, it is advised to build a Custom Job Type and not use the default RDS Job type itself.

For a complete set of instruction on setting up a POM job with an RDS job type, refer to the “Notification-Based Monitoring” section of the “Extension” chapter of the Oracle Retail Data Store Implementation Guide.