Batch Concepts

Batch Job

A Job is representative of a program that is meant to run and is an atomic unit of work that needs to be accomplished.

Each Job in POM is assigned to an Application. Applications allow POM to define rules for groups of Jobs. For example, throttling rules, which control how many jobs can run concurrently, are defined at an Application level. Applications are also directly mapped to Job Agents, so Jobs of a particular Application are only run on the specified Job Agent.

Warning Codes and Error Codes

Shell-script-based jobs (jobs of type EXEC), in addition to the ERROR or COMPLETED status, can also move to a COMPLETED with Warning status. This status essentially indicates that even though the Job has completed, there may be certain corrective actions for Oracle Operations or the Customer to take. A notification of type JobCompletedWithWarning will also be created. Batch Schedules accomplish this by defining which exit codes are to be used as warning codes in the System Options tab.

Similarly, error codes can be defined for shell-script-based jobs to provide a better description on the error itself.

Format

Key = WarningCodeXXX or ErrorCodeXXX ( where XXX is the exit code).

Value = Description of the warning or error.

Auto-retry

For ReST-based jobs, there is a way to have POM auto-retry the Job (up to 3 times) in cases of a known error. To do so, the error messages provided at the time of error are compared with a known list of restartable error messages. If the error exists in this list, then the Job will be retried by POM, without any manual intervention. This list of restartable error messages is defined for each Job type (ReST-based) as a System Option.

Format

Key = XXXRestartableErrorMessages (where XXX is the Job Type)

Value = A list of comma-separated error messages. If the response contains one of those error messages, POM will retry the job for a maximum of 3 attempts.

Every Job is associated with a Job Type that defines the kind of work performed by that Job.

The following Job Types are supported by POM out-of-the-box.

Note:

Custom job types can be created using the Job Type feature of the System Configuration screen. See Generic ReST Jobs for more information.

Job Type	Description
EXEC	Refers to shell-script based Jobs. POM is expected to start a shell script and track its execution. This is the default type of Job. If a Job type has not been specified on the batch schedule spreadsheet, it is assumed to be of this type.
BDI (Bulk Data Integration)	Indicative of BDI Jobs. POM invokes a BDI Process and tracks its execution for this type of Jobs.
RI (Retail Insights)	Refers to DIS (Data Intelligence Services) Jobs for the RI Schedule. POM invokes RI endpoints and tracks their execution.
RASE (Retail Advanced Science Engine)	Refers to DIS (Data Intelligence Services) Jobs for the RSP Schedule. The wrapper endpoints invoked here are the same as the ones used by the RI job type.
RPAS (Retail Predictive Application Server)	Indicative of RPAS-WebService Jobs for the RPAS schedule. POM invokes RPAS endpoints to start and track RPAS batches.
OB (Order Broker)	Indicative of OB-WebService Jobs for the OB schedule. POM invokes OB endpoints to start and track OB batches.
OMS (Order Management System)	Indicative of OMS-WebService Jobs for the OMS schedule. POM invokes OMS endpoints to start and track OMS batches.
DUMMY	Mock Job Type that doesn’t call any executable. It is used for Jobs that are meant to be used as placeholders. Such Jobs are automatically skipped by the Execution Engine.
RDS (Retail Data Services)	Indicative of RDS Jobs. POM invokes RDS endpoints to start and track these Jobs.

Batch Process

A Batch Process is a collection of related Batch Jobs that will always run in sequential order. The dependencies between the Jobs within a Process ensure that these Jobs cannot be run in parallel.

Batch Processes on the Adhoc Cycle can be run from the POM UI or invoked using the ReST API. See Invoking the POM Service in the Integration chapter of this guide.

Batch Flow

A Batch Flow is a collection of related Batch Processes that must be run as a single unit. Within a Batch Flow one Process must be designated as the "first" Process, and another as the "last" Process. Other Processes can be run between them in sequential or parallel fashion, based on the dependencies setup.

Batch Flows can be defined on the Hourly Cycles and the Adhoc Cycle. The Flows on the Hourly Cycle can only be run once, while the Flows on the Adhoc Cycle can be run any number of times. These Flows can be invoked from the POM UI and can also be invoked through the ReST API. See Invoking the POM Service in the Integration chapter of this guide.

Batch Cycle

A Batch Cycle is the container in which Flows, Processes and Jobs are setup. There are three types of Batch Cycles.

Adhoc / Standalone Cycle

The Flows and Processes defined in this Cycle can run multiple times a day on an as-needed basis. Such invocations can run independent of other Processes and Flows on other Cycles.

Example: A Flow can be defined whose main responsibility is to purge data in database tables to boost performance. This Flow can then be run every 15 minutes without impacting Flows or Processes on other Cycles.

Recurring / Hourly Cycles

Recurring or Hourly Cycles are specialized Batch Cycles with the following properties

A Batch Schedule can have a maximum of 24 Hourly Cycles.
The Flows and Processes defined on an Hourly Cycle are replicated across all the defined Hourly Cycles. Dependencies are built between these batch entities, to ensure that unless it is completed in a prior cycle, it will not run in the next cycle.
All the Hourly Cycles are meant to be run prior to running the Nightly Flow. Once the Nightly Flow starts, all the Jobs on the Hourly Cycles are marked completed.

Example: The Sales processing Jobs in RMS support trickle processing by running every 30 minutes during the store trading hours. The schedule is pre-loaded with 24 Hourly Cycles. The Hourly Cycles are time triggered from the Scheduler or through ReST APIs externally. Based on the client's business operations, the individual flows of each cycle need to be scheduled.

Note:

If there are any errors on the Hourly Cycles, then the Nightly Cycle will not start until they are resolved.

Nightly Cycle

This Batch Cycle contains the set of Jobs that are executed at end of the business day. All of these Processes and Jobs belong to the default Nightly Flow of this Cycle. Once the Nightly Flow has started, all the loaded Flows and Processes on the Hourly Cycles are marked complete.

The Nightly Flow can be time-triggered or triggered using Schedule Links. It can be time-triggered through the POM Scheduler or externally through the ReST API.

Batch Dependencies

A dependency is a construct that prevents the execution of a Job, until a condition is satisfied.

Type	Description
Internal Dependency	Internal dependencies are the dependencies between Jobs of the same Batch Schedule.
Inter-schedule Dependency	Inter-Schedule dependencies are the dependencies between Jobs of different schedules running on the same POM instance.
External Dependency	External dependencies are dependencies between processes running on external systems, such as a Customer's system and Jobs running on the POM schedule.
Custom Dependency	Dependency between Nightly jobs and Adhoc Flows/Processes. A Custom Dependency can be created on the Job Detail screen in the Nightly cycle through the Batch Administration screen.

Batch Schedule Links

Schedule Links or Execution Links, allow a Job from one Schedule, to trigger the Nightly Cycle of a different Schedule.

Batch Metadata

All information from the Batch Schedule spreadsheet is referred to as the Batch metadata. This information is modifiable from the POM UI on the Batch Administration Screen.

Batch Scheduler Day

Simply stated, this is a single copy of the Batch metadata or an instance of the Batch Schedule that is associated with a business date. All executions of batches for that business date are recorded on that Scheduler Day or Schedule Instance. Without a Scheduler Day, no Flows or Processes can be run.

A Batch Scheduler Day is marked Closed at the completion of all the Batch Jobs of the Nightly Cycle. The business date is then incremented by one, and a new Scheduler Day is created for the next day.

If however, there are Adhoc Flows or Processes executing when the Nightly Cycle completes, the closure of the Scheduler Day is then deferred until the completion of the running batches.