12 Defining and Using Job Sets

This chapter describes how to define and submit an Oracle Enterprise Scheduler job set, a collection of job definitions that can be grouped together to run as a single unit.

This chapter includes the following sections:

Section 12.1, "Introduction to Defining and Using Job Sets"
Section 12.2, "Defining Job Sets"
Section 12.3, "Cross Application Job Sets"
Section 12.4, "Using Input and Output Forwarding"

12.1 Introduction to Defining and Using Job Sets

Oracle Enterprise Scheduler provides for collections of job definitions that can be grouped together to run as a single unit called a job set. A job set may be nested; thus a job set may contain a collection of job definitions or one or more child job sets. Each job definition or job set included within a job set is called a job set step.

A job set is defined as either a serial job set or a parallel job set. At runtime, Oracle Enterprise Scheduler runs parallel job set steps together, in parallel. When a serial job set runs, Oracle Enterprise Scheduler runs the steps one after another in a specific sequence. Using a serial job set Oracle Enterprise Scheduler supports conditional branching between steps based on the execution status of a previous step.

You can define a serial job set to include a parallel job set, or a parallel job set to include a serial job set. job sets that include a mix of parallel and serial job sets are called complex job sets. For example, when a serial job set contains a child parallel job set, the serial job set runs serially until it reaches the child parallel job set. Then, all the job definitions or job set definitions in the child parallel job set run in parallel. Upon completion of the child parallel job set the serial job set continues running its remaining steps serially. Nested parallel job sets behave the same as non-nested parallel job sets.

For every step in a job set Oracle Enterprise Scheduler supports properties that provide runtime flexibility for how a particular step affects the entire job set. These properties are defined on a per step basis. Table 12-1 shows properties that are useful for job set steps. Any property can be defined on a job set step.

Table 12-1 Job Set Step Properties

Property Description

Property	Description
`EFFECTIVE_APPLICATION`	Specifies if the step is a job, the job will execute in the effective application. If the step is a nested job set, the jobs in the nested job set will execute in the effective application. The effective application becomes the application for the request for the step and for any child requests of the step. This property can be defined for job definitions and job types as well as job sets.
`SELECT_STATE`	Specifies whether the result state of a job set step should be included when determining the state of the job set. Specifies whether the execution state of the step affects the eventual state of entire job set. By default, all job set steps affect the job set state. To prevent the state of a particular job set step from affecting the state of the job set, set `SELECT_STATE` to false for that step. To allow the state of a job set step to affect the overall state of the job set, set SELECT_STATE to true for that step.

EFFECTIVE_APPLICATION

Specifies if the step is a job, the job will execute in the effective application. If the step is a nested job set, the jobs in the nested job set will execute in the effective application. The effective application becomes the application for the request for the step and for any child requests of the step.

This property can be defined for job definitions and job types as well as job sets.

SELECT_STATE

Specifies whether the result state of a job set step should be included when determining the state of the job set. Specifies whether the execution state of the step affects the eventual state of entire job set.

By default, all job set steps affect the job set state. To prevent the state of a particular job set step from affecting the state of the job set, set SELECT_STATE to false for that step. To allow the state of a job set step to affect the overall state of the job set, set SELECT_STATE to true for that step.

Oracle Enterprise Scheduler provides the capability for a job set to execute across multiple applications. A job set runs in its hosting application and by default all job set steps also run in this application. A job set step can be associated with a different application by defining the EFFECTIVE_APPLICATION system property on the step. If the step is a job definition, the job definition executes in the effective application. If the step is a nested job set definition, the job definitions or job set definitions in the nested job set execute in the effective application. The effective application becomes the application for the request for the step and for any child requests of the step. For more information, see Section 12.3, "Cross Application Job Sets".

12.2 Defining Job Sets

You can define a job set in Oracle JDeveloper by specifying the following:

The name, package, and description for the job set
The application defined properties for the job set
The system properties for the job set
Specifying the job set steps

The contents of a job set are specified when you define the job set steps. For example, for a serial job set you specify the name and the execution mode and then you add the job set steps to define the sequence of job definitions or child job sets that run when the job set runs.

12.2.1 How to Define a Job Set

An Oracle Enterprise Scheduler job set is defined by a name, a package, a job set execution mode, step definitions, application defined properties, and system properties.

To create a job set:

In Oracle JDeveloper, right-click in the project to view the New Gallery.
In the All Technologies tab, under Categories, expand Business Tier and select Enterprise Scheduler Metadata.
Under Items, select Job Set and click OK. This displays the Create Job Set window.
In the Create Job Set window, specify the following:
1. In the Name field, enter a name for the job set or accept the default name.
2. In the Package field, optionally enter a package name for the job set.
3. The Location field displays the full path of the directory where the job set file is stored.
4. Click OK. This creates the job set and displays the Job Set Definition page, as shown in Figure 12-1.
  
  Figure 12-1 Job Set Editor with Serial Job Set
In the Job Set Editor pane, in the Description field enter a description for the job set.
In the Job Set Steps area, select the Parallel or Serial radio button to specify parallel or serial execution mode for the job set.
In the Job Set Editor pane add the job set steps. For more information on adding job set steps, see Section 12.2.2, "How to Define Serial Job Set Steps" or Section 12.2.3, "How to Define Parallel Job Set Steps".
In the Application Defined Properties area, click Add to add properties associated with the job set. You use these to represent an application-specific or step-specific application defined property for the job set. For more information on using application defined properties, see Section 7.1, "Introduction to Using Parameters and System Properties". For more information, see Section 7.1.2.2, "What You Need to Know About Job Set Level Parameter Materialization".
In the System Properties area, click Add to add system properties associated with the job set. For more information on using system properties, see Section 7.4, "Using System Properties".
In the Access Control area, click Add to modify the list of roles that have access to this metadata, along with their access levels. For more information on defining access, see Section 18.2.3, "How to Create Grants with Oracle Enterprise Scheduler Metadata Pages".
In the Localization area, enter the following information for localizing this job set:
- Resource Bundle Base Name -- The base name for the resource bundle that specifies internationalized values.
- Display Name Resource Key -- The resource key that specifies the display name value in the resource bundle.
- Description Resource Key -- The resource key that specifies the description text in the resource bundle.
Save the job set.

12.2.2 How to Define Serial Job Set Steps

To define serial job set steps you select the serial execution mode and then add job set steps. Job set steps are created from the available job definitions and job sets defined in the current project. You define serial job set steps when you specify a step ID and a job definition child job set definition associated with the step. You also define links from a job set step terminal states to specify the next step. Table 12-2 lists the possible terminal states that you can specify using JDeveloper.

Table 12-2 Job Set Serial Execution Step Terminal States

Terminal State	Description
`SUCCEEDED`	Oracle JDeveloper indicates this state with a checkmark icon. This path represents a child step or child job set was successfully processed by the system.
`WARNING`	Oracle JDeveloper indicates this step with a warning icon. A child step or child job set resulted in a warning.
`ERROR`	Oracle JDeveloper indicates this step with an error icon. Some aspect of the request to run the child step or child job set processing resulted in an error.

To add serial job set steps:

First, define the appropriate job definitions or job sets and define the parent job set to contain the steps.
In the Job Set Editor pane, in the Job Set Steps area, select Serial execution mode.
Click the Add icon to add a job set step. This displays the Add Step window.
In the Step ID field, enter the step ID. For example, enter step1.
In the Job field, from the dropdown list select a job definition or a job set to associate with the step. For example, select Job1.
If you need to define step level application defined properties, then select the Application Defined Properties tab and add properties for the step.
If you need to define step level system properties, then select the System Properties tab and add job set step system properties for the step.
Select a destination for the step. The step can be added as part of the job set by selecting Insert into main diagram. To make the step available for use in another step, for either error or warning states, select Add to list of available steps.
Click OK, this adds the job set step, as shown in Figure 12-2.

Figure 12-2 Job Set with a Step Added
From the dropdown list next to the error icon, select Stop or select the step for the ERROR terminal state for the step. For example, from the dropdown list select Step_error (Step_error needs to be defined).
From the dropdown list next to the warning icon, select Stop or select the step for the WARNING terminal state for the step. For example, from the dropdown list select Step_warning (Step_warning needs to be defined).
Click the Add icon and add additional steps as needed.
Click OK, as shown in Figure 12-3.

Figure 12-3 Job Set with Two Steps Added

12.2.3 How to Define Parallel Job Set Steps

You can add parallel job set steps to a job set.

To add parallel job set steps:

First, define the appropriate job definitions and job set definitions and the parent job set.
In the Job Set Editor, select the Parallel execution mode.
Click the Add icon to add a job set step to the job set.

The Add Step window displays.
In the Job field, select a job definition or a job set.
If you need to define step level application defined properties, then select the Application Defined Properties tab and add properties for the step.
If you need to define step level system properties, then select the System Properties tab and add job set step system properties for the step.
Click OK, this adds the job set step.
Click the Add icon.
In the Add Step dialog, select the job set or job definition to use for next job in the parallel job set.
Click OK. The job set step displays in the job set, as shown in Figure 12-4.

Figure 12-4 Adding Job Set Steps to a Parallel Job Set

12.2.4 What Happens When You Define a Job Set

When you define a job set with Oracle JDeveloper, Oracle JDeveloper creates an XML file containing elements that represent the steps that you define.

When you define a parallel job set you specify a set of job set steps that run together. A parallel job set only contains steps, and does not contain links between steps, as all the steps execute together and do not depend on each other or upon the order in which each step runs.

When you define a job set Oracle JDeveloper creates an XML document that conforms to the Oracle Enterprise Scheduler job step schema.

12.2.5 What You Need to Know About Serial Job Sets

When you define a serial job set, the associated XML document includes job set steps and links. Oracle Enterprise Scheduler enforces the following limitations for serial job set definitions:

To prevent looping within a job set, job set definitions should not contain circular execution paths. A circular execution path, or a loop, is defined at the job set level as follows: loop is a path from one job set step along the links of any number of other steps back to the same job set step. For example, in a job set with a flow from Job_A, to Job_B, to Job_C defined, Oracle Enterprise Scheduler does not allow you to define an execution path from Job_B or Job_C back to Job_A. For example you could a create circular execution path, or a loop, if one of the links in a job set step for success, error, or warning links back to the same job set step. Thus, each job set step can link to any of the available job definitions or job sets, or they could all use the same job definition or job set as a link for the success, error and warning case. There is only a possible loop based on the path through the job set steps, as identified by the job set step ID. Oracle Enterprise Scheduler validates job sets at submission time to try to prevent job set step level looping. Also, Oracle JDeveloper does not allow you to create a job set containing a job set step level loop.
To prevent looping within a job set, job set definitions should not contain self-referencing execution paths. For example, in a job set with Job_B defined, Oracle Enterprise Scheduler does not allow you to define an execution path from Job_B to Job_B itself if Job_B ends up with a terminal state of ERROR. However using the RETRIES property available for a job definition or a job set, you can have multiple executions up to the configured RETRIES number.
When there is no job set link defined for a terminal state of a step, it implies that the job set should stop if the step ends with the unspecified terminal state. For example if there is no link defined for a step Job_D for the state WARNING, and if the step Job_D ends up with the state of WARNING, the job set stops execution.

Each job set step can be defined to use any of the available job definitions or job sets, and multiple steps may use the same job definition or job set.

12.2.6 What You Need to Know About Job Set Application Defined Properties and System Properties

There are cases where job set application defined properties or system properties may conflict with application defined properties or system properties set either in metadata or when a job request is submitted. For more information on how job set application defined properties and system properties are handled, see Section 7.2, "Using Parameters with the Metadata Service" and Section 7.3, "Using Parameters with the Runtime Service".

12.2.7 What Happens at Runtime for Job Set State Priorities and State Transitions

At runtime, the individual steps in a job set can end up with different terminal states, as indicated in Table 12-2. When a job set step is a job set, the job set step also ends with one of these terminal states. Oracle Enterprise Scheduler provides a priority hierarchy for the terminal states of job set steps. This means that when there are multiple steps in a job set, the job set terminal state is applied the terminal state of the step with the highest priority terminal state. Thus, the highest priority terminal state of the steps determines the resulting state for the entire job set.

The resulting state of a job set affects all subsequent state dependent processing within the system. A job set always follows the basic rule of transitioning to a terminal state based on the terminal states of its child requests, only after the completion of all child requests. As a rule, the job set transitions to one of the computed terminal states only after all child requests have finished and transitioned to terminal states. For example, if a given job set is actually a step within another job set, then the way in which the state of the inner job set request is computed affects the conditional execution within the outer job set.

Table 12-3 shows the possible job set terminal states with the level indicated in the Priority column.

Table 12-3 Job Set Terminal State Transitions

Terminal State	Description	Priority
`ERROR`	If any step in a job set finishes with the terminal state of `ERROR`, the entire job set is marked with the terminal state of `ERROR` no matter what the state of the other steps. For serial job sets, if one step goes to `ERROR`, subsequent steps will not execute. For parallel job sets, all steps begin at the same time, and the job set state is not determined until the job set steps reach a terminal state.	The `ERROR` state has the highest priority.
`WARNING`	If any step in a job set ends up with the terminal state of `WARNING`, and there is no step with the terminal state of `ERROR` then the job set is marked with the terminal state `WARNING`. When the terminal state is `WARNING`, post processing will begin.	Lower than `ERROR`
`EXPIRED`	The job set transitions to `EXPIRED` state if at least one of the child requests expires while there is no step that ends with the terminal state of `ERROR` or `WARNING`.	Lower than `ERROR` and `WARNING`
`CANCELLED`	Based on the actual outcome of a cancellation attempt, the job set can transition to `CANCELLED` if at least one child request successfully processes the cancellation attempt and transitions to `CANCELLED` state. The cancellation might have been requested on the entire job set or just a specific child request. Further the transition to `CANCELLED` follows the priorities of terminal states. Therefore the job set transitions to `CANCELLED` terminal state only if there is no step that ends with the state of `ERROR`, `WARNING`, or `EXPIRED` and there is at least one step with terminal state of `CANCELLED`. When a job set is cancelled, steps that have not been added or run are considered to be `CANCELLED` for the purpose of final state.	Lower than `ERROR`, `WARNING`, and `EXPIRED`
`SUCCEEDED`	The job set is considered as `SUCCEEDED` if and only if all child requests completed with the terminal state of `SUCCEEDED`.	The `SUCCEEDED` state has the lowest priority among all terminal states

Table 12-4 lists additional possible states for a job set:

Table 12-4 Possible Job Set Runtime States

State	Description
`WAIT`	This is the initial state of the submitted job set request. Once the job set request transitions to `RUNNING` state, however, all generated child requests transition directly to `READY` state rather than `WAIT` state.
`READY`	Job sets never go to `READY` state. The submitted job set request transitions from `WAIT` to `RUNNING` state. Nested job sets are generated in `RUNNING` state. The only job set steps that begin in `READY` state are steps composed of job definitions.
`RUNNING`	The submitted job set transitions from `WAIT` to `RUNNING` state when it begins to be processed. Nested job sets start in `RUNNING` state and remain in `RUNNING` state as long as at least one child is in a non-terminal state.
`CANCELLING`	A job set transitions to `CANCELLING` when the user requests a cancellation for the entire job set. This can be done by calling `cancelRequest()` with the request ID of the parent request representing the job set. Passing the parent request ID indicates that the user wants to cancel entire job set irrespective of its current, non-terminal, state and the states of its child requests. In such cases, a cancellation will be attempted on all child requests that are still active and have not already transitioned to a terminal state. On the other hand if cancellation is attempted only on a specific child request in the job set, there won't be any state change for the parent request and only the particular child request will transition to `CANCELLING` if possible. If the cancel happens during post-processing, the state is set to `WARNING` rather than `CANCELLED`. If the job set finishes before the cancel is issued, the job set can have state `SUCCEEDED`.
`COMPLETED`	This state indicates that the job set or job set step has finished executing and post-processing will begin.
`BLOCKED`	The `BLOCKED` state is not a terminal state. However any request can remain in a `BLOCKED` state for a long period until the blocking condition is eliminated (such as incompatibility). In the case of a job set, any individual step might be `BLOCKED` while other steps either complete or may be running. The job set itself, however, remains in a `RUNNING` state. Eventually if all steps in the job set complete except the ones that are in the `BLOCKED` state, the job set cannot continue further until the blocking step is ready to run. When the blocked step unblocks and completes, the job set can proceed. After the steps complete, the job set eventually goes to the appropriate terminal state. For a serial job set, the job set may stop at a step that is in `BLOCKED` state. In such cases, all previous steps are complete and the job set cannot continue until the blocked step executes. However for a parallel job set, multiple steps can remain in `BLOCKED` state. Further, while some steps are blocked, other steps can still continue to run.
`HOLD`	The `HOLD` state is very similar to the `BLOCKED` state. Following the same rules for the `BLOCKED` state, a job set cannot continue running while a step is in `HOLD` state. A serial job set cannot continue if the current step in the execution flow is stuck at `HOLD` state. In the case of a parallel job set, if at least one step is stuck in `HOLD` state while all other steps have completed, the job set can complete when the step is no longer in `HOLD` state.

12.3 Cross Application Job Sets

Oracle Enterprise Scheduler provides the capability for a job or a job set to execute across multiple applications as shown in Figure 12-5:

Job set FIN has three steps, two of which are defined to execute in different applications.
Job set FIN is submitted to the GL application.
Step 1 has the EFFECTIVE_APPLICATION system property set to ODI, so Step 1 executes in the ODI application.
Step 2 does not have an effective application set, so it executes in the GL application.
Step 3 has the EFFECTIVE_APPLICATION system property set to INV, so Step 3 executes in the INV application.

Figure 12-5 Cross Application Job Set Steps

12.3.1 Overview of Cross Application Job Sets

A job set runs in its hosting application and by default, all job set steps also run in this application. A job set step can be associated with a different application by defining the EFFECTIVE_APPLICATION system property on the step. If the step is a job, the job will execute in the effective application. If the step is a nested job set, the jobs in the nested job set execute in the effective application. When EFFECTIVE_APPLICATION is defined for a step, the request for the step and any child requests of the step are associated with the effective application, meaning the APPLICATION system property for those requests will be set to the effective application.

The EFFECTIVE_APPLICATION system property may only be defined in metadata, specifically job set, job set step, job type, and job. The property EFFECTIVE_APPLICATION is not supported in the request parameters. The effective application must be in the same cluster as the hosting application, or an error will result. If a submitted job set defines the effective application, that value must be the same as the hosting application, or the job set submission will be rejected.

Subrequests created by a job set step must run in the same application as the job set step. In other words, EFFECTIVE_APPLICATION is not supported for subrequests. If the job for a subrequest defines the effective application, that value must be the same as the application of the job submitting the subrequest, or the subrequest submission will be rejected.

For a job set that executes across multiple applications, querying for requests by application is not sufficient to retrieve all children. Oracle Enterprise Scheduler supports absolute parent id as a query field, making it possible to query for all requests in a job set regardless of the application. The absolute parent id is the request id of the job set that was submitted to the hosting application.

12.3.2 Requirements for Cross Application Job Sets

Oracle Enterprise Scheduler supports cross-application job set subject to the following requirements:

All applications for a given job set must be deployed in the same cluster.
All applications in the job set must share the same enterprise security.
All request metadata must be accessible from the application the job set is submitted to, referred to as the hosting application. All metadata for the request are persisted to the runtime store for the hosting application. The persisted metadata include all metadata used by the submitted job set and any nested job set.
Metadata for subrequests must be accessible from the application that submits the subrequest, unless the metadata used by the subrequest were already persisted to the runtime store at job set submission time.

12.4 Using Input and Output Forwarding

Oracle Enterprise Scheduler configures a USER_FILE_DIR parameter to specify the directory for all jobs to store their input and output files. This parameter is populated by the property RequestFileDirectory in the ess-config.xml file. When this parameter is set, Oracle Enterprise Scheduler set the system property USER_FILE_DIR for all job requests. When a job request is processed, in the pre- or post-processor or its execution the job can read, write, create, delete and manage files and sub-directories based this property. Oracle Enterprise Scheduler does not impose any structure on the user file directory nor support any file or directory operations.

The purpose of this file support is to allow job implementation to reference files relative to a configurable location so that the job implementation is not tied to a particular environment. It de-couples job implementation with file input and output from the job execution environment.

The USER_FILE_DIR property allows job requests to dynamically change the file.

12.4.1 Supporting Input and Output Forwarding in Job Sets

Sometimes a step in a job set needs input from the previous step in the job set. Oracle Enterprise Scheduler uses two system properties INPUT_LIST and OUTPUT_LIST to facilitate forwarding the output from one step to the input of the next step.

When a job produces information, such as a list of output files, that needs to be passed on to the next step in a job set, the job adds the information to the OUTPUT_LIST property. Upon completion of the job request execution, Oracle Enterprise Scheduler forwards the OUTPUT_LIST property of the request so that it becomes the INPUT_LIST property of the next step before it executes. The next step takes as its input the output of the previous step.

A job set step can be a single job or a job set, Oracle Enterprise Scheduler supports forwarding with nested job sets as well. For a serial job set, Oracle Enterprise Scheduler defines the output of the job set as the output of the last step of the job set, meaning that only the OUTPUT_LIST property of the last step is forwarded to the next step. Similarly, the input to a serial job set is forwarded only to the first step of the job set; that is, only the first step of a serial job set has the INPUT_LIST property set to the value of the OUTPUT_LIST property of the previous step.

For a parallel job set, Oracle Enterprise Scheduler specifies that the output of the job set is the concatenation of the OUTPUT_LIST property of every job in the job set, separated by a delimiter (with no order guaranteed). The input to a parallel job set is forwarded to every job in the job set, meaning that every job in the parallel job set has the same INPUT_LIST property. The system property OUTPUT_LIST_DELIMITER specifies the delimiter used when listing output files.

Suppose a job set has two jobs, each job producing its own output file, file1.txt and file2.txt. The system property OUTPUT_LIST for that job set will have the values file1.txt;file2.txt, assuming the value of OUTPUT_LIST_DELIMITER is a semi-colon. The concatenated list of output files enables the next job step in the job set to access output files generated by previous steps within the job set.

The InputFile class provides access to files as input to a job definition. There is currently no mechanism for accepting a file as an input to a job request.

Except for forwarding the value of the OUTPUT_LIST property of a step to the value of the INPUT_LIST property of the next step, Oracle Enterprise Scheduler treats the two properties like any other system properties. Oracle Enterprise Scheduler does not define the format for the value of the properties (except for the semicolon delimiter in case of parallel job set). It is the responsibility of the job to define the syntax and semantics for the properties; for example using a fully qualified name or relative path name and a comma or space as a delimiter.