This chapter describes how to define and submit an Oracle Enterprise Scheduler job set, a collection of job definitions that can be grouped together to run as a single unit.
This chapter includes the following sections:
Oracle Enterprise Scheduler provides for collections of job definitions that can be grouped together to run as a single unit called a job set. A job set may be nested; thus a job set may contain a collection of job definitions or one or more child job sets. Each job definition or job set included within a job set is called a job set step.
A job set is defined as either a serial job set or a parallel job set. At runtime, Oracle Enterprise Scheduler runs parallel job set steps together, in parallel. When a serial job set runs, Oracle Enterprise Scheduler runs the steps one after another in a specific sequence. Using a serial job set Oracle Enterprise Scheduler supports conditional branching between steps based on the execution status of a previous step.
You can define a serial job set to include a parallel job set, or a parallel job set to include a serial job set. Job sets that include a mix of parallel and serial job sets are called complex job sets. For example, when a serial job set contains a child parallel job set, the serial job set runs serially until it reaches the child parallel job set. Then, all the job definitions or job set definitions in the child parallel job set run in parallel. Upon completion of the child parallel job set the serial job set continues running its remaining steps serially. Nested parallel job sets behave the same as non-nested parallel job sets.
For every step in a job set Oracle Enterprise Scheduler supports a property (
SYS_selectState) that provides runtime flexibility for how a particular step affects the entire job set. This property is defined on a per step basis. Table 17-1 describes SYS_selectState.
Table 17-1 Job Set Step Property
Specifies whether the result state of a job set step should be included when determining the state of the job set. Specifies whether the execution state of the step affects the eventual state of entire job set.
By default, all job set steps affect the job set state. To prevent the state of a particular job set step from affecting the state of the job set, set
Oracle Enterprise Scheduler provides the capability for a job set to execute across multiple applications. A job set runs in its hosting application and by default all job set steps also run in this application.
You can define a job set in Oracle JDeveloper by specifying the following:
The name, package, and description for the job set
The application defined properties for the job set
The system properties for the job set
Specifying the job set steps
The contents of a job set are specified when you define the job set steps. For example, for a serial job set you specify the name and the execution mode and then you add the job set steps to define the sequence of job definitions or child job sets that run when the job set runs.
An Oracle Enterprise Scheduler job set is defined by a name, a package, a job set execution mode, step definitions, application defined properties, and system properties.
To create a job set:
In Oracle JDeveloper, right-click in the project to view the New Gallery.
In the All Technologies tab, under Categories, expand Business Tier and select Enterprise Scheduler Metadata.
Under Items, select Job Set and click OK. This displays the Create Job Set window.
In the Create Job Set window, specify the following:
In the Name field, enter a name for the job set or accept the default name.
In the Package field, enter a package name for the job set.
The Location field displays the full path of the directory where the job set file is stored.
Click OK. This creates the job set and displays the Job Set Definition page, as shown in Figure 17-1.
Figure 17-1 Job Set Editor with Serial Job Set
In the Job Set Editor pane, in the Description field enter a description for the job set.
In the Job Set Steps area, select the Parallel or Serial radio button to specify parallel or serial execution mode for the job set.
In the Application Defined Properties area, click Add to add properties associated with the job set. You use these to represent an application-specific or step-specific application defined property for the job set. For more information on using application defined properties, see Introduction to Using Parameters and System Properties. For more information, see What You Need to Know About Job Set Level Parameter Materialization.
In the System Properties area, click Add to add system properties associated with the job set. For more information on using system properties, see Using System Properties.
In the Access Control area, click Add to modify the list of roles that have access to this metadata, along with their access levels. For more information on defining access, see Oracle Enterprise Scheduler Security .
In the Localization area, enter the following information for localizing this job set:
Resource Bundle Base Name -- The base name for the resource bundle that specifies internationalized values.
Display Name Resource Key -- The resource key that specifies the display name value in the resource bundle.
Description Resource Key -- The resource key that specifies the description text in the resource bundle.
Save the job set.
To define serial job set steps you select the serial execution mode and then add job set steps. Job set steps are created from the available job definitions and job sets defined in the current project. You define serial job set steps when you specify a step ID and a job definition child job set definition associated with the step. You also define links from a job set step terminal states to specify the next step. Table 17-2 lists the possible terminal states that you can specify using JDeveloper.
Table 17-2 Job Set Serial Execution Step Terminal States
Oracle JDeveloper indicates this state with a check mark button. This path represents a child step or child job set was successfully processed by the system.
Oracle JDeveloper indicates this step with a warning button. A child step or child job set resulted in a warning.
Oracle JDeveloper indicates this step with an error button. Some aspect of the request to run the child step or child job set processing resulted in an error.
To add serial job set steps:
Figure 17-2 Job Set with a Step Added
ERRORterminal state for the step. For example, from the dropdown list select Step_error (Step_error must be defined).
WARNINGterminal state for the step. For example, from the dropdown list select
Step_warning(Step_warning must be defined).
Figure 17-3 Job Set with Two Steps Added
You can add parallel job set steps to a job set.
To add parallel job set steps:
The Add Step window displays.
Figure 17-4 Adding Job Set Steps to a Parallel Job Set
When you define a job set with Oracle JDeveloper, Oracle JDeveloper creates an XML file containing elements that represent the steps that you define.
When you define a parallel job set you specify a set of job set steps that run together. A parallel job set only contains steps, and does not contain links between steps, as all the steps execute together and do not depend on each other or upon the order in which each step runs.
When you define a job set Oracle JDeveloper creates an XML document that conforms to the Oracle Enterprise Scheduler job step schema.
When you define a serial job set, the associated XML document includes job set steps and links. Oracle Enterprise Scheduler enforces the following limitations for serial job set definitions:
To prevent looping within a job set, job set definitions should not contain circular execution paths. A circular execution path, or a loop, is defined at the job set level as follows: loop is a path from one job set step along the links of any number of other steps back to the same job set step. For example, in a job set with a flow from
Job_C defined, Oracle Enterprise Scheduler does not allow you to define an execution path from
Job_C back to
Job_A. For example you could a create circular execution path, or a loop, if one of the links in a job set step for success, error, or warning links back to the same job set step. Thus, each job set step can link to any of the available job definitions or job sets, or they could all use the same job definition or job set as a link for the success, error and warning case. There is only a possible loop based on the path through the job set steps, as identified by the job set step ID. Oracle Enterprise Scheduler validates job sets at submission time to try to prevent job set step level looping. Also, Oracle JDeveloper does not allow you to create a job set containing a job set step level loop.
To prevent looping within a job set, job set definitions should not contain self-referencing execution paths. For example, in a job set with
Job_B defined, Oracle Enterprise Scheduler does not allow you to define an execution path from
Job_B itself if
Job_B ends up with a terminal state of
ERROR. However using the
RETRIES property available for a job definition or a job set, you can have multiple executions up to the configured
When there is no job set link defined for a terminal state of a step, it implies that the job set should stop if the step ends with the unspecified terminal state. For example if there is no link defined for a step
Job_D for the state
WARNING, and if the step
Job_D ends up with the state of
WARNING, the job set stops execution.
Each job set step can be defined to use any of the available job definitions or job sets, and multiple steps may use the same job definition or job set.
There are cases where job set application defined properties or system properties may conflict with application defined properties or system properties set either in metadata or when a job request is submitted. For more information on how job set application defined properties and system properties are handled, see Using Parameters with the Metadata Service and Using Parameters with the Runtime Service.
At runtime, the individual steps in a job set can end up with different terminal states, as indicated in Table 17-2. When a job set step is a job set, the job set step also ends with one of these terminal states. Oracle Enterprise Scheduler provides a priority hierarchy for the terminal states of job set steps. This means that when there are multiple steps in a job set, the job set terminal state is applied the terminal state of the step with the highest priority terminal state. Thus, the highest priority terminal state of the steps determines the resulting state for the entire job set.
The resulting state of a job set affects all subsequent state dependent processing within the system. A job set always follows the basic rule of transitioning to a terminal state based on the terminal states of its child requests, only after the completion of all child requests. As a rule, the job set transitions to one of the computed terminal states only after all child requests have finished and transitioned to terminal states. For example, if a given job set is actually a step within another job set, then the way in which the state of the inner job set request is computed affects the conditional execution within the outer job set.
Table 17-3 shows the possible job set terminal states with the level indicated in the Priority column.
Table 17-3 Job Set Terminal State Transitions
If any step in a job set finishes with the terminal state of
For serial job sets, if one step goes to
If any step in a job set ends up with the terminal state of
The job set transitions to
Based on the actual outcome of a cancellation attempt, the job set can transition to
Further the transition to
When a job set is canceled, steps that have not been added or run are considered to be
The job set is considered as
Table 17-4 lists additional possible states for a job set:
Table 17-4 Possible Job Set Runtime States
This is the initial state of the submitted job set request. After the job set request transitions to
Job sets go from
The submitted job set transitions from
A job set transitions to
In such cases, a cancellation is attempted on all child requests that are still active and have not already transitioned to a terminal state.
On the other hand if cancellation is attempted only on a specific child request in the job set, there won't be any state change for the parent request and only the particular child request transitions to
If the cancel happens during post-processing, the state is set to
This state indicates that the job set or job set step has finished executing and post-processing begins.
In the case of a job set, any individual step might be
For a serial job set, the job set may stop at a step that is in
However for a parallel job set, multiple steps can remain in
Oracle Enterprise Scheduler provides the capability for a job or a job set to execute across multiple applications as shown in Figure 17-5:
Job set FIN has three steps, two of which are defined to execute in different applications.
Job set FIN is submitted to the GL application.
Step 1 has the
EFFECTIVE_APPLICATION system property set to ODI, so Step 1 executes in the ODI application.
Step 2 does not have an effective application set, so it executes in the GL application.
Step 3 has the
EFFECTIVE_APPLICATION system property set to INV, so Step 3 executes in the INV application.
Figure 17-5 Cross Application Job Set Steps
A job set runs in its hosting application and by default, all job set steps also run in this application. The system property
SYS_effectiveApplication should be defined on the job definition or job set (rather than the job set step). For a nested job set that defines
SYS_effectiveApplication, the application applies to any child requests of that nested job set. If it is a nested job set, the jobs in the nested job set execute in the effective application. When
SYS_effectiveApplication is defined for a job, the request for the job set and any child requests of the job set are associated with the effective application, meaning the
APPLICATION system property for those requests is set to the effective application.
SYS_effectiveApplication system property may only be defined in metadata, specifically job set, job set step, job type, and job. The property
SYS_effectiveApplication is not supported in the request parameters. The effective application must be in the same cluster as the hosting application, or an error results. If a submitted job set defines the effective application, that value must be the same as the hosting application, or the job set submission is rejected.
For a job set that executes across multiple applications, querying for requests by application is not sufficient to retrieve all children. Oracle Enterprise Scheduler supports absolute parent ID as a query field, making it possible to query for all requests in a job set regardless of the application. The absolute parent ID is the request ID of the job set that was submitted to the hosting application.
Oracle Enterprise Scheduler supports cross-application job set subject to the following requirements:
All applications for a given job set must be deployed in the same cluster.
All applications in the job set must share the same enterprise security.
All request metadata must be accessible from the application the job set is submitted to, referred to as the hosting application. All metadata for the request are persisted to the runtime store for the hosting application. The persisted metadata include all metadata used by the submitted job set and any nested job set.
Metadata for subrequests must be accessible from the application that submits the subrequest, unless the metadata used by the subrequest were already persisted to the runtime store at job set submission time.
Sometimes a step in a job set requires input from the previous step in the job set. Oracle Enterprise Scheduler uses two system properties
SYS_outputList to facilitate forwarding the output from one step to the input of the next step.
When a job produces information, such as a list of output files, that must be passed on to the next step in a job set, the job adds the information to the
SYS_outputList property. Upon completion of the job request execution, Oracle Enterprise Scheduler forwards the
SYS_outputList property of the request so that it becomes the
SYS_inputList property of the next step before it executes. The next step takes as its input the output of the previous step.
A job set step can be a single job or a job set, Oracle Enterprise Scheduler supports forwarding with nested job sets as well. For a serial job set, Oracle Enterprise Scheduler defines the output of the job set as the output of the last step of the job set, meaning that only the
SYS_outputList property of the last step is forwarded to the next step. Similarly, the input to a serial job set is forwarded only to the first step of the job set; that is, only the first step of a serial job set has the
SYS_inputList property set to the value of the
SYS_outputList property of the previous step.
For a parallel job set, Oracle Enterprise Scheduler specifies that the output of the job set is the concatenation of the
SYS_outputList property of every job in the job set, separated by a delimiter (with no order guaranteed). The input to a parallel job set is forwarded to every job in the job set, meaning that every job in the parallel job set has the same
INPUT_LIST property. The system property
OUTPUT_LIST_DELIMITER specifies the delimiter used when listing output files.
Suppose a job set has two jobs, each job producing its own output file,
file2.txt. The system property
SYS_outputList for that job set has the values
file1.txt;file2.txt, assuming the value of
OUTPUT_LIST_DELIMITER is a semi-colon. The concatenated list of output files enables the next job step in the job set to access output files generated by previous steps within the job set.
InputFile class provides access to files as input to a job definition. There is currently no mechanism for accepting a file as an input to a job request.
Except for forwarding the value of the
SYS_outputList property of a step to the value of the
SYS_inputList property of the next step, Oracle Enterprise Scheduler treats the two properties like any other system properties. Oracle Enterprise Scheduler does not define the format for the value of the properties (except for the semicolon delimiter in case of parallel job set). It is the responsibility of the job to define the syntax and semantics for the properties; for example using a fully qualified name or relative path name and a comma or space as a delimiter.