Tips For Setting Up Scheduler Batch Stream Definitions
Configuring scheduler batch stream definitions to avoid open-ended stream issue
The scheduler batch stream definitions should always be configured such that the stream “ends” only once all the steps in the stream are completed either by configuring it on the “end of stream” step condition or by defining the inter-dependencies between the steps. This ensures the scheduler batch stream run execution does not stop in the middle of the stream.
Transferring batch job stream definitions to SaaS batch scheduler
-
(Optional Step) Migrate existing batch job stream definitions from production environment to any pre-prod environment.
-
In any pre-prod environment, use the transfer option to move the existing batch job stream definition to scheduler batch stream definition (new object for SaaS Batch Scheduler).
-
Perform thorough testing of newly created scheduler batch stream definitions.
-
Before migrating to the target environments, the batch job stream definitions in the target environment should be canceled(if needed) and deleted.
-
Migrate the newly created scheduler batch stream definitions by using Content Migration Assistant (CMA) to production. Migration Plans for scheduler batch stream definitions are available for review.
Jobs that run past midnight must have same business date as other jobs in a stream
In situations where a scheduler batch stream runs too long and has some jobs run past midnight, all jobs in the scheduler batch stream can be set to run with same business date. Set the business date option value on the scheduler batch stream definition to ‘SYSDATE’ and the scheduler engine processes all the steps in the scheduler batch stream with same date when the scheduler batch stream started.
How to split a big stream into multiple smaller stream
How to configure scheduler batch stream definitions to run only on OUAF working calendar dates
To setup Scheduler Batch Stream Definition to run only on working days, use the batch control K1-CWCEL. Create a Batch Scheduler Batch Control with this batch control, batch parameter Work Calendar set to required work calendar, Verify that Run On Business Day is set to true. Use this Batch Scheduler Batch Control as the first step to run on the stream. Define the stream to end if this step fails. Define the stream’s subsequent step to execute if this step succeeds. When the stream runs on a scheduled day, the first step checks if the day falls on a working day as per the work calendar. If the day is not a working day then the stream ends execution ensuring stream runs only on desired dates.
Scheduler batch stream definitions that check existence of a file
In situations where a scheduler batch stream definition needs to check if a file has been uploaded/created by another process before it can continue with rest of it’s processing, a special batch program “K1-OSCFE” is available. A stream step can be introduced referring to this new batch program with parameter specifying which object storage bucket to check the file in and also status to return if the file does not exist. This stream step can be added right before the step that has dependency on checking the existence of the file and continue processing or not.
Handling scheduler batch stream definitions inter-dependencies
-
The new batch program accepts two parameters script name and script data. Write a CM script that checks whether a dependent job stream is running. If it’s running, then the script should terminate with an error.
- On the scheduler batch stream that needs to be paused/hold if another job stream is running, add a new stream step as the first step of the job stream. Use the new batch program on the first step(by setting the CM script as the input parameter) or create a batch scheduler batch control if the batch control is to be used across many job streams with different CM service scripts as input parameters. Set the dependent job stream name as the script data parameter.
- Update the "End Of Stream" step condition such that if first step has failed, end the stream. This ensures the scheduler batch stream does not run if the other dependent scheduler batch stream is already running.
- Update the step(s) intended to execute after the first step, the step condition that if first step succeeded only then these steps should continue.
Note. The special batch program is not restricted to only these situations. It can be used in many custom scenarios by writing the corresponding CM script that terminates if a desired situation is met.