Using the Master Run Utility to Automatically Run Your ETL Jobs
This topic provides an overview of the Master Run Utility and discusses how to run the utility.
Understanding the Master Run Utility
PeopleSoft provides the Master Run Utility, which enables you to automatically run a set of jobs located in a flat file on the DataStage Server. When you use the Master Run Utility, it reads a list of jobs that are present in a specified flat file and triggers the jobs to run in serial mode, using the dependency logic specified in the Input flat file.
The following features are incorporated into the Master Run Utility:
Run jobs from a specified flat file.
Reset jobs and rerun when they are in an aborted stage.
Active restart recovery.
Run jobs in dependent or independent modes.
Generate a job status report in the log file at the end of a run.
Location of the Master Run Utility
The Master Run Utility is available in the Common_Utilities.dsx file. After you import this .dsx file, you can locate the utility using the following navigation: Jobs, Utilities, Job_Utils, Master_Run_Utility.
Input File for Master Run Utility
The Input flat file contains the list of ETL jobs that are available in the DataStage Server and indicates whether a job is independent or dependent. Dependency information is used to determine job execution in the Master Run Utility. If a job labeled as independent fails to run properly, the Master Run Utility logs the information and proceeds to the next job. However, if a job labeled as dependent fails to run properly, the utility itself aborts. All jobs listed in the flat file contain the suffix N (Independent job) or D (Dependent Job) to indicate their dependency.
The Input flat file also contains comments that describe each set of ETL jobs. Comments are prefixed with an asterisk (*) to clearly indicate they are comments. The following screen shot provides an example flat file:
Image: Sample Flat File (HCM_OWS_E_Base_HashFiles.txt)
This example illustrates the Sample Flat File (HCM_OWS_E_Base_HashFiles.txt).
The sample file above contains the list of HCM jobs used to load data into OWS target tables.
Restart Recovery Feature
When a dependent job fails to run properly, the Master Run Utility automatically aborts the job and all subsequent jobs. Assuming you fix the problem that caused the job to fail, you must rerun the Master Run Utility to complete the process of running those jobs that were aborted. Instead of running all the jobs in that particular folder, the restart recovery feature enables you to rerun only those jobs that were aborted.
For example, assume the file SAMPLE_HCM_E_GLOBAL_DIMENSIONS_Base_Tables.txt contains Global Dimension jobs that are all dependent to each other. If the SEQ_J_Dim_PS_D_POS job is aborted, the entire utility aborts. Later you fix the issue in the SEQ_J_Dim_PS_D_POS job. This time, you can run the Master_Run_Utility with the Restart Recovery option set to Yes so that it runs the jobs from SEQ_J_Dim_PS_D_POS only and ignores the jobs that have previously completed successfully.
Running the Master Run Utility
Perform the following steps to run the Master Run Utility:
In DataStage Director, navigate to the Master Run Utility using the following navigation: Jobs, Utilities, Job_Utils.
Select Master_Run_Utility and click Run.
The Master_Run_Utility - Job Run Options window appears.
Image: Master_Run_Utility - Job Run Options Window
This example illustrates the Master_Run_Utility - Job Run Options Window.
Enter the path to the location to the flat file and specify whether you want to use the restart recovery feature.
Click Run.
From the DataStage Director menu, select View, Log.
The Master Run Utility generates a report with the following information:
Jobs that have completed successfully.
Jobs that have completed with warnings.
Jobs that have aborted
Jobs that have not compiled
Jobs that have incorrect names.
If you double-click log entries that contain the phrase COMPLETED SUCCESSFULLY, the Event Details window appears and displays all the jobs that have successfully completed.
Image: Event Details Window
This example illustrates the Event Details Window.
Likewise, if you double-click log entries that contain the key word ABORTED, the Event Details window appears and displays all the jobs that have aborted.