| Oracle® Retail Bulk Data Integration Cloud Service Implementation Guide Release 19.1.000 F31810-01 |
|
 Previous |
 Next |
| Oracle® Retail Bulk Data Integration Cloud Service Implementation Guide Release 19.1.000 F31810-01 |
|
Previous |
Next |
The Scheduler application in the Bulk Data Integration (BDI) product suite assists in scheduling of batch processes to run at predefined configured intervals of times. A schedule determines when a job or a process or any program needs to be executed and the frequency of execution.
The Scheduler application runtime is based on the container-managed Java EE timer service to execute the schedules and utilizes the Oracle WebLogic Server's implementation and management of the timer service when deployed on the WebLogic server.
The Scheduler supports various schedules ranging from simple interval schedules such as hourly, daily, and so on, to advanced cron-like scheduling.
Scheduler currently supports calling of REST services. The application out-of-the-box contains schedules to run BDI Process Flows in scope for this release.
The Scheduler Console (Admin UI) enables runtime monitoring and administration of schedules where the user can view, create, edit, delete schedules, manually run a schedule, enable/disable schedule, set up notifications for schedules and so on.
This section describes the scheduler core concepts.
Schedule Definition
A schedule definition comprises details of a schedule such as Schedule Name, and Schedule Group which indicates logical or functional grouping of schedules, and Schedule Description.
Schedule Execution
A schedule execution is an instance of scheduled run of a schedule at the specified frequency.
A schedule can be an interval-based schedule or calendar-based schedule.
An interval-based schedule is a schedule that repeats at fixed interval of time starting from a specific time. For example, hourly, daily, weekly, every 5 minutes and so on.
A calendar-based schedule is a cron-type of schedule that specifies different times that the schedule runs. More complex schedules that can be specified as cron expression are defined as calendar-based schedules.
The following parameters define a calendar-based schedule, same as the parameters in a cron expression: Minutes, Hours, Day of Week, Day of Month and Month.
|
Note: The Scheduler does not currently support Seconds and Year parameters in a calendar schedule. |
This section describes the scheduling mechanisms.
Simple schedules are predefined schedule frequencies that are available as options for the user to choose readily. The following are the simple schedules that the Scheduler supports.
Hourly
Daily
Weekly
Monthly
Weekday (Monday-Friday)
Weekend (Saturday-Sunday)
Saturday
Sunday
First day of every month
Last day of every month
One time only (run once), and
User-specified frequency with interval in the units of:
minutes, hours, days or weeks
BDI Scheduler supports advanced scheduling which is cron-like scheduling. Calendar-based schedules that can be expressed in cron-format can be setup with the advanced scheduling capability of the Scheduler. Advanced scheduling is defined with the following parameters (similar to that of cron expression) and the corresponding range of values:
Minutes : 0-59
Hours : 0-23 (12:00 a.m. - 11:00 p.m.)
Day of Week : Monday - Sunday
Day of Month : 1-31
Month : 1-12 (January - December)
If a schedule is created with multiple values for the above parameters, then the schedule will repeat at all those specified times.
The schedule frequency defines the frequency at which a schedule has to be repeated at the configured time and interval starting from a given point of time. The schedule frequency has the following parameters that determines when the schedule has to be run.
It specifies the start date and time when a particular schedule has to start executing.
For interval based schedules, this is the first time the schedule runs and then repeats based on the specified interval.
For example, a schedule with start datetime as 2016-08-15 10:00a.m. and repeat 'Daily' will first run at 2016-08-15 10:00 a.m. and next run at 2016-08-16 10:00 a.m. and so on.
For calendar schedules (cron schedules), this defines the time from when the schedule will become effective and starts executing based on the frequency. So it is not necessarily the first run of the schedule, though it very well may be.
For example, a schedule with start datetime as 2016-08-15 10:00 a.m. (which is a Monday) but repeat every Thursday, will first run at 2016-08-18 10:00 a.m. (Thursday) and subsequently next run at 2016-08-25 10:00 a.m. (Thursday) and so on.
So the Start Datetime here signifies the datetime the schedule becomes effective and that it will not run before that datetime. However, here the Start Datetime can very well be specified as 2016-08-18 10:00 a.m. (Thursday) and repeat every Thursday.
So in summary, for interval-based schedules, first run of the Schedule equals Schedule Start Datetime. For calendar-based schedules, first run of the Schedule may or may not be equal to the Schedule Start Datetime, based on the schedule recurrence specified.
It specifies an end date and time when the schedule should stop executing and no longer run. When a schedule has no end datetime specified, it runs indefinitely.
|
Note: The end datetime is inclusive for the schedule execution, meaning if the schedule recurrence coincides with the end datetime, the schedule will execute at the end datetime and only then does not repeat. |
For example, say, Schedule Start Datetime: 2016-08-15 10:00 a.m., repeat 'Hourly', Schedule End Datetime: 2016-08-15 11:00 a.m., then the schedule will run at 10:00 a.m. and also at 11:00 a.m. before ceasing to run.
This specifies the frequency at which the schedule repeats. This is same as described in Simple and Advanced Scheduling.
This indicates the date and time of the next occurrence of the schedule, obtained based on the configured schedule frequency.
All the date and times in the Scheduler are based on the timezone of the server (JVM) where the application is deployed.
The Scheduler Console (UI) displays the server's current date and time with timezone (the current time displayed gets refreshed when the UI is refreshed).
|
Note: When creating or updating a schedule and in monitoring schedule executions in the Scheduler Console, the date and time are as per the timezone setup in the application server and not the local timezone. |
The Schedule Action defines what is executed when the schedule runs at the specified frequency. It is a DSL (domain specific language) that is based on Groovy. The schedule action has a simple syntax as follows.
action {
//Define what needs to be executed here. Say invoke a REST service.
}
Currently Schedule Action supports calling REST services. BDI process flows are called by the Scheduler as REST services.
For example, to trigger a BDI process flow named Store_Fnd_ProcessFlow_From_RMS, the following schedule action is defined:
action {
(POST[externalVariables.processFlowAdminBaseUrl + "/resources/batch/processes/operator/Store_Fnd_ProcessFlow_From_RMS"]^externalVariables.processFlowAdminBaseUrlUserAlias) as String
}
POST denotes the REST method.
processFlowAdminBaseUrl is an entry key in 'externalVariables' map variable used by the Scheduler runtime and specifies the BDI Process Flow Admin's base URL. The value for processFlowAdminBaseUrl is specified during install time and gets stored in the BDI System Options. The value of processFlowAdminBaseUrl will be like https://<host>:<port>/bdi-process-flow.
For example, https://example.com:8001/bdi-process-flow
The value for processFlowAdminBaseUrl is specified during install time and gets stored in the BDI System Options.
/resources/batch/processes/operator/Store_Fnd_ProcessFlow_From_RMS is the relative REST URL to call the process flow.
It is of the form /resources/batch/processes/operator/<process flow name>.
processFlowAdminBaseUrlUserAlias is an entry key in 'externalVariables' map variable used by the Scheduler runtime and specifies the alias name for BDI Process Flow Admin's user credentials to access the process flow REST service.
The value for processFlowAdminBaseUrlUserAlias is specified during install time and gets stored in the BDI System Options.
Basic authentication is used to access the BDI process flows. The Scheduler uses processFlowAdminBaseUrlUserAlias to lookup the credentials in the runtime secure wallet where the credentials specified at install-time are stored.
Scheduler by itself does not manage executions of process flows called from within the schedule action and any dependencies associated thereof. The Scheduler only triggers process flows. The execution of process flows is done by the Process Flow engine.
For any dependencies between execution of process flows to be managed, it is recommended that such dependencies are defined in the BDI Process Flow Admin and not in the Schedule Action. For example, if process-flow-2 needs to be run after process-flow-1 completes, use Process Flow Admin to define this dependency and not in the Schedule Action.
It is recommended to avoid time based dependency management in execution of process flows from within the Scheduler, but rather use Process Flow Admin to coordinate such dependency execution requirements.
|
Note: For security reasons, usage of certain keywords like Thread, System, java etc are not allowed in the Schedule Action DSL. When defining the schedule action in the Scheduler UI, any such forbidden keywords if used will prevent the schedule from being created or updated. A schedule cannot be run if such keywords are present in the schedule action definition. |
There are two types of Schedule Action - Sync and Async. When creating a schedule and defining a schedule action, the user needs to specify whether the schedule action is sync or async. The Scheduler determines the action execution statuses according to the action type specified.
Executes synchronously and returns a result after its successful or failed completion (however long the action may run).
The action is asynchronous and returns a response immediately when triggered, but will continue to execute. The actual process completes at a later time. The end result of the action is not known to Scheduler in this case.
Indicates the status of execution of the schedule action when the schedule has run at the configured frequency of time.
A schedule execution can be in one of the following statuses depending upon the Schedule Action Type and its execution.
Triggered (applicable only for ’Async' action)
Started (applicable only for ’Sync' action)
Failed (applicable for both ’Async' and ’Sync' actions)
The Schedule action type determines the schedule action status during the execution lifecycle.
Sync Action Execution Statuses
Executes synchronously and returns a result after its successful or failed completion (however long the action may run).
When sync action starts, the Schedule Action status will marked 'STARTED'.
When the action completes and returns a successful result, the status will be marked 'COMPLETED'.
When the action does not complete because of an exception or returns a failed response (return value = "FAILED"), then the status will be marked 'FAILED'.
Async Action Execution Statuses
The schedule action status will only be 'TRIGGERED' when the Scheduler successfully invokes the schedule action.
In case there is an exception in invoking the action itself, then the status is 'FAILED'.
By default, all BDI process flows are asynchronous that return an execution Id when triggered, but continue to run to invoke the batch jobs that complete at a later time.
How the Action Execution Statuses are determined
The Scheduler marks the Action Execution Status as 'FAILED' when there is an exception in executing the action or when an exception is thrown from the schedule action. In order for the Scheduler to mark the execution of the schedule action as 'FAILED' when the action has been executed, the action should either throw an exception or return a value as 'FAILED'.
If the schedule action returns null or any other return value, the action execution status will be be marked 'TRIGGERED' for async action and 'COMPLETED' for sync action and the returned response is stored as such in the Schedule Action Execution Log.
A schedule can be in one of the following statuses:
Active: An Active schedule indicates that the schedule is running at the specified frequency.
Inactive: An Inactive schedule indicates that the schedule has reached its end datetime and no longer runs.
Disabled: A Disabled schedule indicates that the user has disabled the schedule to not run at its specified frequency.
This section describes the scheduler runtime.
As the Scheduler is deployed and the application starts up, the Scheduler service performs the following actions:
Loads the schedules defined in the seed data sql script in the installer. This means schedule definitions are inserted in the corresponding Scheduler infrastructure table.
Loads the schedule action dsl for each corresponding schedule from the *_Action.sch files in the installer. Each schedule definition in the table is updated to include its corresponding schedule action.
The Scheduler service sets up the default runtime timers for each schedule.
When the application is deployed first-time, all schedules will be setup new. However, when the application is redeployed or the application server is restarted, the schedule timers that are already created and exist, will not be recreated.
All BDI schedules are 'DISABLED' in seed data user can make the schedules active as per their requirement.
When a schedule action dsl contains any restricted keyword, the schedule will be 'Disabled' at startup and will not run. The User has to correct the schedule action definition from the Scheduler UI and enable the schedule to make it active.
The Scheduler uses the application server's implementation of Java EE compliant timer service to execute the schedules at runtime. When a schedule is created, Scheduler sets up a timer in the application server based on the schedule frequency configured. At each scheduled time, the application server invokes a callback method where the Scheduler will execute the schedule action.
Each schedule timer executes in a separate thread, so schedule executions do not block each other. Each schedule execution itself is run synchronously in its own thread, that is, the execution is blocked until it completes. But the schedule action can be specified to be asynchronous (Async action) or synchronous (Sync action) based on the action dsl defined for the schedule.
It is appropriate to specify a schedule action as 'async' when all the service calls made within the schedule action are non-blocking asynchronous calls and the action defined runs in different thread from that of the Scheduler.
If any of the service calls within the schedule action is a blocking synchronous call and the action is not defined to run in a separate thread, then the action type should be 'sync'.
Specifying the schedule action type 'async' or 'sync' based on the action dsl definition determines the runtime execution behavior and statuses of the schedule execution. This is explained below.
BDI process flow invocations (REST service calls) are asynchronous by default and the corresponding schedule actions are specified 'Async'.
The Process Flow does not block the calling thread (that is, the Scheduler) and returns immediately. The Process flow returns an Execution ID of the process execution instance as a future handle, but will continue to execute the activities defined in the DSL. The activities in BDI process flows may run for longer duration. Here, the Scheduler simply acts as a trigger for the process flows at the scheduled frequency of execution.
The actual status of the process flow instance may be completed or failed after its execution, but the status of schedule execution in Scheduler will remain 'TRIGGERED'. The execution of the process flow and the eventual status thereof will not be known to the Scheduler.
When the schedule action execution starts for async action, the action execution status is set to 'TRIGGERED', and the action is executed. As the action type is specified 'Async', the action should be non blocking, either returning a response immediately or not returning a response and continuing execution, but runs in a separate thread returning the control immediately.
The execution of the action and the eventual status thereof will not be known to the Scheduler. Once the control is returned, the schedule action execution ends but the status remains 'TRIGGERED'. In case of an exception when the action is triggered, the status is set to 'FAILED' and the execution ends.
When the schedule action execution is started for 'sync' action, the action execution status is set to 'STARTED'. As the action type is specified 'sync', the action is blocking and runs in the same thread as the schedule execution.
The schedule execution ends only when the action completes returning a response or throws an exception, thereby releasing the execution thread.
After the schedule action completes successfully, the status is set to 'COMPLETED'. But if the action return value is 'FAILED' or the action returns throwing an exception, the status is set to 'FAILED'.
For sync actions, the action execution status in Scheduler can indicate the actual execution status (either completed or failed) of the process that was executed.
All schedule timers created by the Scheduler are persistent. This enables a failover feature that in case of unexpected server shutdown or downtime, the missed schedules will be run once the server is back up. That is, the schedules that should have been run during the downtime, will be run as soon as the server is back up and the application is in running state.
|
Note: A missed schedule will be run only once, not as many times as it was missed during the downtime. For example, if a schedule is scheduled to run every 5 minutes and the application server is down for 15 minutes and restarted, the schedule will be run only one time and not 3 times. This is a feature supported by the Java EE container. |
The Scheduler supports email notification of scheduled runs at runtime. The available options of events for notifications on a scheduled run are:
Notify when the schedule action execution begins
This occurs when the schedule action execution is 'Started' for sync action and before triggering of action execution for async action.
Notify when the schedule action execution ends successfully
This occurs when the schedule action execution status is 'Triggered' for async actions and 'Completed' for sync actions.
Notify when the schedule action execution fails
This occurs when the status of schedule action execution is 'Failed', when one of the following occurs: An exception is caught in the Scheduler service itself, when an exception is thrown by the schedule action dsl, when the schedule action dsl returns the string 'FAILED'.
All schedule notifications are persisted to the BDI_EMAIL_NOTIFICATION table. There is a subtab Schedule Notifications added in Manage Configurations tab which displays all the notifications.
One notification icon appears right top corner of the screen adjacent to the user if there is any notification in PENDING status. User will be navigated to the Schedule Notifications subtab by clicking on the image.
User can modify the status to COMPLETED after going through the notification and click on save button so that next time it doesn't appear on the screen.
The Scheduler infrastructure relies on the following schema to store the schedule definitions and schedule executions.
| Table Name | Description |
|---|---|
| BDI_SCHEDULE_DEFINITION | This table contains all the schedule definitions created, including schedule frequency, schedule notification information and schedule action dsl for each schedule.
Seed data schedules are loaded in this table at deployment time during application startup. |
| BDI_SCHEDULE_EXECUTION | All schedule executions at runtime are persisted in this table. |
| BDI_EMAIL_ NOTIFICATION | All schedule email alerts are persisted in this table. |
| BDI_SYSTEM_OPTIONS | This table contains system-level global parameters as key-value pairs used by the Scheduler at runtime, such as, Process Flow Admin Base URL, Process Flow Admin User Alias, which are configured at install time by the user. User can also add system parameters to be made available to the schedule actions. |
| BDI_GROUP | This table stores group names and its attributes |
| BDI_GROUP_MEMBER | This table stores all group member details |
The Scheduler service captures all schedule executions at runtime and persist the execution instances in the corresponding infrastructure table.
Scheduler provides certain RESTful services to retrieve information about schedules and run the schedule manually.
All the below REST resources can be accessed by Monitor, Operator and Admin role users, except for the run-schedule-now service that can be accessed by Admin and Operator role users, but not by users with only the monitor role.
| REST Resource | Method | Description |
|---|---|---|
| /resources/discover | GET | Lists all the available Scheduler REST resources |
| /batch/schedules | GET | Returns all the schedules in the application (including active, inactive and disabled schedules) |
| batch/schedules/{scheduleName} | GET | Returns the schedule definition of the specified schedule |
| /batch/schedules/upcoming-schedules/days/{days} | GET | Returns the upcoming schedules from now to next number of {days} specified |
| /batch/schedules/upcoming-schedules | GET | Returns the upcoming schedules for the next 1 day from now |
| /batch/schedules/executions/{scheduleName} | GET | Returns all the historical schedule executions of the given schedule since the beginning |
| /batch/schedules/executions/past/days/{days} | GET | Returns the historical schedule executions of the given schedule for past number of {days} |
| /batch/schedules/executions/failed | GET | Returns all the failed executions for all the schedules since the beginning |
| /batch/schedules/executions/today | GET | Returns today's schedule executions starting from midnight today (12:00 a.m.) to now |
| /batch/schedules/executions/today/completed | GET | Returns today's schedule executions that are either in 'Triggered' status (for async actions) or in 'Completed' status (for sync actions), starting from midnight today (12:00 a.m.) to now |
| /batch/schedules/executions/today/failed | GET | Returns today's schedule executions that are in 'Failed' status, starting from midnight today (12:00 a.m.) to now |
| /batch/schedules/operator/run-schedule-now/{scheduleName} | POST | Runs the specified schedule, that is, executes the Schedule Action of the schedule and returns the Schedule Execution detail response.
This is synchronous invocation, so client needs to wait for the response. |
| /batch/schedules/activateOrDisable-schedules | POST | To update status of one or more schdules to ACTIVE or DISABLED |
The Scheduler Console (Admin UI) is a web user interface provided by the Scheduler where users can monitor and manage schedules, including creating, updating, deleting, disabling or enabling schedules, manually running schedules, viewing schedule executions and schedule logs.
The following describes various functions available in Scheduler Console in the current release.
|
Note: It is recommended to use Chrome web browser to access Scheduler Console since the calendar widget for datetime fields are supported by Chrome browser and not by Firefox or IE as of now. |
This is the home page that provides the overall summary of the scheduler runtime. It displays the following information.
This displays the total count of:
Active Schedules
Schedule Executions today
Schedule Executions that were successful today
Schedule Executions that failed today
|
Note: Today here indicates the duration from midnight to now. |
Lists the future schedules that are expected to run in the next 24 hours from now.
Lists the schedule executions that have failed today (from midnight to now).
Lists the schedule executions that are completed or triggered today (from midnight to now). A status of 'Completed' represents sync actions and the status of 'Triggered' represents async actions.
The Manage Schedules page displays a list of all schedules and details of each schedule in the Schedule Detail view and corresponding schedule executions in the Schedule Executions view for the schedule.
The schedules list provides options to filter schedules based on the Schedule Name, Schedule Group, Schedule Status, Schedule Frequency. There is also an option to filter upcoming schedules based on date range.
The 'Create Schedule' function will be available in this page for admin users.
The 'Create Schedule' option displays one page where the user can enter and save all required information to create a schedule. The page displays input fields under four sections as follows.
Schedule Name, Schedule Group and Schedule Description are entered under Basic Info. Schedule Name and Schedule Group are required fields.
Schedule Name must be unique. The User can choose an existing Schedule Group or add a new group name for the schedule.
There is limitation for the number of characters that these fields can accept.
Specify a valid schedule action definition here that will get executed when the schedule runs.
If any restricted keyword is present in the action definition, the schedule cannot be saved, and when saving the schedule an error highlighting the restricted keyword will be displayed.
Also choose here whether the schedule action is 'Async' (which is the default selected option) or 'Sync'.
|
Note: The schedule action is not validated or compiled for syntax when creating the schedule, so any syntax or programming error in the action definition will result in an exception at runtime and the schedule execution will fail. |
It consists of Schedule Start Datetime, End Datetime and Schedule Recurrence.
The Schedule End Datetime is 'Never' by default meaning the schedule never ends and repeats indefinitely. If the schedule has an end datetime, the user can enter a specific datetime.
The Start Datetime defaults to 5 minutes from the current time and the End Datetime defaults to 6 minutes from the current time when chosen.
The Start and End datetimes should be future dates. The Schedule End datetime if specified should be after the scheduled start datetime. These validations will be done when saving the schedule.
Scheduler provides two options to specify recurrence of the schedule - Simple Scheduling and Advanced Scheduling. Use the options tab to toggle between Simple and Advanced Scheduling options.
Simple Scheduling provides the following predefined schedules that the user can choose from a drop-down list.
Hourly
Daily (selected by default)
Weekly
Monthly
Every Weekday [Mon-Friday]
On Weekends [Sat-Sunday]
Every Saturday
Every Sunday
First day of every month
Last day of every month
One time only
Specify a different frequency
The User can use this option to specify a recurring interval in minutes, hours, days or weeks, for example, 30 minutes, 2 hours, 3 days, and so on.
Advanced Scheduling enables the user to specify complex schedules similar to a cron expression. The User can choose multiple values for Hours, Minutes, Day of Week, Day of Month and Month options using the multi-select lists.
The default schedule frequency here is daily midnight (Hours: 12 a.m., Minutes: 0 are the values selected by default).
Use the schedule notification option to enable email notification for the schedule when schedule execution starts or fails or is completed.
Enter valid email addresses for notification. When enabled, email alerts will be sent based on the options selected.
Starts:
When this option is chosen, email will be sent when the schedule execution starts, that is, when the schedule runs at the scheduled interval, and just before the execution of the schedule action.
Fails:
An Email will be sent when there is an exception in the schedule execution or when the schedule action throws an exception or returns a 'Failed' response. This means the schedule action execution will be in 'Failed' status.
Triggered / Completed:
An Email will be sent when the schedule action execution status is 'Triggered' (for async actions) and 'Completed' (for sync actions). This essentially means the schedule execution is successful.
|
Note: For schedule notification to work, the mail session needs to have been configured in the WebLogic server. Refer the BDI Installation Guide for details on the configuration of the mail session. |
A schedule can be updated by selecting the schedule from the Manage Schedules page and using the Edit option in Schedule Detail view.
The Edit page is the same as that of the Create Schedule page with the schedule information populated. Update the values as required in the relevant sections as explained previously for creating the schedule. Only admin users can edit a schedule.
|
Note: The updating schedule frequency will validate schedule start datetime and end datetime (if specified) similar to when creating a schedule. |
Updating any other details other than schedule frequency will not validate the existing schedule frequency as the schedule will continue to run at the already defined frequency and only the other details of the schedule definition will get updated as modified by the user.
When changing the schedule action definition, it will be verified for any restricted key-words.
A schedule can be disabled by selecting the schedule from the Mange Schedule page and using the 'Disable schedule' option in the Schedule Detail view. Only admin and operator users can disable a schedule.
Disabling a schedule will change the schedule status to 'Disabled' and the schedule will no longer run at the specified frequency. However the schedule can be manually run using the 'Run Schedule Now' option.
|
Note: The Inactive schedule cannot be disabled, as an inactive schedule has reached its end already and no longer runs. |
A disabled schedule can be enabled again using the 'Enable schedule' option from the Schedule Detail view. Only admin and operator users can enable a schedule.
Enabling the schedule will change the status of the schedule to 'Active' and the schedule will resume running at the specified frequency.
A schedule can be deleted using the 'Delete schedule' option in the Schedule Detail view. Only admin users can delete a schedule.
|
Note: Deleting a schedule will delete the schedule definition and also its entire execution history. The schedule will no longer exist and will not run after deletion. There is no way to restore a deleted schedule except by creating the schedule again. |
Any schedule can be manually run using the 'Run Schedule Now' option from the Schedule Detail view. Inactive and disabled schedules can also be manually run.
This option is provided so that the user can run a schedule on demand when required. Only admin and operators can access this function.
When the schedule is run manually, the schedule action is submitted for execution in the backend and the result of the execution can be seen from the Schedule Executions view.
From the Schedule Executions page, the user can view all available historical schedule executions. The page will display schedule executions for the last one week by default. The user can use the search option to enter a different date range to fetch the corresponding schedule executions.
Within the list of schedule executions, the records can be filtered based on the Schedule Name, Action Execution Status and any string within the Action Execution Log. The list of scheduled executions are sorted by schedule execution datetime, the latest first.
From the Manage Configurations page, user can manage log levels, notifications, and system options.
Log Level
The Log Level page displays log levels for all schedules. Users can change log level for one or more schedules.
Build version and date is displayed on the info icon when the user selects the same. The icon is on the extreme right top corner of the page.
Notifications
User can view/update notifications details from Scheduler Notifications page.
System Options
Users can add, update, or delete system options from the System Options page. Credentials can also be created when a system option is created.
The System Logs page displays a list of all schedule log files and log contents. Each schedule has its own log file enabling easy access for the user to view the execution logs and other information from the log files for diagnosing and troubleshooting issues.
The list of log files are sorted by the last modified time of file with most recently modified file first.
This section describes the scheduler security considerations.
The Scheduler application uses basic authentication to authenticate users and allow access to the requested resources based on authorization. Only valid users can access the Scheduler Console and the REST resources. The Scheduler accesses the BDI process flows using basic authentication.
Users need to belong to one of these roles:
Admin (assigned to BdiSchedulerAdminGroup in WebLogic Server)
Operator (assigned to BdiSchedulerOperatorGroup in WebLogic Server)
Monitor (assigned to BdiSchedulerMonitorGroup in WebLogic Server)
Only authorized users of specific role are allowed to access certain functionalities in the Scheduler Console.
Users of the Admin role have access to all the functions in Scheduler, the users of Operator role have limited authorizations to use only certain functions, and users of the Monitor role only have view/read-only access to the Scheduler Console.
| Function | Admin Role | Operation Role | Monitor Role |
|---|---|---|---|
| View and search | Yes | Yes | Yes |
| Create schedule | Yes | No | No |
| Edit schedule | Yes | No | No |
| Delete schedule | Yes | No | No |
| Manual run schedule | Yes | Yes | No |
| Disable schedule | Yes | Yes | No |
| Enable schedule | Yes | Yes | No |
This section describes the scheduler operational considerations.
The Scheduler application is secured with role based security authorization. It is recommended to use separate users for Monitor, Operator and Admin roles.
Schedules and executions can be effectively monitored using the Scheduler Console. The console provides detailed action execution log and log files for each of the schedules that can be used to verify the runtime executions of schedules and related information.
Each schedule execution contains a 'Schedule Action Execution Log' that provides descriptive information on the scheduled run or manual run of the schedule. The Schedule Action Execution Log provides information as follows.
<SCHEDULED or MANUAL> RUN: Action triggered at: <Date and time> Action Type: <ASYNC or SYNC> Action Status: <TRIGGERED or STARTED or COMPLETED or FAILED> Action Response: <The response string as returned by the schedule action dsl, or the error message in case of an exception>
For example, for a successful execution of schedule ItemHdr_Fnd_From_RMS_Schedule at the scheduled frequency, and action that triggers the process flow ItemHdr_Fnd_ProcessFlow_From_RMS, the Schedule Action Execution Log will be:
SCHEDULED RUN: Action triggered at: Wed Jul 27 12:00:01 EDT 2016
Action Type: ASYNC
Action Status: TRIGGERED
Action Response: {"executionId":"ItemHdr_Fnd_ProcessFlow_From_RMS#0d3d656d-041a-4068-8daf-8d17ee1ad899","processName":"ItemHdr_Fnd_ProcessFlow_From_RMS"}
In case of an exception (say, connection error when invoking process flow), the action execution log will look as follows:
SCHEDULED RUN: Action triggered at: Sat Aug 06 00:40:00 EDT 2016 Action Type: ASYNC Action Status: FAILED Action Response: Exception: java.net.ConnectException: Tried all: '1' addresses, but could not connect over HTTP to server: java.net.ConnectException: Connection refused Check the logs for more details.
The above action execution log examples indicate async actions. For sync actions, the the action execution log also shows when the schedule action started and when it completed, which is particularly useful for a long running action for which the Scheduler waits for the response until completion. For example,
SCHEDULED RUN: Action execution started at: Wed Aug 03 12:00:00 EDT 2016 Action Type: SYNC Action execution ended at: Wed Aug 03 12:22:10 EDT 2016 Action Status: COMPLETED Action Response: Batch process completed.
|
Note: The Action Response shows the value that the schedule action dsl finally returns after completion. |
Each schedule has its own log file. For example, a schedule named Store_Fnd_From_RMS_Schedule will have its log file named Store_Fnd_From_RMS_Schedule.log.
The log file contains detailed information on schedule executions which can be scheduled runs or manual runs, logs of actions such as disabling and enabling the schedule, action log on schedule updates such as change in schedule frequency, and in case of any exceptions, the exception stack trace.
Users can use the following keywords to search for specific information in the schedule log file.
| Keyword | Description |
|---|---|
| ScheduleId | The primary key Id of the schedule |
| ScheduleName | The schedule name |
| ScheduleExecutionId | The execution Id of schedule run instance |
| Action Execution Begin | Indicates the start of the log when schedule action begins. |
| Action Execution End | Indicates the end of the log when schedule action ends. The log of the schedule action execution can be found between the two strings: ***Schedule Run :
Action Execution Begin*** and ***Schedule Run : Action Execution End*** For manual run, it will be ***Manual Run : Action Execution Begin*** and ***Manual Run : Action Execution End*** |
| Action execution exception | The detailed exception message and stacktrace will be shown following this string, when an exception has occurred in schedule action execution. |
As the schedules run, schedule execution records are stored in the BDI_SCHEDULE_EXECUTION table.
This table will grow larger as the number of schedule executions increase. Hence it is recommended to periodically purge historical scheduled executions from the tables that are older and no longer necessary, and only retain recent schedule executions of a particular period, say for the last one month to now. This will help keep the table size within a certain limit and prevent database growth.
This section describes the scheduler customization.
The sql script containing the seed data schedule definitions is located in the bdi-scheduler-home/setup-data/dml folder.
During the initial deployment of the Scheduler application, seed data schedules get loaded to the schedule definition table and the corresponding schedules are created.
If the Scheduler application needs to be redeployed and the seed data schedules need to be reloaded during the redeployment (that is, to reset the schedules to the initial state as per seed data), set the LOADSEEDDATA column in the BDI_SYSTEM_OPTIONS table to TRUE, and undeploy and redeploy the application.
|
Note: The above redeployment procedure will reset the current schedule definitions (that is, existing schedules and any changes will be deleted) and the schedules will be recreated as per seed data definitions. Use this option with caution and only when absolutely necessary. |
By default all BDI seed data schedules are scheduled to run daily starting at midnight (each schedule running with a gap of 5 minutes). The User can edit the seed data and add new schedules to be loaded during deployment, by updating the seed data sql script and adding corresponding schedule action scripts in the bdi-scheduler-home install directory, before starting the installation.
Seed data sql file: bdi-scheduler-home/setup-data/dml/seed-data.sql Schedule Action dsl files: bdi-scheduler-home/setup-data/dsl
An insert statement for a schedule seed data definition will look as follows (SQL for Oracle database):
INSERT INTO BDI_SCHEDULE_DEFINITION (schedule_id, schedule_name, schedule_group, schedule_description, schedule_status, schedule_start_datetime, schedule_type, schedule_frequency, schedule_notification, schedule_notification_email, schedule_action_type, schedule_action_definition) VALUES (7, 'InvAvailStore_Tx_From_RMS_Schedule', 'Inventory', 'Schedule created from seed data. This schedule calls process flow: InvAvailStore_Tx_ProcessFlow_From_RMS.', 'ACTIVE', TIMESTAMP '2016-03-12 00:30:00', 'SIMPLE', 'DAILY', 'ON_SUCCESS,ON_ERROR', 'user@example', 'ASYNC', 'InvAvailStore_Tx_From_RMS_Schedule_Action.sch')
|
Note: When adding or editing schedule definitions in seed data to be loaded at application startup. All these fields (as shown in the sql statement above) are required fields to create a schedule at startup. |
schedule_id should be a unique number for each schedule.
schedule_name should be unique.
schedule_status needs to be ’ACTIVE' for schedule to be created and active.
schedule_type should be ’SIMPLE' with any of the schedule_frequency values mentioned above. Advanced schedule (calendar schedules with complex cron expression) is not supported through seed data during deployment.
schedule_start_datetime:
Need to be in the format yyyy-mm-dd hh:mm:ss For example, 2016-01-01 00:00:00, 2016-01-01 18:30:00
schedule_frequency:
Valid values are: DAILY, HOURLY, WEEKLY, MONTHLY, WEEKDAY, WEEKEND, SATURDAY, SUNDAY, FIRSTDAYOFMONTH, LASTDAYOFMONTH, ONCE
schedule_notification:
Valid values are: ON_START, ON_SUCCESS, ON_ERROR (separate multiple values by comma)
schedule_email:
Valid email-id for notification (separate multiple emails by comma). Email is required if schedule_notification is specified.
schedule_action_type:
Valid values are (based on the action specified): ’ASYNC' or ’SYNC'.
schedule_action_definition in seed data refers to the name of the corresponding schedule action dsl file (this will get loaded at startup).
Each schedule should have a corresponding schedule action dsl script defined. This will be the action that gets executed when the schedule runs.
To load the schedule action dsl during deployment, add the schedule action dsl file under bdi-scheduler-home/setup-data/dsl with file name convention: <Schedule Name>_Action.sch.
For example for adding a new schedule named Schedule_1, add schedule action dsl script Schedule_1_Action.sch. During deployment, the Scheduler will create Schedule_1 and update the schedule definition with the action script from the corresponding file Schedule_1_Action.sch.
The seed data schedules in the Scheduler are the schedules that call the BDI process flows provided out-of-the-box. The Schedule Actions define the REST calls to the BDI process flows.
In an enterprise implementation, there will be requirements to schedule batch processes, any recurring jobs or activities that are not BDI process flows. There can also be existing batch processes or services that need to be scheduled.
The Scheduler can be used for such scheduling requirements by defining appropriate Schedule Actions to invoke the services.
The Scheduler can be used to schedule RESTful services and as the Schedule Action is a DSL based on Groovy, valid Groovy or Java code can also be used within the action part that will be executed by the Scheduler based on the defined schedule.
The syntax for Schedule Action is as simple as follows.
action {
//your implementation goes here
}
The following Schedule Action syntax specifies how a REST service can be called from the Scheduler (assuming the REST resource does not require any authentication). The response from the REST service will be treated as string.
action {
(POST[<your REST service URL here>]) as String
}
This is a simple approach for scheduling existing and new services that can be exposed as REST services.
The Schedule Action syntax to call a REST service with authentication and with base URL configured in System Options will look as follows.
action {
POST[externalVariables.myRESTServiceBaseUrl + "/resources/myRESTresource"]^externalVariables.myRESTServiceBaseUrlUserAlias) as String
}
The externalVariables is the name of the variable used internally by the Scheduler to access system options parameters. Any parameters (key-values) configured in System Options can be accessed using the notation externalVariables.<my-system-option-parameter>
Admin users can utilize System Setting RESTful service to add or update system options parameters, and setting up credentials (stored in wallet) for any authentication to be used by the application. Refer Appendix E for details on the System Setting REST resources.
In the above example, the user can add system option parameters named 'myRESTServiceBaseUrl' with the REST resource base url value (for example, http://<myserverhost>:<port>/myapp) and 'myRESTServiceBaseUrlUserAlias' which will be the alias name to be used for authentication and the value of this parameter should be GET_FROM_WALLET:GET_FROM_WALLET to indicate that the corresponding credentials for the alias need to be obtained from the wallet during runtime by the application.
Any failure in schedule execution can be analysed in the Scheduler application by checking the Scheduler log files for the corresponding schedule.
If a schedule execution is 'FAILED' due to an exception response from the process flow, then the details of corresponding process flow execution instance, the exception details and any stack trace can be viewed in the corresponding process flow logs using the Process Flow Admin console for further troubleshooting.
|
Note: The schedule execution where the BDI process flow is called is only a trigger for the process flow execution, hence the actual execution of the process flow and the status and logs thereof can only be viewed in the BDI Process Flow Admin console. |
The Scheduler Console provides a calendar widget for datetime fields that are currently supported only by the Chrome browser. Hence it is recommended to use the latest version of the Chrome browser to access the Scheduler Console.
If any other browser is used that does not support the calendar widget for the datetime input, the datetime fields may appear as textbox. Users can enter the datetime input as text, but the value should be in the format of 'yyyy-MM-ddTHH:mm', for example, 2016-01-01T20:00. There is no loss of functionality due to this limitation however.
