| Oracle® Retail Bulk Data Integration Implementation Guide Release 16.0 E81413-01 |
|
 Previous |
 Next |
| Oracle® Retail Bulk Data Integration Implementation Guide Release 16.0 E81413-01 |
|
Previous |
Next |
A process flow is a composition of one or more activities. It is written in a DSL script that contains all the activities that make a data flow from source to destination complete.
A process flow is a generic concept and is not limited to BDI. However all the out-of-box process flows are for data transfers from a retail application to one or more retail applications.
A process flow encapsulates a sequence of activities. An activity can be synchronous or asynchronous. In BDI some of these activities are invocations of batch jobs.
This section describes the process flow definitions.
Process flow definition is specified in a Domain Specific Language (DSL) built on the top of Groovy. Since Groovy is built on the top of Java Virtual Machine (JVM) Groovy can understand Java and Groovy language constructs. Hence the process flow DSL can understand the DSL, Groovy and Java language constructs. A process is a list of activities. "begin", "end" and "activity" are the main DSL keywords used in process flow definition. These are described in detail below.
The "begin" activity in process flow definition appears as the first activity. There should be only one "begin" activity. The out of the box process flows may not contain any executable statements in this activity. This activity is intended to be the one used for any initialization needed for the process flow.
Activity has two parts. Name and Action. Name attribute is mandatory and should be used to name the activity.
The Action section is where the executable code should reside. Any Groovy or Java code can be coded in this section.
There can be one or more Activities in a process.
The "end" activity in the process flow definition appears as the last activity. There should be only one "end"activity. The out-of-the-box process flows may not contain any executable statements in this activity. This activity is intended to be the one used for any finalization needed for the process flow.
Variables used between activities can be created and stored in the processVariables map. The process engine also uses some of the variables for its own working in the process variable map. These variables are prefixed with "bdi_internal_". These variables must not be modified inside any custom code.
Here is how you can use the process variable map for your own use.
// Set Variable processVariables["VariableName"] = "Some Value" // Use a variable value def anotherVariable = processVariables["VariableName"]
Some of the system level configuration values are available in the externalVariables map. These values are read-only. The process flow DSL can use these values, but should not attempt to change it.
For example, externalVariables["rxmJobAdminBaseUrlUserAlias"]
Each activity instance and the process instance maintain the status of execution in the process schema. The following are the possible values for Activities and Process.
At the "begin" activity, the process is marked as PROCESS_STARTED. If any activity fails, the process is marked as PROCESS_FAILED. After the "end" action is completed, the process is marked PROCESS_COMPLETED.
Similar to process statuses, each activity has also a status. There values are ACTIVITY_STARTED, ACTIVITY_FAILED, ACTIVITY_COMPLETED. All the runtime status are persisted in the process schema at runtime when the DSL is executed.
When the process engine executes the process flow, the before and after snapshots of the Activity is recorded in the process schema.
The information is reported through Process Flow Monitor application. This is useful for tracking the process flows as well as troubleshooting. The snapshots also help in case of restarting a failed process. From the schema, the process engine can recreate the context to execute a restart and can resume execution from the activity that failed in the previous run.
Process Flow (Admin UI) is a web user interface provided by Process Flow where users can view and execute processes, including managing, updating process flow, manually running processes, viewing process executions and process flow logs.
The following describes various functions available in the Process Flow UI in the current release.
|
Note: It is recommended to use the Chrome web browser to access Process Flow UI since the calendar widget for datetime fields are supported by Chrome browser and not by Firefox or IE as of now. |
The Process Flow Live tab shows the details of the currently running processes. The first section shows the summary of all processes running in the system. The next section shows the list of all processes running since midnight. The last section shows the activity details of the selected process. Users also have the option to search for a process by its name.
The Manage Process Flow tab allows the user to Start a process flow, Restart a failed process flow, View/Edit a process flow, List the executions instances of a process flow. User can search process details on this tab. A failed process flow instance can be restarted only if it is the latest failed instance and there are no successful executions after that. A process flow can be edited only by a user with Admin privileges.
This tab shows the executions of the selected process. It can be used to restart the failed executions of a process. The Restart button is available only for restartable executions in the status column. When the user clicks the restart button it is redirected to the process launch tab.
This tab can be used to launch the selected process. Process Parameters is an optional input from the user to launch the process. Process parameter acts as query parameter and refers to a key value pair. Multiple process parameters can be entered in comma separated value format.
This tab shows process definition in form of a DSL file configured during deployment of the selected process. The Admin user also has the option to modify process DSL. Once updated the process DSL from the UI, changes will take into effect into the BDI_PROCESS_DEFINITION table and no need for process redeployment.
The Historical Process Flow Execution tab allows the user to look at the history of process flow executions. The user can specify a date, a time interval and process status. The application will list all the process flow executions matching the criteria. The User can select any of the flow to see the activities details of that execution instance. The page also provides the option to view the before and after values of all process variables for each activity.
The Manage Configurations tab allows users to view, edit and create system options. This page displays the list of system options of the application. The User can modify the value of the existing system options, create new system options and delete the existing system options. The User needs admin privileges for editing and creating system options. The Search option based on the system options name and value is also provided on this page.
The System Logs tab shows all the log files created by the process flow execution. Clicking on the View icon will show the log file contents in the screen.
The Process Flow notification options can be set in the System Options of the Process Flow. This can be done either at deployment time (through seed data) or at runtime (through the Manage Configuration tab of the Process Flow Monitoring application)
The options available for notification are:
processFlowNotification.<scope>.enable - value must be True or false. This is for global enabling or disabling of process flow notification.
processFlowNotification.<scope>.onStart - value must be True or false. True means notification will be sent at the start of the process.
processFlowNotification.<scope>.onRestart - value must be True or false. True means notification will be sent at the restart of the process.
processFlowNotification.<scope>.onCompletion - value must be True or false. True means notification will be sent at the completion of the process.
processFlowNotification.<scope>.onFailure - value must be True or false. True means notification will be sent when the process fails.
processFlowNotification.<scope>.recipients - list of recipient email ids
processFlowNotification.<scope>.subject – Template of the email subject line
processFlowNotification.<scope>.content – template of email content
where <scope> value is global or the Process Name.
If Process Name is specified, the global notification option is ignored for that process. For Subject and Content, if nothing is specified either at the global or process scope, an internal default format is used.
If Mail Session is not setup in WebLogic, notifications will not be sent. If processFlowNotification.<scope>.recipients is not set, the value from mail.to property in the WebLogic Mail Session is used.
For Subject and Content template, following variables can be used. The variable is case sensitive and the format must match exactly as given below. For multi-line content, \n can be used to indicate line breaks.
${processUrl}
${processName}
${processExecutionId}
${processStartTime}
${processEndTime}
${processStatus}
When the activities within a process flow fail, the process status is marked as failed. A failed process flow can be restarted. If there are multiple failed processes, only the latest failed instance can be restarted.
|
Note: Restart is for an already run and failed instance. This is different from running a new instance of the process flow. |
When a process flow is restarted, the system knows the activity that failed in the previous run. During restart, the process engine will skip all the activities prior to the failed activity. It will restore the context for the activity and resume execution at the failed activity.
Process flow execution does not keep the activity history at restart. It will overwrite the activity records on restart.
The Process Flow Application uses basic authentication to access the system. The user must belong to the BdiProcessAdminGroup or BdiProcessOperatorGroup or BdiProcessMonitorGroup to use the process flow REST services and process flow admin application.
There are three authorization roles designed for process flow application; Admin Role, Operator Role and Monitor Role. The Admin role has permissions to use all the functions provided by the process flow application. The Operator Role has limited access compared to Admin. The Monitor role has the least access permissions from all roles, as identified in the table below.
| Service/Action | Monitor Role | Operator Role | Admin Role |
|---|---|---|---|
| Update Process DSL | No | No | Yes |
| Start/Restart Process | No | Yes | Yes |
| All other services | Yes | Yes | Yes |
This section describes the customizing process flows.
The Process Flow is written in a custom DSL for process. This DSL allows a limited set of keywords to define a process. These keywords are identified in the table below. The execution section (Action keyword) can be written in Groovy or Java, since the DSL is developed on the top of Groovy.
| Keyword | Description |
|---|---|
| process | Identifies the process flow. Only one keyword in a process flow. |
| name | Used for naming processes and activities |
| var | Used for initializing process variables |
| begin | A special activity that occurs at the beginning of the process execution. Only one begin activity per process flow |
| action | The main executable section of the Activity. The body of Action can be in Groovy or Java |
| on .. moveTo .. | Jump to a specific activity on matching the condition. |
| activity | The executable component of the process flow. A process flow is composed of many activities. |
| end | A special activity that occurs at the end of the process execution. Only one begin activity per process flow |
The process flow engine also provides a few APIs specific to BDI batch jobs. The DSL writers can use these in the activity section of the script.
This API is used to start (or restart) a Retail Batch job. If the job has failed in the previous run of the process, the job will be restarted.
When an activity in a process fails, the activity name is persisted. On the subsequent restart of the process, all the activities ahead of the failed activity are skipped.
The execution begins at the failed activity. If the activity is calling a Retail Batch job, it will call the "restart", instead of new start, in this case. The batch job will then resume execution from the previously failed position.
startOrRestartJob( externalVariables["rxmJobAdminBaseUrl"], "WhAddr_Fnd_UploaderJob", externalVariables["rxmJobAdminBaseUrlUserAlias"],processVariables["processDataSetId")
This API will cause the process instance to wait until the job it is waiting for returns a FAILED or COMPLETED status.
If the job does not return either of the statuses, this API will wait until JTA timeout set in the server.
If the job returns a FAILED status, the activity whose name is specified in the second parameter, will be marked as failed.
For all other errors, including system exceptions, the activity containing the waitForJobCompletedOrFailed API will be marked as failed.
waitForJobCompletedOrFailed( "ItemSuppUom_Fnd_UploaderActivity", externalVariables["externalJobAdminBaseUrl"] + "/resources/batch/jobs/ItemSuppUom_Fnd_UploaderJob/" + processVariables["jobExecutionId"], externalVariables["externalJobAdminBaseUrlUserAlias"])
A process flow can be modified at deployment time. At deployment through the Process Flow Admin app the flow files that come with the application are in the setup-files/dsl/available_process_flow_options folder. These files have an extension ".flo". The user can edit these files in any text editor.
After editing the file save the file to the setup-files/dsl/flows-in-scope folder. The deployment script will take the process flow file and save in the process flow schema.
After deployment, the process flow can be edited by the Admin user through the Process Flow Admin application. The changes will be picked up at the next run.
It is recommended to make any permanent changes at deployment time, since the change through the Admin App may get overwritten at redeployment.
|
Note: For security reasons, usage of certain keywords are not allowed in the Process Flow DSL. When defining the process action in the process flow UI, any such forbidden keywords if used will prevent the process from being created or updated. A process cannot be saved or run if such keyword is present in the process action definition. |
In multi-destination process flows, one process may invoke one or more processes asynchronously. All the processes may run at the same time.
In order to identify these subprocesses they are named accordingly. Once invoked, the main process has no control over the sub processes. Each of the process will run in the same way as they are invoked independently.
The process instrumentation captures the state of the process at the beginning and end of each activity. This information is persisted into the process schema. For each activity there will be two records, one for before activity and the other for after activity. The schema details are in the Appendix B.
| Table Name | Description |
|---|---|
| BDI_PROCESS_DEFINITION | This table stores all the process flow definitions. It is loaded at deployment time. |
| BDI_PROCESS_EXEC_INSTANCE | This table tracks all the process flow executions. There is a row for each process flow execution. |
| BDI_ACTIVITY_EXEC_INSTANCE | This table tracks all the activity executions. There are 2 rows for each activity execution. One to store the before context and one to store after context |
| BDI_SYSTEM_OPTIONS | This table has all the system level information such as URLs, credential aliases and so on. It is loaded at deployment time. |
Process Flow services are exposed as REST endpoints for the use of other applications. The list of REST endpoints are given in the Appendix C
Since the process flow can be written in Groovy and DSL, it is prone to programmer's mistakes. Any custom DSL must be properly tested before deploying. At present, the process flow engine can detect syntax errors only at runtime. So it is possible to load an incorrect process flow and fail during runtime.
At the end of an activity, the process engine invokes the next activity depending on the result of activity execution (The "moveTo" statement). If you have empty activities (possibly because you commented out the existing invocation statements), make sure the activity result is valid (for example, "okay")
If any activity fails, the process is marked as failed. So in case of process failure, look at the activity details to find out which activity failed. Once the failed activity is identified, the process variables can be inspected to look for any issues. Next step would be to look at the logs, through the Process Flow Monitor application to see the details of the issue. Once the issue is fixed, either restart or a new run of the process flow can be used depending on the requirement.
Error
BDI Process Flow fails and GUI is showing this exception:
Runtime Process Flow exception
[Thread-55] ERROR Logger$error$0 - Error calling activity. javax.ws.rs.ProcessingException: Unable to unmarshall json object to java object.
OR
Caused by: javax.xml.bind.UnmarshalException - with linked exception:
[Exception [EclipseLink-25004] (Eclipse Persistence Services - 2.6.1.v20150916-55dc7c3): org.eclipse.persistence.exceptions.XMLMarshalException.
Exception Description: An error occurred unmarshalling the document
Internal Exception: javax.json.stream.JsonParsingException: Unexpected char 73 at (line no=1, column no=1, offset=0)]
Reason
Process flow deployed with wrong credentials for apps.
Solution
Delete existing process flow deployment from weblogic domain. Redeploy process flow with setting up new credentials.
Issue:
BDI Process Flow keeps in running status and does not end with failed or completed state. This even does not allow to cancel an existing running process or start a new process.
Reason:
This happens because of default JTA Timeout in domain configuration, and resource connections not able to timeout. There are instructions in BDI installation guide "How to Set JTA Timeout".
Resolution:
Follow the instructions in the BDI Implementation guide and set JTA timeout. Redeploy the processflow app to stop the running flow and rebounce the server.
