5 Job Admin UI
The BDI Job Admin UI is a web application that provides the GUI for managing batch jobs and runtime.
The User Interface provides ability to:
-
Start/restart, and track status of jobs
-
Enable or Disable the jobs
-
Trace data
-
View diagnostic errors
-
Manage options at job and system level
-
View the logs
Job Admin UI Security
Security in the integration layer is a big concern for every retail enterprise. The security system should be open enough to allow trusted remote applications to integrate easily and, at the same time, lock down unauthorized remote access. To address security concerns, the Job Admin utilizes the security models allowed in the Oracle middleware and database systems.
Authentication
Both the Job Admin UI and REST Services are secured with SSL and basic authentication.
Authorization
The below mentioned roles are defined to restrict access to operations in Job Admin.
-
BdiJobAdminRole
-
BdiJobOperatorRole
-
BdiJobMonitorRole
There are three categories of users in Job Admin: Job Administrators, Job Operators, and Job Monitors. Batch jobs can be run from Job Admin UI or through the Batch REST service. Here are the operations that can be performed by the users based on their role.
| Function | Admin Role | Operator Role | Monitor Role |
|---|---|---|---|
|
Edit configuration from UI |
Yes |
No |
No |
|
Create/update/delete system options |
Yes |
No |
No |
|
Create/update/delete system credentials |
Yes |
No |
No |
|
View credentials |
Yes |
No |
No |
|
Run Jobs |
Yes |
Yes |
No |
|
Monitor jobs |
Yes |
Yes |
Yes |
Monitoring Batch Jobs Using BDI Job Admin
Batch jobs can be monitored using the Job Admin UI.
Batch Summary Tab
Figure 5-1 Batch Summary Tab
This tab shows the summary of the system and details about the latest batch job executions. It can be used to quickly find out whether the latest jobs are successful or not. The last section of this page displays the step summary of the selected job.
Manage Jobs Tab
This tab displays the list of available jobs with their details and allows you to Start a job, Restart failed jobs, enable or disable the jobs, list the executions of a job.
By default all the jobs are enabled. Select the job row and check/uncheck the check box of each job and click on save image button in enable column. Only enabled jobs can be launched/restarted. The Launch/Restart button is disabled for the disabled jobs. There is an option to enable or disable all the jobs at a time by clicking on checkbox, present in the enable column, highlighted in red and click on save image button.
Note:
Only enabled jobs can be launched or restarted.
The Launch or Restart button is disabled for the disabled jobs.
Figure 5-2 Manage Jobs Tab
Job Executions
This tab shows the executions of the selected jobs. It can be used to restart the failed executions of a job. The Restart button is available only for restartable executions in the status column. When the user clicks the restart button it is redirected to the job launch tab with the restart option and pre-populated value of the job parameters from last run of the execution. Only enabled jobs can be restarted otherwise the Restart button is disabled. User can edit the value of the existing parameters except the dataSetId and enter new parameters in comma separated format.
Note:
Editing the dataSetId during restart can result in errors.
Note:
The url is a infrastructure parameter, the user is not allowed to change its value.
DataSetId in job parameter is not supposed to be edited, and updating same during restart can result into errors.
Job Launch
This tab can be used to launch the jobs. Only enabled jobs can be launched. The Launch button is disabled for the disabled jobs. Job Parameters is an optional input to launch the jobs. Multiple job parameters can be entered in comma separated value format. On restart, the user is redirected to the Job Launch tab, and the launch button is replaced with the restart button. The Job parameters values are pre-populated from the last failed run of the instance. The user has an option to add or update existing key values, except dataSetId.
Figure 5-3 Job Launch
Job Details
This tab shows the details of the selected job such as Job Description, Family, Rest Service Url and Job Xml content.
Figure 5-4 Job Details
System Logs Tab
This tab shows logs at job and system level. If a job fails, the job level log provides details about the failure. Information about a job in the log file starts and ends with a banner that shows details such as job name, instance id, execution id and so on.
Figure 5-5 System Logs Tab
Sample Begin Job Banner
*******************************************************************************
2016-08-03T02:15:00,764 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - Beginning of Downloader JOB_NAME(Diff_Fnd_DownloaderAndTransporterToExtJob).
2016-08-03T02:15:00,764 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - INTERFACE_MODULE(Diff_Fnd) EXECUTION_ID(3844) INSTANCE_ID(3844).
2016-08-03T02:15:00,764 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - *******************************************************************************
Sample End Job Banner
*******************************************************************************
2016-08-03T02:15:02,080 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - End of Downloader JOB_NAME(Diff_Fnd_DownloaderAndTransporterToExtJob).
2016-08-03T02:15:02,080 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - INTERFACE_MODULE(Diff_Fnd) EXECUTION_ID(3844) INSTANCE_ID(3844).
2016-08-03T02:15:02,081 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - Total time for Downloader job: 1 seconds.
2016-08-03T02:15:02,081 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - *******************************************************************************
It also shows whether it processed a data set or not. Here are the keywords that can be used to search the job level log files.
| Key Word | Description |
|---|---|
|
JOB_NAME |
Name of the job |
|
EXECUTION_ID |
Execution Id of the job |
|
INSTANCE_ID |
Instance Id of the job |
|
TRANSACTION_ID |
Transaction Id (Tx#<Job Instance Id>_<Current Time in millis>_<Source System Name>) |
|
INTERFACE_MODULE |
Name of the interface module |
|
INTERFACE_SHORT_NAME |
Name of the interface |
|
PROCESSING_DATA_SET |
Indicates and shows the details about the data set that job is processing |
|
DATA_SET_PROCESSED |
Indicates that the job successfully processed the data set |
|
DATA_SET_FAILED |
Indicates that the job failed to process the data set |
Diagnostics Tab
This tab shows general job level error information such as error description, stack trace etc as well as job specific error information. Use this tab to identify where a job failed and fix the issue.
Outbound Job Execution Errors
Figure 5-6 Outbound Job Executions Errors
The following information is displayed for outbound job execution errors. There can be multiple outbound job execution errors for a job instance.
| Field Name | Description |
|---|---|
|
Partition Index |
Partition in which the error occurred |
|
Block Number |
Block in which the error occurred |
|
Begin Sequence Number |
Beginning sequence number in the block |
|
End Sequence Number |
Ending sequence number in the block |
Inbound Job Execution Errors
The following information is displayed for inbound job execution errors. There can be multiple inbound job execution errors for a job instance.
| Field Name | Description |
|---|---|
|
File Name |
Name of the file in which the error occurred |
|
Begin Row Number |
Beginning row number in the file |
|
End Row Number |
Ending row number in the file |
Trace Data
This tab shows data movement in the BDI. Use this tab to verify that data moved from the sender to destination inbound tables.
Sender Data
Figure 5-7 Sender Data
This tab shows the following information about the sender data by the sender side Job Admin (for example rms-batch-job-admin)
| Field Name | Description |
|---|---|
|
Transaction Id |
Transaction Id (Tx#<Job Instance Id>_<Current Time in millis>_<Source System Name>) of the job |
|
Interface Module |
Name of the interface module |
|
Job Name |
Name of the batch job |
|
Data Set Ready Time |
Time when sender moved the data outbound tables |
|
Data Set Type |
Type of data set (FULL or PARTIAL) |
|
Status |
Status of the job (COMPLETED or FAILED) |
|
Transaction Duration |
Duration of the job |
Receiver Data
Figure 5-8 Receiver Data
This tab shows the following information about the receiver data by the destination application (for example rms-batch-job-admin).
Receiver Transactions
| Field Name | Description |
|---|---|
|
Source Transaction Id |
Transaction Id of the sender job |
|
Source System |
Name of the sender |
|
Family |
Name of the interface module |
|
Transaction Status |
Status of the transaction (COMPLETED or FAILED) |
|
Duration |
Time it took to send data to Receiver |
|
Source System URL |
URL of the source job |
Receiver Transmission Details - Partition Level
| Field Name | Description |
|---|---|
|
Transmission Id |
Transmission Id of the partition |
|
Family |
Name of the interface module |
|
Interface Short Name |
Name of the interface |
|
Source System Partition Name |
Partition Number |
|
Partition Begin Sequence Number |
Beginning sequence number in the partition |
|
Partition End Sequence Number |
Ending sequence number in the partition |
|
Begin Block Number |
Beginning block number in the partition |
|
End Block Number |
Ending block number in the partition |
|
Status |
Status of the transmission (COMPLETED or FAILED) |
|
Duration |
Time it took to send data for a partition |
Receiver Transmission Details - Block Level
| Field Name | Description |
|---|---|
|
Block Number |
Block Number in a partition |
|
Block Item Count |
Number of items in the block |
|
Block Status |
Status (COMPLETED or FAILED) |
|
File Location |
Location of the file for the block |
Inbound job Executions
| Field Name | Description |
|---|---|
|
Transaction Id |
Transaction Id (Tx#<Job Instance Id>_<Current Time in millis>_<Source System Name>) of the uploader job |
|
Remote Transaction Id |
Transaction Id of the downloader job |
|
Interface Module |
Name of the interface module (for example Diff_Fnd) |
|
Source System |
Name of the source system (for example RMS) |
|
Data Set Type |
Type of data set (FULL or PARTIAL) |
|
Source Sys Data Set Ready Time |
Time when source system moved data set to outbound tables |
|
Data Set Ready Time |
Time when uploader job uploaded data set to inbound tables |
|
File Merge Level |
Merge level of the file (NO_MERGE, MERGE_TO_PARTITION_LEVEL, MERGE_TO_INTERFACE_LEVEL) |
|
Status |
Status of uploader job (COMPLETED or FAILED) |
Importer Data Control
| Field Name | Description |
|---|---|
|
Interface Short Name |
Name of the interface |
|
Begin Sequence Number |
Begin sequence number of data set in the inbound table |
|
End Sequence Number |
End sequence number of data set in the inbound table |
|
Data Partition |
Number of partitions used by uploader job |
|
Thread |
Number of threads used by uploader job |
|
Merge Strategy |
Merge strategy used for merging files |
|
Auto Purge Data |
Flag that indicates whether files need to be cleaned up or not |
Importer Job Executions
| Field Name | Description |
|---|---|
|
Transaction Id |
Transaction Id (Tx#<Job Instance Id>_<Current Time in millis>_<Source System Name>) of the uploader job |
|
Interface Module |
Name of the interface module (for example Diff_Fnd) |
|
Source System |
Name of the source system (for example RMS) |
|
Source Data Set ID |
Generated by extractor job for group of data |
|
Data Set Ready Time |
Time when importer job uploaded data set to inbound tables |
|
Source Sys Data Set Ready Time |
Time when source system moved data set to inbound tables |
|
Data Set Type |
Type of data set (FULL or PARTIAL) |
|
Status |
Status of importer job (COMPLETED or FAILED) |
Importer Data Control for Interface
| Field Name | Description |
|---|---|
|
Interface Short Name |
Name of the interface |
|
Begin Sequence Number |
Begin sequence number of data set in the inbound table |
|
End Sequence Number |
End sequence number of data set in the inbound table |
|
Record Count - Sent by Remote App |
Record count sent by remote application |
|
Record Count - Received |
Record count received |
Manage Configurations
This tab allows you to view and edit configurations for the BDI jobs, and it also allows the user to view, edit and create System Options.
Outbound Interface Controls
Figure 5-10 Outbound Interface Controls
This tab allows the user to manage the outbound interfaces and downloader and transmitter options for BDI jobs. The user with Admin privileges can edit the configurations.
Inbound Interface Controls
Figure 5-11 Inbound Interface Controls
This tab allows the user to manage the inbound interfaces, receiver and uploader options for BDI jobs. The user with Admin privileges can edit the configurations.
System Options
Figure 5-12 System Logs
This tab allows the user to view, edit and create system options. This page displays the list of system options of the application. The user can modify the value of the existing system options, create new system options and delete the existing system options. The user needs admin privileges for editing and creating system options. The Search option based on system options name and value is also provided on this page.
Job Admin Troubleshooting
This section describes the job admin errors and its troubleshooting.
BDI apps deployment Error
Issue:
Bdi Job Admin deployment can run into this error if database credentials are invalid:
Caught: javax.management.RuntimeMBeanException: java.lang.RuntimeException: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
javax.management.RuntimeMBeanException: java.lang.RuntimeException: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
at weblogic.utils.StackTraceDisabled.unknownMethod()
Caused by: java.lang.RuntimeException: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
... 1 more
Caused by: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
... 1 more
Caused by: java.lang.NullPointerException
... 1 more
Solution:
Undo all changes in the Weblogic domain session. Redeploy app with setting up new credentials and verify deployment is successful.
BDI Job Admin runtime WSMException
Issue:
Log files contain this exception:
oracle.wsm.common.sdk.WSMException: WSM-07620 : Agent cannot enforce policies due to either failure in retrieving polices or error in validations, detail= "WSM-02557 The documents required to configure the Oracle Web Services Manager runtime have not been retrieved from the Policy Manager application (wsm-pm), possibly because the application is not running or has not been deployed in the environment. The query "&(@appliesTo~="REST-CLIENT()")(policysets:global:%)" is queued for later retrieval.
Solution:
Follow BDI Installation guide, and verify WSM- policy manager is configured for admin server URL.
Open weblogic domain console and Target wsm-pm app to Admin Server.
Bounce Admin server and verify wsm-pm app is in Active State.
REST Service from SOAP UI for Downloader and Transporter job
Issue:
Diff_Fnd_DownloaderAndTransporterJob is successful, Job status is "completed" but data not transferred from outbound to inbound table and .csv file not created
Rest call to DownloaderAndTransporterJob is successful, Job status is "completed" but data not transferred from the outbound to inbound table and .csv file not created
Solution:
-
Verify the receiverEndpointUrl for the Table DownloaderTransmitterOptions is updated to point to where receiver app is deployed in my case ‘blr00abi.idc.oracle.com:7001' in bdi_rms_seed_data.sql.
-
Verify the values in the Interface table DownloaderInterfaceDataControl such as begin and end sequence number matches with the values mentioned in bdi_rms_seed_data.sql.
-
Verify the values in the interface table in DB DownloaderTransmitterOptions,the receiverEndpointUrl is updated to match with bdi_rms_seed_data.sql.
BDI Job Admin not able to find UploaderJob.xml file
Issue:
BDI App B (EXT) Job Admin GUI is showing this exception:
Caused By: java.lang.RuntimeException: Could not find jobName(OrgHier_Fnd_UploaderJob) xml file. You may have renamed
the job file or your job repository has more jobs than your application. To resolve the issue either delete the job repository
or add the correct job xml file to the app.
Managed server log contains:
Truncated. see log file for complete stacktrace
Caused By: java.lang.RuntimeException: Could not find jobName(OrgHier_Fnd_UploaderJob) xml file. You may have renamed
the job file or your job reposiotry has more jobs than your application. To resolve the issue either delete the job repository
or add the correct job xml file to the app.
at com.oracle.retail.bdi.batch.job.operator.JobOperatorServiceBean.allAvailableBatchJobs(JobOperatorServiceBean.java:167)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
Truncated. see log file for complete stacktrace
Solution:
The process flow has changed and part or all of a flow has been removed, but batch-db has not been updated to match. Either log in to the database and delete references to the job from all tables, or recreate the batch-db using RCU and redeploy BDI.
Job Fails and Job Admin Log Files Contain No Details of the Failure
Issue:
A job fails and the Job Admin log files contain no evidence of or details about the failure.
Solution:
Take a look at the WebLogic Server log files to identify the root cause of the job failure. One example of this is improper data source configuration.