| Oracle® Retail Bulk Data Integration Implementation Guide Release 16.0 E81413-01 |
|
 Previous |
 Next |
| Oracle® Retail Bulk Data Integration Implementation Guide Release 16.0 E81413-01 |
|
Previous |
Next |
The BDI Job Admin UI is a web application that provides the GUI for managing batch jobs and runtime.
The User Interface provides ability to:
Start/restart, and track status of jobs
Trace data
View diagnostic errors
Manage options at job and system level
View the logs
Security in the integration layer is a big concern for every retail enterprise. The security system should be open enough to allow trusted remote applications to integrate easily and, at the same time, lock down unauthorized remote access. To address security concerns, the Job Admin utilizes the security models allowed in the Oracle middleware and database systems.
Both the Job Admin UI and REST Services are secured with SSL and basic authentication.
The below mentioned roles are defined to restrict access to operations in Job Admin.
BdiJobAdminRole
BdiJobOperatorRole
BdiJobMonitorRole
There are three categories of users in Job Admin: Job Administrators, Job Operators, and Job Monitors. Batch jobs can be run from Job Admin UI or through the Batch REST service. Here are the operations that can be performed by the users based on their role.
| Function | Admin Role | Operator Role | Monitor Role |
|---|---|---|---|
| Edit configuration from UI | Yes | No | No |
| Create/update/delete system options | Yes | No | No |
| Create/update/delete system credentials | Yes | No | No |
| View credentials | Yes | No | No |
| Run Jobs | Yes | Yes | No |
| Monitor jobs | Yes | Yes | Yes |
Batch jobs can be monitored using the Job Admin UI.
This tab shows the summary of the system and details about the latest batch job executions. It can be used to quickly find out whether the latest jobs are successful or not. The last section of this page displays the step summary of the selected job.
This tab displays the list of available jobs with their details and allows you to Start a job, Restart failed jobs, list the executions of a job.
This tab shows the executions of the selected jobs. It can be used to restart the failed executions of a job. The Restart button is available only for restartable executions in the status column. When the user clicks the restart button it is redirected to the job launch tab with the restart option and pre-populated value of the job parameters from last run of the execution. User can edit the value of the existing parameters except the dataSetId and enter new parameters in comma separated format.
|
Note: Editing the dataSetId during restart can result in errors. |
|
Note: The url is a infrastructure parameter, the user is not allowed to change its value. nDataSetId in job parameter is not supposed to be edited, and updating same during restart can result into errors. |
This tab can be used to launch the jobs. Job Parameters is an optional input to launch the jobs. Multiple job parameters can be entered in comma separated value format. On restart, the user is redirected to the Job Launch tab, and the launch button is replaced with the restart button. The Job parameters values are pre-populated from the last failed run of the instance. The user has an option to add or update existing key values, except dataSetId.
This tab shows the details of the selected job such as Job Description, Family, Rest Service Url and Job Xml content.
This tab shows logs at job and system level. If a job fails, the job level log provides details about the failure. Information about a job in the log file starts and ends with a banner that shows details such as job name, instance id, execution id and so on.
*******************************************************************************
2016-08-03T02:15:00,764 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - Beginning of Downloader JOB_NAME(Diff_Fnd_DownloaderAndTransporterToSimJob).
2016-08-03T02:15:00,764 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - INTERFACE_MODULE(Diff_Fnd) EXECUTION_ID(3844) INSTANCE_ID(3844).
2016-08-03T02:15:00,764 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - *******************************************************************************
*******************************************************************************
2016-08-03T02:15:02,080 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - End of Downloader JOB_NAME(Diff_Fnd_DownloaderAndTransporterToSimJob).
2016-08-03T02:15:02,080 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - INTERFACE_MODULE(Diff_Fnd) EXECUTION_ID(3844) INSTANCE_ID(3844).
2016-08-03T02:15:02,081 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - Total time for Downloader job: 1 seconds.
2016-08-03T02:15:02,081 [[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO DownloaderInterfaceJobListener - *******************************************************************************
It also shows whether it processed a data set or not. Here are the keywords that can be used to search the job level log files.
| Key Word | Description |
|---|---|
| JOB_NAME | Name of the job |
| EXECUTION_ID | Execution Id of the job |
| INSTANCE_ID | Instance Id of the job |
| TRANSACTION_ID | Transaction Id (Tx#<Job Instance Id>) |
| INTERFACE_MODULE | Name of the interface module |
| INTERFACE_SHORT_NAME | Name of the interface |
| PROCESSING_DATA_SET | Indicates and shows the details about the data set that job is processing |
| DATA_SET_PROCESSED | Indicates that the job successfully processed the data set |
| DATA_SET_FAILED | Indicates that the job failed to process the data set |
This tab shows general job level error information such as error description, stack trace etc as well as job specific error information. Use this tab to identify where a job failed and fix the issue.
The following information is displayed for outbound job execution errors. There can be multiple outbound job execution errors for a job instance.
| Field Name | Description |
|---|---|
| Partition Index | Partition in which the error occurred |
| Block Number | Block in which the error occurred |
| Begin Sequence Number | Beginning sequence number in the block |
| End Sequence Number | Ending sequence number in the block |
The following information is displayed for inbound job execution errors. There can be multiple inbound job execution errors for a job instance.
| Field Name | Description |
|---|---|
| File Name | Name of the file in which the error occurred |
| Begin Row Number | Beginning row number in the file |
| End Row Number | Ending row number in the file |
This tab shows data movement in the BDI. Use this tab to verify that data moved from the sender to destination inbound tables.
This tab shows the following information about the sender data by the sender side Job Admin (for example bdi-rms-batch-job-admin)
| Field Name | Description |
|---|---|
| Transaction Id | Transaction Id (Tx#<Job Instance Id>) of the job |
| Interface Module | Name of the interface module |
| Job Name | Name of the batch job |
| Data Set Ready Time | Time when sender moved the data outbound tables |
| Data Set Type | Type of data set (FULL or PARTIAL) |
| Status | Status of the job (COMPLETED or FAILED) |
| Transaction Duration | Duration of the job |
This tab shows the following information about the receiver data by the destination application (for example bdi-rxm-batch-job-admin).
| Field Name | Description |
|---|---|
| Source Transaction Id | Transaction Id of the sender job |
| Source System | Name of the sender |
| Family | Name of the interface module |
| Transaction Status | Status of the transaction (COMPLETED or FAILED) |
| Duration | Time it took to send data to Receiver |
| Source System URL | URL of the source job |
| Field Name | Description |
|---|---|
| Transmission Id | Transmission Id of the partition |
| Family | Name of the interface module |
| Interface Short Name | Name of the interface |
| Source System Partition Name | Partition Number |
| Partition Begin Sequence Number | Beginning sequence number in the partition |
| Partition End Sequence Number | Ending sequence number in the partition |
| Begin Block Number | Beginning block number in the partition |
| End Block Number | Ending block number in the partition |
| Status | Status of the transmission (COMPLETED or FAILED) |
| Duration | Time it took to send data for a partition |
| Field Name | Description |
|---|---|
| Block Number | Block Number in a partition |
| Block Item Count | Number of items in the block |
| Block Status | Status (COMPLETED or FAILED) |
| File Location | Location of the file for the block |
| Field Name | Description |
|---|---|
| Transaction Id | Transaction Id (Tx#<Job Instance Id>) of the uploader job |
| Remote Transaction Id | Transaction Id of the downloader job |
| Interface Module | Name of the interface module (for example Diff_Fnd) |
| Source System | Name of the source system (for example RMS) |
| Data Set Type | Type of data set (FULL or PARTIAL) |
| Source Sys Data Set Ready Time | Time when source system moved data set to outbound tables |
| Data Set Ready Time | Time when uploader job uploaded data set to inbound tables |
| File Merge Level | Merge level of the file (NO_MERGE, MERGE_TO_PARTITION_LEVEL, MERGE_TO_INTERFACE_LEVEL) |
| Status | Status of uploader job (COMPLETED or FAILED) |
| Field Name | Description |
|---|---|
| Interface Short Name | Name of the interface |
| Begin Sequence Number | Begin sequence number of data set in the inbound table |
| End Sequence Number | End sequence number of data set in the inbound table |
| Data Partition | Number of partitions used by uploader job |
| Thread | Number of threads used by uploader job |
| Merge Strategy | Merge strategy used for merging files |
| Auto Purge Data | Flag that indicates whether files need to be cleaned up or not |
This tab allows you to view and edit configurations for the BDI jobs, and it also allows the user to view, edit and create System Options.
This tab allows the user to manage the outbound interfaces and downloader and transmitter options for BDI jobs. The user with Admin privileges can edit the configurations.
This tab allows the user to manage the inbound interfaces, receiver and uploader options for BDI jobs. The user with Admin privileges can edit the configurations.
This tab allows the user to view, edit and create system options. This page displays the list of system options of the application. The user can modify the value of the existing system options, create new system options and delete the existing system options. The user need admin privileges for editing and creating system options. The Search option based on system options name and value is also provided on this page.
This section describes the job admin errors and its troubleshooting.
Issue:
Bdi Job Admin deployment can run into this error if database credentials are invalid:
Caught: javax.management.RuntimeMBeanException: java.lang.RuntimeException: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
javax.management.RuntimeMBeanException: java.lang.RuntimeException: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
at weblogic.utils.StackTraceDisabled.unknownMethod()
Caused by: java.lang.RuntimeException: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
... 1 more
Caused by: weblogic.management.provider.EditFailedException: java.lang.NullPointerException
... 1 more
Caused by: java.lang.NullPointerException
... 1 more
Solution:
Undo all changes in the Weblogic domain session. Redeploy app with setting up new credentials and verify deployment is successful.
Issue:
Log files contain this exception:
oracle.wsm.common.sdk.WSMException: WSM-07620 : Agent cannot enforce policies due to either failure in retrieving polices or error in validations, detail= "WSM-02557 The documents required to configure the Oracle Web Services Manager runtime have not been retrieved from the Policy Manager application (wsm-pm), possibly because the application is not running or has not been deployed in the environment. The query "&(@appliesTo~="REST-CLIENT()")(policysets:global:%)" is queued for later retrieval.
Solution:
Follow BDI Installation guide, and verify WSM- policy manager is configured for admin server URL.
Open weblogic domain console and Target wsm-pm app to Admin Server.
Bounce Admin server and verify wsm-pm app is in Active State.
Issue:
Diff_Fnd_DownloaderAndTransporterJob is successful, Job status is "completed" but data not transferred from outbound to inbound table and .csv file not created
Rest call to DownloaderAndTransporterJob is successful, Job status is "completed" but data not transferred from the outbound to inbound table and .csv file not created
Solution:
Verify the receiverEndpointUrl for the Table DownloaderTransmitterOptions is updated to point to where receiver app (for eg: RXM ) is deployed in my case ’blr00abi.idc.oracle.com:7001' in bdi_rms_seed_data.sql.
Verify the values in the Interface table DownloaderInterfaceDataControl such as begin and end sequence number matches with the values mentioned in bdi_rms_seed_data.sql.
Verify the values in the interface table in DB DownloaderTransmitterOptions,the receiverEndpointUrl is updated to match with bdi_rms_seed_data.sql.
Issue:
BDI App B (SIM) Job Admin GUI is showing this exception:
Caused By: java.lang.RuntimeException: Could not find jobName(OrgHier_Fnd_UploaderJob) xml file. You may have renamed the job file or your job repository has more jobs than your application. To resolve the issue either delete the job repository or add the correct job xml file to the app.
Managed server log contains:
Truncated. see log file for complete stacktrace
Caused By: java.lang.RuntimeException: Could not find jobName(OrgHier_Fnd_UploaderJob) xml file. You may have renamed the job file or your job reposiotry has more jobs than your application. To resolve the issue either delete the job repository or add the correct job xml file to the app.
at com.oracle.retail.bdi.batch.job.operator.JobOperatorServiceBean.allAvailableBatchJobs(JobOperatorServiceBean.java:167)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
Truncated. see log file for complete stacktrace
Solution:
The process flow has changed and part or all of a flow has been removed, but batch-db has not been updated to match. Either log in to the database and delete references to the job from all tables, or recreate the batch-db using RCU and redeploy BDI.
Issue:
A job fails and the Job Admin log files contain no evidence of or details about the failure.
Solution:
Take a look at the WebLogic Server log files to identify the root cause of the job failure. One example of this is improper data source configuration.
