Batch jobs are tasks that can be executed without user interaction and are best suited for non-interactive, bulk-oriented and long-running tasks that are resource intensive, can execute sequentially or parallel, and may be initiated ad hoc or through scheduling.
As described in the Java EE 7 Tutorial, the batch framework consists of:
A job specification language based on XML.
A set of batch annotations and interfaces for application classes that implement the business logic.
A batch container that manages the execution of batch jobs.
Supporting classes and interfaces to interact with the batch container.
For detailed information about batch jobs, batch processing, and the batch processing framework, see "Batch Processing" in The Java EE 7 Tutorial. Also see the Java Specification Request 352: Batch Applications for the Java Platform (http://jcp.org/en/jsr/detail?id=352
). The specification defines the programming model for batch applications and the runtime for scheduling and executing batch jobs.
WebLogic Server supports the ability for multiple batch runtime instances to run in a domain, whereby each instance is hosted on an individual Managed Server instance. However, there is no state replication across batch jobs running in a domain; that is, one batch runtime instance cannot be aware of another. Consequently, the processing for a given batch job occurs only on the batch runtime instance that is hosted on a single Managed Server. Once a batch job is started on a Managed Server instance, the job runs on that instance to completion. Any step in the job that is started, including any concurrent steps, runs only on that Managed Server instance. This behavior has important implications in both clustered and nonclustered environments, particularly with regards to load balancing, as follows:
If your deployed applications allow external users or processes to access those applications to start batch jobs, the load balancing mechanism for distributing work across the Managed Servers instances typically ensures that the workload on each batch runtime instance, over time and on average, is similarly balanced.
If you have an application that contains a batch job, and the application is replicated on multiple nonclustered Managed Server instances, you must create a data source on each Managed Server that points to the same database. You then configure each batch runtime instance to use that database for the job repository. In this manner, an incoming request that is routed by the load balancer can land on any Managed Server instance and start a new batch job.
Although batch job processing cannot be clustered, batch applications can be deployed to a cluster. You can do this by creating a data source and targeting it to the cluster. Then you configure a group of batch runtime instances, each running in a Managed Server instance in the cluster, to use the same job repository.
For more information about batch processing in a clustered environment, see Batch Applications in Administering Clusters for Oracle WebLogic Server.
There is no guarantee that the batch load at any point in time is equivalent across the Managed Server instances if different requests can generate different types of batch load. The likelihood of uneven load distribution increases if there is high degree of variability of the types of batch load that can be generated by different requests. For example, if one request is sent to one batch runtime instance, and a second request is sent to the other batch runtime instance, it is possible that the first request could start 10 batch jobs and the second request start only 2 jobs. In this scenario, it is possible for the batch job workload to become unevenly distributed.
When you install WebLogic Server complete with the examples, the examples source code is placed in the EXAMPLES_HOME
\examples\src\examples
directory. The default path is ORACLE_HOME
\wlserver\samples\server
. From this directory, you can access the source code and instruction files for the Batch 1.0 examples without having to set up the samples domain.
The ORACLE_HOME
\user_projects\domains\wl_server
directory contains the WebLogic Server examples domain; it contains your applications and the XML configuration files that define how your applications and Oracle WebLogic Server will behave, as well as startup and environment scripts. See Sample Applications and Code Examples in Understanding Oracle WebLogic Server.
Using the Batch Job Operator – demonstrates how to use the javax.batch.operations.JobOperator
interface to submit batch jobs. The JobOperator
interface provides a set of operations to start, stop, restart, and inspect jobs. This sample will also demonstrates how to use listeners to notify about specific event occurring during the batch processing execution.
EXAMPLES_HOME/examples/src/examples/javaee7/batch/joboperator-api
Using Batch Parallelization Model to Run Partitioned Job Steps – demonstrates how to use the PartitionMapper
interface to enable finer control over parallel processing.
EXAMPLES_HOME/examples/src/examples/javaee7/batch/partition
Avitek Medical Records (MedRec) – A comprehensive educational sample application that demonstrates WebLogic Server and Java EE features, as well as best practices. For Java EE 7, Medrec showcases batch processing's capability by compiling drug statistics in the background for the administrator. The statistics sum up the cost by record, physician and drug perspectives with a start date and end date, altogether in one batch, but with three outputs.
Avitek Medical Records is optionally installed with the WebLogic Server installation. You can start MedRec from the ORACLE_HOME
/user_projects/domains/medrec
directory, where ORACLE_HOME
is the directory you specified as the Oracle Home when you installed Oracle WebLogic Server.
Oracle recommends that you run these examples before programming your own applications that use batch.
Batch applications can be deployed and started on WebLogic Server out-of-the-box with no runtime configuration. This is useful for smaller development environments that do not process and store large amounts of data.
When no batch runtime configuration exists, WebLogic Server uses:
The demo Derby database to create a data source needed to update the job repository to persist batch job details.
The executor service that is bound to default JNDI name of java:comp/DefaultManagedExecutorService
(as required by the Java EE 7 specification).
In orders to access the default batch runtime configuration, WebLogic Server must be started using the startWeblogic.sh
script.
The batch runtime in WebLogic Server uses an XA-capable data source to access the JobRepository tables for batch jobs and a managed executor service to execute asynchronous batch jobs. The managed executor service processes the jobs and the JobRepository data source stores the status of current and past jobs.
The default batch runtime in a WebLogic domain can be used without any configuration, which is useful in development mode environments that only require the Derby demo database. For data-driven production environments that use a database schema, you can configure a dedicated job repository data source and managed executor service for the domain.
For enterprise-level production environments that process and store large amounts of data, a dedicated batch runtime can be configured to store the batch job details in a specific database.
The database administrator must create the job repository tables needed to persist batch job details. The schema name used to created these tables will be denoted by the getSchemaName()
method in the BatchConfigMBean
when configuring the batch runtime for the domain. See Configure the Batch Runtime to Use a Dedicated Batch Data Source and Managed Executor Service.
The job repository tables can be created using the Repository Create Utility (RCU) or using SQL scripts for the databases supported for use with WebLogic Server 12c. Schemas for creating these tables are in the following location:
ORACLE_HOME
/oracle_common/common/sql/wlservices/batch/
dbname
Iwhere ORACLE_HOME
represents the top level installation directory for Oracle WebLogic Server, and dbname
represents the name of the database.
For information about the supported databases for WebLogic Server 12c, see the Oracle Fusion Middleware Supported System Configurations page on Oracle Technology Network.
It is important to note the following when using RCU to create the job repository tables and schema owner:
JBatch_WLS
. Write this name down because you will use it when you create the batch data source and the batch runtime.Now you can create the batch data source, as described in Create a JDBC Data Source for the Job Repository. Remember that you must use the schema owner you chose on the Select Components page as the data source's user name.
Note:
The batch runtime caches the schema name and when it acquires a connection to update the job repository tables, it sets the schema name on the connection. Due to this limitation, it is not possible to use the same data source for both application and job repository (if they use separate schemas).
For more information about the Repository Clean Utility (RCU), see Creating Schemas with the Repository Creation Utility.
If you are not using RCU utility to create the job repository tables for batch, you can use the SQL command-line utility and the provided batch.sql
script to create them. For example, when you create job repository tables for Oracle Database and Oracle EBR (Edition-Based Redefinition), which require SQL to create the tables.
The batch.sql
SQL script is provided for all supported databases (such as, mysql, db2, etc.) to create the job repository tables, and are in the following location:
ORACLE_HOME
/oracle_common/common/sql/wlservices/batch/
dbname
To use the batch.sql
SQL script to create the tables, follow these steps:
Now you can create the batch data source, as described in Create a JDBC Data Source for the Job Repository. Remember that you must use the jbatch
schema owner you created as the data source's user name.
For a dedicated batch runtime within a domain, the WebLogic administrator must configure an XA-capable data source for the database that will contain the job repository tables. When a Java EE component submits a batch job, the batch runtime updates the job repository tables using this XA data source, which is obtained by looking up the data source's JNDI name.
When you create the batch data source using Administration Console or WLST, you must use the schema owner created with RCU (e.g., jbatch_wls
) or the SQL script jbatch
, as described in Create the Job Repository Tables.
For instructions on configuring JDBC data source, see Creating a JDBC Data Source in Administering JDBC Data Sources for Oracle WebLogic Server.
For optimum performance, the batch runtime can be configured to use application-scoped Managed Executor Services by configuring Managed Executor Service Templates (MES Template) that use the same name as the batch runtime setBatchJobsManagedExecutorServiceName()
. If no MES Template is specified when configuring the batch runtime, it will instead use the default Java EE Managed Executor Service that is bound to (java:comp/DefaultManagedExecutorService
).
When a new instance of a Managed Executor Service is created for each MES template, it will then run batch jobs that are submitted for applications that are deployed to the domain. For example, if there are two MES Templates named MES1
and MES2
in a domain, then when BatchApp1
and BatchApp2
are deployed, each application will get an instance of MES1
and MES2
.
However, if you have set the setBatchJobsExecutorServiceName("MES2")
, then all batch jobs submitted from BatchApp1
or BatchApp1
(or from any application deployed to the domain), will use MES2
.
For instructions on configuring a Managed Executor Service Template, see Configuring Concurrent Managed Objects.
The job repository data source and Managed Executor Service you created in Prerequisite Steps: Configure the Job Repository Tables, Batch Data Source, and Managed Executor Service can now be used to configure a dedicated batch runtime using any of these WebLogic administrative tools:
Tip:
The schema name used in Create the Job Repository Tables must be specified when following the configuration steps in these sections. For example when using MBeans, the schema name must be denoted by getSchemaName()
in the BatchConfigMBean
for the domain.
A dedicated batch runtime can be configured for a domain scope using the Administration Console. In the Settings for domain-name page, open the Batch page and complete these configuration fields:
Data Source JNDI Name – Select the JNDI name of the batch runtime's job repository data source, which will be used to store data for batch jobs submitted from applications deployed to the domain.
Schema Name – Enter the schema name used when the job repository tables were created by RCU or the JBatch SQL script, as described in Create the Job Repository Tables.
Note that if the data source is shared by applications then this schema name must be the same name that the application expects.
Executor Service Template – Select the managed executor service (MES) template that will be used to run batch jobs that are submitted from applications deployed in the domain. A MES template by the same name must exist and be targeted to the domain scope when a batch jobs is submitted.
If an MES template is not selected, then the batch runtime will use the default Java EE ManagedExecutorService that is bound to the default JNDI name of: java:comp/DefaultManagedExecutorService
.
To configure a Managed Executor Service Template in the domain configuration using the WebLogic Server Administration Console, use the Environment -> Concurrent Templates page. See Configuring Concurrent Managed Objects.
You can use WLST with the BatchRuntimeConfigMBean
and DomainMBean
to configure the batch runtime to use a specific database for the job repository:
def update_domain_batch_config(domainName, jndiName, schemaName): connect('admin','passwd') edit() startEdit() cmo.setDataSourceJndiName(jndiName) cd('/BatchConfig/' + domainName) cmo.setSchemaName(schemaName) save() activate()
In this example, if the administrator has created a data source with the JNDI name jdbc/batchDS
, then, calling update_domain_batch_config('mydomain','jdbc/batchDS','BATCH')
will configure the batch runtime to store all the job repository tables in the schema 'BATCH'
in the database that is pointed by the data source that is bound to the jndiName
: 'jdbc/batchDS'
.
You can use WLST to configure the batch runtime to use specific Managed Executor Services for batch job execution. However, you must first create an Managed Executor Service and the name of the Managed Executor Service must be provided to the DomainMBean
.
connect('admin','passwd') edit() startEdit() cmo.setBatchJobsExecutorServiceName('mesName') save() activate()
where mesName
is the name of the Managed Executor Service that has already been created (and targeted) to this domain.
The batch runtime can be configured to use different Managed Executor Services using the getBatchJobsManagedExecutorServiceName()
method in the DomainMBean
. However, a Managed Executor Service Template by the same name must exist and be targeted to the domain scope when a batch job is submitted.
See the BatchConfigMBean and DomainMBean in the MBean Reference for Oracle WebLogic Server.
See WebLogic Server WLST Online and Offline Command Reference in the WLST Command Reference for WebLogic Server.
You can query the batch runtime's JobRepository for domain scope using administrative tools such as Administration Console and MBeans.
Note:
Make sure that the database that contains the batch job repository is running. For example, the default Derby database is not automatically started when you boot WebLogic Server using the java weblogic.Server
command. If your database is not running, an exception will be thrown by the Batch RI when you submit a batch job or when you access the BatchJobRepositoryRuntimeMBean, either through WLST or the Administration Console. See Troubleshooting Tips.
A job repository can be queried using the Administration Console to obtain details about batch jobs in a domain.
In the Settings for domain-name page, open the Monitoring -> Batch Jobs page to view details about all the jobs submitted by applications deployed to the domain.
Table 8-1 All Batch Jobs
Element Name | Description |
---|---|
Job Name |
The name of the batch job. |
Application Name |
The name of the application that submitted the batch job. |
Instance ID |
The instance ID. |
Execution ID |
The execution ID. |
Batch Status |
The batch status of this job. |
Start Time |
The start time of the job. |
End Time |
The completion time of the job. |
Exit Status |
The exit status of the job. |
You can view step execution details about a job by selecting it and clicking View.
Table 8-2 Job Executions Details
Element Name | Description |
---|---|
Job Name |
The name of the batch job. |
Instance ID |
The instance ID. |
Execution ID |
The execution ID. |
Batch Status |
The batch status of this job. |
Start Time |
The start time of the job. |
End Time |
The completion time of the job. |
Exit Status |
The exit status of the job. |
You can view metrics about each step in a job execution by selecting it and clicking View.
Table 8-3 Step Executions Details
Element Name | Description |
---|---|
Step Name |
The name of the batch job step. |
Step ID |
The step ID. |
Execution ID |
The execution ID. |
Batch Status |
The batch status of this job. |
Start Time |
The start time of the job. |
End Time |
The completion time of the job. |
Exit Status |
The exit status of the job. |
The job repository can be queried using WLST using the BatchJobRepositoryRuntimeMBean
to obtain details about batch jobs in a domain.
See BatchJobRepositoryRuntimeMBean in the MBean Reference for Oracle WebLogic Server.
getJobDetails
The getJobDetails()
attribute returns details about all the jobs submitted by applications deployed to the domain. Each entry in the collection contains an array of the following elements:
Table 8-4 Elements in getJobDetails() Attribute
Element Name | Description |
---|---|
JOB_NAME |
The name of the batch job. |
APP_NAME |
The name of the application that submitted the batch job (String). |
INSTANCE_ID |
The instance ID (long). |
EXECUTION_ID |
The execution ID (long). |
BATCH_STATUS |
The batch status of this job (String). |
START_TIME |
The start time of the job ( |
END_TIME |
The completion time of the job ( |
EXIT_STATUS |
The exit status of the job (String). |
Here is an example of a WLST script that uses getJobDetails()
to print a list of all batch jobs deployed in a domain.
connect('admin', 'admin123')
domainRuntime()
cd('BatchJobRepositoryRuntime')
cd('myserver')
executions=cmo.getJobDetails(6)
print "JobName AppName InstanceID ExecutionID BatchStatus StartTime EndTime ExitStatus"
print e[0], " ", e[1], " ", e[2], " ", e[3], " ", e[4], " ", e[5], " ", e[6], ",e[7]
Here is sample output after running getJobDetails()
:
JobName InstanceID ExecutionID BatchStatus StartTime EndTime ExitStatus PayrollJob 6 6 COMPLETED Fri Apr 24 10:11:00 PDT 2015 Fri Apr 24 10:11:01 PDT 2015 COMPLETED PayrollJob 5 5 COMPLETED Fri Apr 24 10:10:57 PDT 2015 Fri Apr 24 10:10:58 PDT 2015 COMPLETED PayrollJob 4 4 COMPLETED Fri Apr 24 10:10:56 PDT 2015 Fri Apr 24 10:10:56 PDT 2015 COMPLETED PayrollJob 3 3 COMPLETED Mon Apr 20 11:32:12 PDT 2015 Mon Apr 20 11:32:12 PDT 2015 COMPLETED PayrollJob 2 2 COMPLETED Mon Apr 20 11:32:10 PDT 2015 Mon Apr 20 11:32:11 PDT 2015 COMPLETED PayrollJob 1 1 COMPLETED Mon Apr 20 11:25:26 PDT 2015 Mon Apr 20 11:25:26 PDT 2015 COMPLETED
getJobExecutions
The getJobExections
attribute returns details about a particular job execution. Each entry in the collection contains an array of the following elements:
Table 8-5 Elements in getJobExecutions() Attribute
Element Name | Description |
---|---|
JOB_NAME |
The name of the batch job (String). |
INSTANCE_ID |
The instance ID (long). |
EXECUTION_ID |
The execution ID (long). |
BATCH_STATUS |
The batch status of this job (String). |
START_TIME |
The start time of the job ( |
END_TIME |
The completion time of the job ( |
EXIT_STATUS |
The exit status of the job (String). |
Here is an example of using getJobExectuions()
in a WLST script to get details for a given ExecutionID: getJobExecutions(6)
. To get a list of all ExecutionIDs, use the getJobDetails()
method.
connect('admin', 'admin123')
domainRuntime()
cd('BatchJobRepositoryRuntime')
cd('myserver')
executions=cmo.getJobExecutions(6)
print "JobName InstanceID ExecutionID BatchStatus StartTime EndTime ExitStatus"
for e in executions
print e[0], " ", e[1], " ", e[2], " ", e[3], " ", e[4], " ", e[5], " ", e[6]
Here is sample output after running getJobExecutions()
:
JobName InstanceID ExecutionID BatchStatus StartTime EndTime ExitStatus PayrollJob 6 6 COMPLETED Fri Apr 24 10:11:00 PDT 2015 Fri Apr 24 10:11:01 PDT 2015 COMPLETED
getStepExecutions
The getStepExecutions
attribute returns metrics about each step in a Job execution. Each entry in the collection contains an array of the following elements:
Table 8-6 Elements in getStepExecutions() Attribute
Element Name | Description |
---|---|
STEP_NAME |
The name of the batch job step (String). |
STEP_ID |
The step ID (long). |
EXECUTION_ID |
The execution ID (long). |
BATCH_STATUS |
The batch status of this job (String). |
START_TIME |
The start time of the job ( |
END_TIME |
The completion time of the job ( |
EXIT_STATUS |
The exit status of the job (String). |
Here is an example of using getStepExecutions()
in a WLST script to get details for a given StepExecutionID: getStepExecutions(6)
. To get a list of all ExecutionIDs, use the getJobDetails()
method.
connect('admin', 'admin123')
domainRuntime()
cd('BatchJobRepositoryRuntime')
cd('myserver')
executions=cmo.getStepExecutions(6)
print "StepName StepExecutionID BatchStatus StartTime EndTime ExitStatus"
print e[0], " ", e[1], " ", e[2], " ", e[3], " ", e[4], " ", e[5], "]
Here is sample output after running getStepExecutions()
:
StepName StepExecutionID BatchStatus StartTime EndTime ExitStatus PayrollJob 6 6 COMPLETED Fri Apr 24 10:11:00 PDT 2015 Fri Apr 24 10:11:01 PDT 2015 COMPLETED
Learn tips for configuring and using the batch runtime with WebLogic Server.
A common mistake made by users of the Batch RI (reference implementation) is to neglect starting the database that contains the job repository tables. For example, if you boot WebLogic using the java weblogic.Server
command, the Derby database is not automatically started. If the DB isn't running, then when you submit a batch job or access the JobRepositoryRuntimeMBean
(either through WLST or through the Administration Console), that job will fail and a cryptic exception will be thrown by the Batch RI:
[1] Exception thrown by Refernce Implementation (from IBM): Caused by: weblogic.common.resourcepool.ResourceSystemException: Cannot load driver class org.apache.derby.jdbc.ClientDataSource for datasource '<<<Data Source name>>>>'
Here is another error message that could be thrown by the Batch RI when the job repository's database isn't running:
Caused By: com.ibm.jbatch.container.exception.PersistenceException: weblogic.jdbc.extensions.ConnectionDeadSQLException: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection for datasource '_com_oracle_weblogic_batch_connector@_com_oracle_weblogic_batch_connector_impl_@_com_oracle_weblogic_batch_connector_impl_WLSDatabaseConfigurationBean@_com_oracle_batch_internal__derby_batch_DataSource'. The DBMS driver exception was: java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused. at com.ibm.jbatch.container.services.impl.JDBCPersistenceManagerImpl.getConnectionToDefaultSchema(JDBCPersistenceManagerImpl.java:354) at com.ibm.jbatch.container.services.impl.JDBCPersistenceManagerImpl.isDerby(JDBCPersistenceManagerImpl.java:182) at com.ibm.jbatch.container.services.impl.JDBCPersistenceManagerImpl.init(JDBCPersistenceManagerImpl.java:143) at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.getService(ServicesManagerImpl.java:404) at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.access$300(ServicesManagerImpl.java:388)
If you see these errors, make sure that the database that contains the batch runtime job repository is running.