8 Using the Batch Runtime

WebLogic Server implements the batch runtime to provide support for defining, implementing, and running batch jobs, as defined for Java EE 7 in Batch Applications for the Java Platform (JSR 352).

About Batch Jobs

Batch jobs are tasks that can be executed without user interaction and are best suited for non-interactive, bulk-oriented and long-running tasks that are resource intensive, can execute sequentially or parallel, and may be initiated ad hoc or through scheduling.

As described in the Java EE 7 Tutorial, the batch framework consists of:

  • A job specification language based on XML.

  • A set of batch annotations and interfaces for application classes that implement the business logic.

  • A batch container that manages the execution of batch jobs.

  • Supporting classes and interfaces to interact with the batch container.

For detailed information about batch jobs, batch processing, and the batch processing framework, see "Batch Processing" in The Java EE 7 Tutorial. Also see the Java Specification Request 352: Batch Applications for the Java Platform (http://jcp.org/en/jsr/detail?id=352). The specification defines the programming model for batch applications and the runtime for scheduling and executing batch jobs.

Use of Multiple Batch Runtime Instances

WebLogic Server supports the ability for multiple batch runtime instances to run in a domain, whereby each instance is hosted on an individual Managed Server instance. However, there is no state replication across batch jobs running in a domain; that is, one batch runtime instance cannot be aware of another. Consequently, the processing for a given batch job occurs only on the batch runtime instance that is hosted on a single Managed Server. Once a batch job is started on a Managed Server instance, the job runs on that instance to completion. Any step in the job that is started, including any concurrent steps, runs only on that Managed Server instance. This behavior has important implications in both clustered and nonclustered environments, particularly with regards to load balancing, as follows:

  • If your deployed applications allow external users or processes to access those applications to start batch jobs, the load balancing mechanism for distributing work across the Managed Servers instances typically ensures that the workload on each batch runtime instance, over time and on average, is similarly balanced.

  • If you have an application that contains a batch job, and the application is replicated on multiple nonclustered Managed Server instances, you must create a data source on each Managed Server that points to the same database. You then configure each batch runtime instance to use that database for the job repository. In this manner, an incoming request that is routed by the load balancer can land on any Managed Server instance and start a new batch job.

  • Although batch job processing cannot be clustered, batch applications can be deployed to a cluster. You can do this by creating a data source and targeting it to the cluster. Then you configure a group of batch runtime instances, each running in a Managed Server instance in the cluster, to use the same job repository.

    For more information about batch processing in a clustered environment, see Batch Applications in Administering Clusters for Oracle WebLogic Server.

  • There is no guarantee that the batch load at any point in time is equivalent across the Managed Server instances if different requests can generate different types of batch load. The likelihood of uneven load distribution increases if there is high degree of variability of the types of batch load that can be generated by different requests. For example, if one request is sent to one batch runtime instance, and a second request is sent to the other batch runtime instance, it is possible that the first request could start 10 batch jobs and the second request start only 2 jobs. In this scenario, it is possible for the batch job workload to become unevenly distributed.

Batch 1.0 Code Examples in WebLogic Server

When you install WebLogic Server complete with the examples, the examples source code is placed in the EXAMPLES_HOME\examples\src\examples directory. The default path is ORACLE_HOME\wlserver\samples\server. From this directory, you can access the source code and instruction files for the Batch 1.0 examples without having to set up the samples domain.

The ORACLE_HOME\user_projects\domains\wl_server directory contains the WebLogic Server examples domain; it contains your applications and the XML configuration files that define how your applications and Oracle WebLogic Server will behave, as well as startup and environment scripts. See Sample Applications and Code Examples in Understanding Oracle WebLogic Server.

  • Using the Batch Job Operator – demonstrates how to use the javax.batch.operations.JobOperator interface to submit batch jobs. The JobOperator interface provides a set of operations to start, stop, restart, and inspect jobs. This sample will also demonstrates how to use listeners to notify about specific event occurring during the batch processing execution.

    EXAMPLES_HOME/examples/src/examples/javaee7/batch/joboperator-api
    
  • Using Batch Parallelization Model to Run Partitioned Job Steps – demonstrates how to use the PartitionMapper interface to enable finer control over parallel processing.

    EXAMPLES_HOME/examples/src/examples/javaee7/batch/partition
    
  • Avitek Medical Records (MedRec) – A comprehensive educational sample application that demonstrates WebLogic Server and Java EE features, as well as best practices. For Java EE 7, Medrec showcases batch processing's capability by compiling drug statistics in the background for the administrator. The statistics sum up the cost by record, physician and drug perspectives with a start date and end date, altogether in one batch, but with three outputs.

    Avitek Medical Records is optionally installed with the WebLogic Server installation. You can start MedRec from the ORACLE_HOME/user_projects/domains/medrec directory, where ORACLE_HOME is the directory you specified as the Oracle Home when you installed Oracle WebLogic Server.

Oracle recommends that you run these examples before programming your own applications that use batch.

Using the Default Batch Runtime Configuration with the Derby Database

Batch applications can be deployed and started on WebLogic Server out-of-the-box with no runtime configuration. This is useful for smaller development environments that do not process and store large amounts of data.

When no batch runtime configuration exists, WebLogic Server uses:

  • The demo Derby database to create a data source needed to update the job repository to persist batch job details.

  • The executor service that is bound to default JNDI name of java:comp/DefaultManagedExecutorService (as required by the Java EE 7 specification).

In orders to access the default batch runtime configuration, WebLogic Server must be started using the startWeblogic.sh script.

See Querying the Batch Runtime.

Configuring the Batch Runtime to Use a Dedicated Database

The batch runtime in WebLogic Server uses an XA-capable data source to access the JobRepository tables for batch jobs and a managed executor service to execute asynchronous batch jobs. The managed executor service processes the jobs and the JobRepository data source stores the status of current and past jobs.

The default batch runtime in a WebLogic domain can be used without any configuration, which is useful in development mode environments that only require the Derby demo database. For data-driven production environments that use a database schema, you can configure a dedicated job repository data source and managed executor service for the domain.

Prerequisite Steps: Configure the Job Repository Tables, Batch Data Source, and Managed Executor Service

For enterprise-level production environments that process and store large amounts of data, a dedicated batch runtime can be configured to store the batch job details in a specific database.

Create the Job Repository Tables

The database administrator must create the job repository tables needed to persist batch job details. The schema name used to created these tables will be denoted by the getSchemaName() method in the BatchConfigMBean when configuring the batch runtime for the domain. See Configure the Batch Runtime to Use a Dedicated Batch Data Source and Managed Executor Service.

The job repository tables can be created using the Repository Create Utility (RCU) or using SQL scripts for the databases supported for use with WebLogic Server 14c. Schemas for creating these tables are in the following location:

ORACLE_HOME/oracle_common/common/sql/wlservices/batch/dbname

Iwhere ORACLE_HOME represents the top level installation directory for Oracle WebLogic Server, and dbname represents the name of the database.

For information about the supported databases for WebLogic Server 14c, see the Oracle Fusion Middleware Supported System Configurations page on Oracle Technology Network.

Creating Job Repository Tables Using RCU

It is important to note the following when using RCU to create the job repository tables and schema owner:

  1. On the Select Components page, select WebLogic Services as the component. Also, note that Schema Owner name will default to the schema prefix string you chose plus "WLS", such as JBatch_WLS. Write this name down because you will use it when you create the batch data source and the batch runtime.
  2. On the Schema Passwords page, choose the Select same password for all schemas option.
  3. When you click Finish, RCU will create tables and schemas for all WebLogic related components, including Batch, EJB Timers, Diagnostics, etc.

Now you can create the batch data source, as described in Create a JDBC Data Source for the Job Repository. Remember that you must use the schema owner you chose on the Select Components page as the data source's user name.

Note:

The batch runtime caches the schema name and when it acquires a connection to update the job repository tables, it sets the schema name on the connection. Due to this limitation, it is not possible to use the same data source for both application and job repository (if they use separate schemas).

For more information about the Repository Clean Utility (RCU), see Creating Schemas with the Repository Creation Utility.

Creating Job Repository Tables Using an SQL Script

If you are not using RCU utility to create the job repository tables for batch, you can use the SQL command-line utility and the provided batch.sql script to create them. For example, when you create job repository tables for Oracle Database and Oracle EBR (Edition-Based Redefinition), which require SQL to create the tables.

The batch.sql SQL script is provided for all supported databases (such as, mysql, db2, etc.) to create the job repository tables, and are in the following location:

ORACLE_HOME/oracle_common/common/sql/wlservices/batch/dbname

To use the batch.sql SQL script to create the tables, follow these steps:

  1. Open an SQL command-line session for your database.
  2. Create a new user called jbatch that will be identified by the batch.sql script.
  3. Grant Connect privileges to user jbatch.
  4. Grant Resource privileges to user jbatch.
  5. Run the batch.sql script from the directory containing the Oracle database's SQL scripts. For example:

    ORACLE_HOME/oracle_common/common/sql/wlservices/batch/oracle/batch.sql

Now you can create the batch data source, as described in Create a JDBC Data Source for the Job Repository. Remember that you must use the jbatch schema owner you created as the data source's user name.

Create a JDBC Data Source for the Job Repository

For a dedicated batch runtime within a domain, the WebLogic administrator must configure an XA-capable data source for the database that will contain the job repository tables. When a Java EE component submits a batch job, the batch runtime updates the job repository tables using this XA data source, which is obtained by looking up the data source's JNDI name.

When you create the batch data source using Administration Console or WLST, you must use the schema owner created with RCU (e.g., jbatch_wls) or the SQL script jbatch, as described in Create the Job Repository Tables.

For instructions on configuring JDBC data source, see Creating a JDBC Data Source in Administering JDBC Data Sources for Oracle WebLogic Server.

Optionally, Create a Managed Executor Service Template

For optimum performance, the batch runtime can be configured to use application-scoped Managed Executor Services by configuring Managed Executor Service Templates (MES Template) that use the same name as the batch runtime setBatchJobsManagedExecutorServiceName(). If no MES Template is specified when configuring the batch runtime, it will instead use the default Java EE Managed Executor Service that is bound to (java:comp/DefaultManagedExecutorService).

When a new instance of a Managed Executor Service is created for each MES template, it will then run batch jobs that are submitted for applications that are deployed to the domain. For example, if there are two MES Templates named MES1 and MES2 in a domain, then when BatchApp1 and BatchApp2 are deployed, each application will get an instance of MES1 and MES2.

However, if you have set the setBatchJobsExecutorServiceName("MES2"), then all batch jobs submitted from BatchApp1 or BatchApp1 (or from any application deployed to the domain), will use MES2.

For instructions on configuring a Managed Executor Service Template, see Configuring Concurrent Managed Objects.

Configure the Batch Runtime to Use a Dedicated Batch Data Source and Managed Executor Service

The job repository data source and Managed Executor Service you created in Prerequisite Steps: Configure the Job Repository Tables, Batch Data Source, and Managed Executor Service can now be used to configure a dedicated batch runtime using any of these WebLogic administrative tools:

Tip:

The schema name used in Create the Job Repository Tables must be specified when following the configuration steps in these sections. For example when using MBeans, the schema name must be denoted by getSchemaName() in the BatchConfigMBean for the domain.

Configuring the Batch Runtime Using the Administration Console

A dedicated batch runtime can be configured for a domain scope using the Administration Console. In the Settings for domain-name page, open the Batch page and complete these configuration fields:

  • Data Source JNDI Name – Select the JNDI name of the batch runtime's job repository data source, which will be used to store data for batch jobs submitted from applications deployed to the domain.

  • Schema Name – Enter the schema name used when the job repository tables were created by RCU or the JBatch SQL script, as described in Create the Job Repository Tables.

    Note that if the data source is shared by applications then this schema name must be the same name that the application expects.

  • Executor Service Template – Select the managed executor service (MES) template that will be used to run batch jobs that are submitted from applications deployed in the domain. A MES template by the same name must exist and be targeted to the domain scope when a batch jobs is submitted.

    If an MES template is not selected, then the batch runtime will use the default Java EE ManagedExecutorService that is bound to the default JNDI name of: java:comp/DefaultManagedExecutorService.

To configure a Managed Executor Service Template in the domain configuration using the WebLogic Server Administration Console, use the Environment -> Concurrent Templates page. See Configuring Concurrent Managed Objects.

Configuring the Batch Runtime Using WLST

You can use WLST with the BatchRuntimeConfigMBean and DomainMBean to configure the batch runtime to use a specific database for the job repository:

def update_domain_batch_config(domainName, jndiName, schemaName):
    connect('admin','passwd')
    edit()
    startEdit()
    cmo.setDataSourceJndiName(jndiName)
    cd('/BatchConfig/' + domainName)
    cmo.setSchemaName(schemaName)
    save()
    activate()

In this example, if the administrator has created a data source with the JNDI name jdbc/batchDS, then, calling update_domain_batch_config('mydomain','jdbc/batchDS','BATCH') will configure the batch runtime to store all the job repository tables in the schema 'BATCH' in the database that is pointed by the data source that is bound to the jndiName: 'jdbc/batchDS'.

You can use WLST to configure the batch runtime to use specific Managed Executor Services for batch job execution. However, you must first create an Managed Executor Service and the name of the Managed Executor Service must be provided to the DomainMBean.

   connect('admin','passwd')
    edit()
    startEdit()
    cmo.setBatchJobsExecutorServiceName('mesName')
    save()
    activate()

where mesName is the name of the Managed Executor Service that has already been created (and targeted) to this domain.

The batch runtime can be configured to use different Managed Executor Services using the getBatchJobsManagedExecutorServiceName() method in the DomainMBean. However, a Managed Executor Service Template by the same name must exist and be targeted to the domain scope when a batch job is submitted.

See the BatchConfigMBean and DomainMBean in the MBean Reference for Oracle WebLogic Server.

See WebLogic Server WLST Online and Offline Command Reference in the WLST Command Reference for WebLogic Server.

Querying the Batch Runtime

You can query the batch runtime's JobRepository for domain scope using administrative tools such as Administration Console and MBeans.

Note:

Make sure that the database that contains the batch job repository is running. For example, the default Derby database is not automatically started when you boot WebLogic Server using the java weblogic.Server command. If your database is not running, an exception will be thrown by the Batch RI when you submit a batch job or when you access the BatchJobRepositoryRuntimeMBean, either through WLST or the Administration Console. See Troubleshooting Tips.

Using the Administration Console to Query the Batch Runtime

A job repository can be queried using the Administration Console to obtain details about batch jobs in a domain.

Get Details of all Batch Jobs

In the Settings for domain-name page, open the Monitoring -> Batch Jobs page to view details about all the jobs submitted by applications deployed to the domain.

Table 8-1 All Batch Jobs

Element Name Description

Job Name

The name of the batch job.

Application Name

The name of the application that submitted the batch job.

Instance ID

The instance ID.

Execution ID

The execution ID.

Batch Status

The batch status of this job.

Start Time

The start time of the job.

End Time

The completion time of the job.

Exit Status

The exit status of the job.

Get Details about a Job's Execution

You can view step execution details about a job by selecting it and clicking View.

Table 8-2 Job Executions Details

Element Name Description

Job Name

The name of the batch job.

Instance ID

The instance ID.

Execution ID

The execution ID.

Batch Status

The batch status of this job.

Start Time

The start time of the job.

End Time

The completion time of the job.

Exit Status

The exit status of the job.

Get Details about a Job's Step Execution

You can view metrics about each step in a job execution by selecting it and clicking View.

Table 8-3 Step Executions Details

Element Name Description

Step Name

The name of the batch job step.

Step ID

The step ID.

Execution ID

The execution ID.

Batch Status

The batch status of this job.

Start Time

The start time of the job.

End Time

The completion time of the job.

Exit Status

The exit status of the job.

Using Runtime MBeans to Query the Batch Runtime

The job repository can be queried using WLST using the BatchJobRepositoryRuntimeMBean to obtain details about batch jobs in a domain.

See BatchJobRepositoryRuntimeMBean in the MBean Reference for Oracle WebLogic Server.

Get Details of all Batch Jobs Using getJobDetails

The getJobDetails() attribute returns details about all the jobs submitted by applications deployed to the domain. Each entry in the collection contains an array of the following elements:

Table 8-4 Elements in getJobDetails() Attribute

Element Name Description

JOB_NAME

The name of the batch job.

APP_NAME

The name of the application that submitted the batch job (String).

INSTANCE_ID

The instance ID (long).

EXECUTION_ID

The execution ID (long).

BATCH_STATUS

The batch status of this job (String).

START_TIME

The start time of the job (java.util.Date).

END_TIME

The completion time of the job (java.util.Date).

EXIT_STATUS

The exit status of the job (String).

Here is an example of a WLST script that uses getJobDetails() to print a list of all batch jobs deployed in a domain.

connect('admin', 'admin123')
domainRuntime()
cd('BatchJobRepositoryRuntime')
cd('myserver')
executions=cmo.getJobDetails(6)
print "JobName  AppName   InstanceID  ExecutionID  BatchStatus  StartTime  EndTime  ExitStatus"
 print e[0], " ", e[1], " ", e[2], "   ", e[3], "  ", e[4], "  ", e[5], " ", e[6],   ",e[7]

Here is sample output after running getJobDetails():

JobName   InstanceID  ExecutionID   BatchStatus       StartTime                      EndTime                   ExitStatus
PayrollJob     6           6         COMPLETED   Fri Apr 24 10:11:00 PDT 2015   Fri Apr 24 10:11:01 PDT 2015   COMPLETED
PayrollJob     5           5         COMPLETED   Fri Apr 24 10:10:57 PDT 2015   Fri Apr 24 10:10:58 PDT 2015   COMPLETED
PayrollJob     4           4         COMPLETED   Fri Apr 24 10:10:56 PDT 2015   Fri Apr 24 10:10:56 PDT 2015   COMPLETED
PayrollJob     3           3         COMPLETED   Mon Apr 20 11:32:12 PDT 2015   Mon Apr 20 11:32:12 PDT 2015   COMPLETED
PayrollJob     2           2         COMPLETED   Mon Apr 20 11:32:10 PDT 2015   Mon Apr 20 11:32:11 PDT 2015   COMPLETED
PayrollJob     1           1         COMPLETED   Mon Apr 20 11:25:26 PDT 2015   Mon Apr 20 11:25:26 PDT 2015   COMPLETED
Get Details of a Job Execution Using getJobExecutions

The getJobExections attribute returns details about a particular job execution. Each entry in the collection contains an array of the following elements:

Table 8-5 Elements in getJobExecutions() Attribute

Element Name Description

JOB_NAME

The name of the batch job (String).

INSTANCE_ID

The instance ID (long).

EXECUTION_ID

The execution ID (long).

BATCH_STATUS

The batch status of this job (String).

START_TIME

The start time of the job (java.util.Date).

END_TIME

The completion time of the job (java.util.Date).

EXIT_STATUS

The exit status of the job (String).

Here is an example of using getJobExectuions() in a WLST script to get details for a given ExecutionID: getJobExecutions(6). To get a list of all ExecutionIDs, use the getJobDetails() method.

connect('admin', 'admin123')
domainRuntime()
cd('BatchJobRepositoryRuntime')
cd('myserver')
executions=cmo.getJobExecutions(6)
print  "JobName    InstanceID   ExecutionID   BatchStatus   StartTime   EndTime    ExitStatus"
for e in executions
   print e[0], "  ", e[1], "    ", e[2], "    ", e[3], "    ", e[4], "  ", e[5], "  ", e[6]

Here is sample output after running getJobExecutions():

JobName   InstanceID  ExecutionID   BatchStatus       StartTime                      EndTime                   ExitStatus
PayrollJob     6           6         COMPLETED   Fri Apr 24 10:11:00 PDT 2015   Fri Apr 24 10:11:01 PDT 2015    COMPLETED
Get Details of a Job Step Execution Using getStepExecutions

The getStepExecutions attribute returns metrics about each step in a Job execution. Each entry in the collection contains an array of the following elements:

Table 8-6 Elements in getStepExecutions() Attribute

Element Name Description

STEP_NAME

The name of the batch job step (String).

STEP_ID

The step ID (long).

EXECUTION_ID

The execution ID (long).

BATCH_STATUS

The batch status of this job (String).

START_TIME

The start time of the job (java.util.Date).

END_TIME

The completion time of the job (java.util.Date)

EXIT_STATUS

The exit status of the job (String).

Here is an example of using getStepExecutions() in a WLST script to get details for a given StepExecutionID: getStepExecutions(6). To get a list of all ExecutionIDs, use the getJobDetails() method.

connect('admin', 'admin123')
domainRuntime()
cd('BatchJobRepositoryRuntime')
cd('myserver')
executions=cmo.getStepExecutions(6)
print  "StepName   StepExecutionID   BatchStatus    StartTime    EndTime    ExitStatus"
   print e[0], "     ", e[1], "      ", e[2], "     ", e[3], " ", e[4], "   ", e[5], "]

Here is sample output after running getStepExecutions():

StepName   StepExecutionID  BatchStatus       StartTime                      EndTime                   ExitStatus
PayrollJob     6           6         COMPLETED   Fri Apr 24 10:11:00 PDT 2015   Fri Apr 24 10:11:01 PDT 2015    COMPLETED

Troubleshooting Tips

Learn tips for configuring and using the batch runtime with WebLogic Server.

Make Sure the Database Containing the Job Repository Tables is Running

A common mistake made by users of the Batch RI (reference implementation) is to neglect starting the database that contains the job repository tables. For example, if you boot WebLogic using the java weblogic.Server command, the Derby database is not automatically started. If the DB isn't running, then when you submit a batch job or access the JobRepositoryRuntimeMBean (either through WLST or through the Administration Console), that job will fail and a cryptic exception will be thrown by the Batch RI:

[1] Exception thrown by Refernce Implementation (from IBM):

Caused by: weblogic.common.resourcepool.ResourceSystemException: Cannot load driver class org.apache.derby.jdbc.ClientDataSource for datasource '<<<Data Source name>>>>'

Here is another error message that could be thrown by the Batch RI when the job repository's database isn't running:

Caused By: com.ibm.jbatch.container.exception.PersistenceException:
weblogic.jdbc.extensions.ConnectionDeadSQLException: 
weblogic.common.resourcepool.ResourceDeadException: 
0:weblogic.common.ResourceException: Could not create pool connection for datasource '_com_oracle_weblogic_batch_connector@_com_oracle_weblogic_batch_connector_impl_@_com_oracle_weblogic_batch_connector_impl_WLSDatabaseConfigurationBean@_com_oracle_batch_internal__derby_batch_DataSource'.
The DBMS driver exception was: java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.
     at
com.ibm.jbatch.container.services.impl.JDBCPersistenceManagerImpl.getConnectionToDefaultSchema(JDBCPersistenceManagerImpl.java:354)
     at
com.ibm.jbatch.container.services.impl.JDBCPersistenceManagerImpl.isDerby(JDBCPersistenceManagerImpl.java:182)
     at
com.ibm.jbatch.container.services.impl.JDBCPersistenceManagerImpl.init(JDBCPersistenceManagerImpl.java:143)
     at
com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.getService(ServicesManagerImpl.java:404)
     at
com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.access$300(ServicesManagerImpl.java:388)

If you see these errors, make sure that the database that contains the batch runtime job repository is running.