Controlling Batch Operations

14 Controlling Batch Operations

Learn how to use the Batch Controller to launch batch handlers that check or process data for your Oracle Communications Billing and Revenue Management (BRM) system.

Topics in this document:

About the Batch Controller

The BRM Batch Controller lets you specify when to run programs or scripts automatically, either at timed intervals or upon creation of certain files, such as log files.

The Batch Controller configuration file (BRM_home/apps/batch_controller/Infranet.properties) has entries that tell the Batch Controller when to run the specified batch handlers. These programs or scripts can be triggered at specified times or by specified kinds of occurrences, such as creation of a new log file. You can specify scheduled execution, starting the handler at fixed times of the day only, or metronomic execution, starting the handler repeatedly at a fixed interval.

A batch handler can be any executable program or script that can be run from a command line. For more information about the requirements for writing batch handlers, see "Writing Custom Batch Handlers" in BRM Developer's Guide.

Only one Batch Controller can run on a single computer, but it can control many batch handlers to launch various applications.

BRM installation includes a generic batch handler, called SampleHandler, which you can use with most applications. See "About SampleHandler".

Setting Activity Times and Triggers

Certain general parameters apply to all batch activity. Other parameters identify the batch handlers and regulate when those handlers are to be run.

General Batch Controller Parameters

The Batch Controller's Infranet properties file (BRM_home) specifies how to connect the Batch Controller to the Connection Manager (CM), when to allow high batch traffic, how much logging to record, and how long to wait for handlers.

Connection Parameters

Batch Controller requires the following parameters to connect to a CM: a user ID and password (specified in the infranet.connection parameter), the CM's port number, and the address of the CM's host computer.

Note:

To only use a user ID, set the infranet.login.type parameter to 0; to use both a user ID and a password, set infranet.login.type to 1.

In the batch.lock.socket.addr parameter, specify the socket number to lock on. If in doubt, check with your system administrator for the number of an unused socket that can be used exclusively by the Batch Controller. You can use the default value, 11971, unless some other application is using that socket number.

To write a stack trace of any runtime exceptions in the log file, set infranet.log.logallebuf to True. To disable this feature, set it to False.

Time-to-Run Parameters

When your system's heaviest load typically occurs, you might want some handlers to run less often than they do when the load is lighter. The Batch Controller divides the day into high-load and low-load periods to balance the demand for system resources.

Specify the starting time of your system's busiest period in the batch.start.highload.time parameter. Specify the starting time of your system's slowest period in the batch.end.highload.time parameter. For each of these times, specify the hour, minute, and second, in hhmmss, using the 24-hour clock. For each handler, you can specify the maximum number of simultaneous actions during each of these two periods.

The end parameter must be greater than the start parameter. If you do specify start, it must be greater than end. Specifying the same value for both parameters causes an error.

In the batch.check.interval parameter, specify the time, in seconds, between checks for occurrences of the type specified in batch.timed.events or batch.random.events. Omitting batch.check.interval causes an error.

Log-File Parameter

For logging, you can specify the level of information reported, the full path of the log file, and whether to print a stack trace of runtime exceptions. Set infranet.log.level to 0 for no logging; to 1 for error messages only; to 2 for error messages and warnings; or to 3 for error messages, warnings, and debug messages. Set infranet.log.file to the path and file name for the Batch Controller log file.

If you omit infranet.log.file, BRM uses default.pinlog in the current directory. Omitting infranet.log.level causes an error.

Timeout-Limit Parameters

You can also set timeout limits for handlers to start their objects and to complete their execution. Set batch.handler.start.wait to the number of seconds allowed for the handler to update its own object status from STARTING to STARTED, and set batch.handler.end.wait to the number of seconds allowed for updating from STARTED to some other state, such as COMPLETED. See "Writing Custom Batch Handlers" in BRM Developer's Guide for descriptions of all of the handler states.

Note:

Omitting either batch.handler.start.wait or batch.handler.end.wait causes an error.

Example of Parameters

This example demonstrates the general parameters:

infranet.connection         pcp://root.0.0.0.1:password@myserver:11960/service/pcm_client
infranet.login.type         1
infranet.log.logallebuf     true
batch.lock.socket.addr      11971
batch.start.highload.time   083000
batch.end.highload.time     211500
infranet.log.file           BRM_home/apps/batch_controller/batch.pinlog
infranet.log.level          2
batch.handler.start.wait    600
batch.handler.end.wait      43200

In this example, the Batch Controller logs on to the CM host on myserver, port 11960, as user root.0.0.0.1. High usage is expected between 8:30 a.m. and 9:15 p.m. Logging writes a normal level of information to file batch.pinlog and prints a stack trace if any program errors are found. Timeouts for updating object status are 10 minutes (600 seconds) for going to STARTED and 12 hours (43,200 seconds) for going to COMPLETED or some other state.

Handler Identification

To identify each of the batch handlers:

In the handler_name.name parameter, enter a descriptive label for the handler. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.
In the handler_name.max.at.highload.time parameter, specify the highest number of instances of this batch handler that are permitted to run simultaneously during the time from batch.start.highload.time to batch.end.highload.time.
In the handler_name.max.at.lowload.time parameter, specify the highest number of instances of this batch handler that are permitted to run simultaneously during the low-usage time; the time from batch.end.highload.time to batch.start.highload.time.
In the handler_name.start.string parameter, specify the command line to start this batch handler.

Note:

When the Batch Controller issues this command, it appends -p handler_poid -d failed_handler_poid to the command, as described in "Writing Custom Batch Handlers" in BRM Developer's Guide. If you are not using custom batch handlers, you can ignore these options.

This example demonstrates the identification parameters:

handler1.name                     Visa Handler #1
. . .
handler1.max.at.lowload.time      4
handler1.max.at.highload.time     2
handler1.start.string    BRM_home/apps/visa-handler/visa.pl

handler2.name                     Discover Handler #1
. . .
handler2.max.at.lowload.time      5
handler2.max.at.highload.time     3
handler2.start.string      BRM_home/apps/discover-handler/discover -y 2001

In this example, the internal name Visa Handler #1 applies to the program started by the command string BRM_home/apps/visa-handler/visa.pl, which runs a Perl script. The parameters in the above example limit this program to one or two concurrent actions during the specified high-load period or as many as four during the low-load period.

The other batch handler in this example, Discover Handler #1, runs the application

BRM_home/apps/discover-handler/discover with the additional option -y 2001.

Occurrence-Driven Execution

To trigger execution based on specified occurrences:

In the batch.random.events parameter, specify the event identifiers. If you have two or more event identifiers, separate each with a comma, but no blank space.
In the event_name.name parameter, enter a descriptive label for the event. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.
In the event_name.handlers parameter, specify the identifiers of one or more handlers to trigger when the event occurs. You must specify at least one handler. If you have two or more handlers, separate each with a comma; no blank spaces are allowed.
In the event_name.file.location parameter, specify the full path name of the directory to monitor for the arrival of new files that match the pattern in event_name.file.pattern.
In the event_name.file.pattern parameter, specify the file name pattern to look for. You can use an asterisk (*) to represent zero or more characters in the file name. No other wild cards (metacharacters) are supported.
Note:
- Depending on your configuration, the file pattern might conflict with a file pattern used by another component, such as Rated Event Loader. To prevent conflicts, use a specific pattern, for example, test*.out.
- You must specify batch.timed.events or batch.random.events or both. Specifying neither causes the Batch Controller to shut down just after it starts because it has no tasks to perform.

When the Batch Controller starts, it tests the file name pattern against every file name in the specified directory and runs the batch handler for each file where the name matches the pattern. It then monitors the files entering that directory and runs the batch handler whenever it finds a match.

Note:

By default, the Batch Controller appends .bc to the file name of each file it processes. This prevents batch handlers from retrieving files that the Batch Controller has yet to process. You can change the default .bc extension by editing the batch.file.rename.extension entry in the Batch Controller Infranet.properties file.

Timed Execution

The Batch Controller provides two time-based scheduling options: metronomic execution and scheduled execution.

Metronomic Execution

To set up metronomic execution:

In the batch.timed.events parameter, specify the event identifiers. If you have two or more event identifiers, separate each with a comma; blank spaces are not allowed.
In the event_name.name parameter, enter a descriptive label for the event. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.
In the event_name.handlers parameter, specify identifiers for one or more handlers to trigger when the event occurs. If you have two or more handlers, separate each with a comma; blank spaces are not allowed.
(Optional) In the event_name.start parameter, specify when you want the first execution to occur, in hhmmss, using the 24-hour clock. If you omit this parameter, the first execution occurs immediately after the Batch Controller starts.
In the event_name.interval parameter, specify the frequency, in seconds, for triggering the associated handler. Failing to specify the interval causes an error.
(Optional) In the event_name.count parameter, specify how many times to run this batch handler. If you do not specify this limit, batch handlers run repeatedly at the fixed interval for as long as the Batch Controller is running.

Note:

You must specify batch.timed.events or batch.random.events or both. Specifying neither causes the Batch Controller to shut down just after it starts because it has no tasks to perform.

This example demonstrates the metronomic parameters:

batch.timed.events                event4
. . .
event4.name                       Hourly tally
event4.handlers                   handler4
event4.start                      000000
event4.interval                   3600
event4.count                      4

In this example, the occurrence specified as event4 is named Hourly tally. It runs handler4 for the first time at midnight (000000), and then runs it again every hour (3600 seconds) after that until it has run four times. If it cannot run at a scheduled time because previous executions are not finished, it runs again immediately as soon as possible. For example, consider the time line in Figure 14-1 for event4, above:

Figure 14-1 Hourly Tally Run 1 Exceeds 1 Hour

Description of "Figure 14-1 Hourly Tally Run 1 Exceeds 1 Hour"

In this example, the first run of handler4 continues for more than an hour, taking it past the time when the second run is supposed to begin. The second scheduled run cannot start at the one-hour interval, so it starts as soon as possible after that (1:20 a.m.). The third and fourth scheduled executions start at regular multiples of the interval, measured from the first run.

If the overly long run continues past two scheduled run start times (occurrences), only one run starts on the delayed basis. For example, suppose the midnight run lasts until 2:25 a.m., as shown in Figure 14-2:

Figure 14-2 Hourly Tally Run1 Exceeds 2 Hours

Description of "Figure 14-2 Hourly Tally Run1 Exceeds 2 Hours"

In this case, the run scheduled for 2:00 begins immediately at 2:25. The fourth run, scheduled for 3:00 begins on time. The second run, scheduled for 1:00 is skipped.

Scheduled Execution

To set up scheduled execution:

In the batch.timed.events parameter, specify the event identifiers. If you have two or more event identifiers, separate each with a comma; blank spaces are not allowed.
In the event_name.name parameter, enter a descriptive label for the event. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.
In the event_name.handlers parameter, specify identifiers for one or more handlers that are to be triggered when the event occurs. If you have two or more handlers, separate each with a comma; blank spaces are not allowed.
In the event_name.at parameter, specify each time when you want execution to occur, in hhmmss, using a 24-hour clock.
In the event_name.file.location parameter, specify the full path name of the directory to monitor for the arrival of new files that match the pattern in event_name.file.pattern.
In the event_name.file.pattern parameter, specify the file name pattern to look for. You can use an asterisk (*) to represent zero or more characters in the file name. No other wild cards (metacharacters) are supported.

Note:

Depending on your configuration, the file pattern might conflict with a file pattern used by another component, such as Rated Event Loader. To prevent conflicts, use a specific pattern, for example, test*.out.

Starting the Batch Controller

Use this command to start the Batch Controller:

start_batch_controller

About SampleHandler

BRM software includes a generic batch handler, called SampleHandler, which you can use with any batch application that processes data from files. The Batch Controller can call this batch handler whenever a specified directory receives a file whose name matches a specified pattern. The input and output files are then moved to directories that you specify.

Preparing SampleHandler for use involves:

If SampleHandler does not meet your needs, you can write your own batch handler, as described in "Writing Custom Batch Handlers" in BRM Developer's Guide.

Copying SampleHandler

The directory BRM_home/apps/sample_handler contains these files:

pin.conf: the batch handler's configuration file for BRM-related parameters.
SampleHandler.pl: the actual code of the batch handler.
SampleHandler_config.values: the batch handler's configuration file for application-specific parameters.

Copy the entire directory and name the copy appropriately. For example, if your new handler is for the Widget application, then you might name the new directory BRM_home/apps/<widget_handler>.

In the new directory, you can rename the SampleHandler_config.values file to include the application's name, such as <widget_handler>_config.values. If you do so, you must also edit the SampleHandler.pl file to change SampleHandler_config.values to the new name.

Customizing SampleHandler

To configure the new batch handler for the desired application:

Open the pin.conf file for the batch handler (BRM_home/apps/<widget_handler>/pin.conf, for example).
Edit the BRM connection parameters. For information, see "Using Configuration Files".
Save and close the batch handler's pin.conf file.
Open the .values file for the batch handler (BRM_home/apps/<widget_handler>/SampleHandler_config.values, unless you have renamed the file).
Ensure that the $HANDLER_DIR entry specifies the full path to the directory containing the batch application's log, input, output, and other files.
Edit the $FILETYPE entry to specify the file name pattern to look for.

The batch handler retrieves all files with this file name pattern from the specified directory.
Note:
- The file name pattern must have the .bc file extension. The Batch Controller automatically appends .bc to each file name before it runs a batch handler.
- You can use an asterisk (*) to represent zero or more characters in the file name. No other wild cards (metacharacters) are supported.
(Optional) To change the batch handler's log file to a directory other than $HANDLER_DIR, edit the $LOGFILE entry to specify the full path to the desired directory.
Edit the $pinUEL entry to specify the name of the application to run.

Ensure that the $pinUELDir entry specifies the full path to the directory containing the application to run.
(Optional) To configure the batch handler to get input files from a directory other than $HANDLER_DIR, edit the $STAGING entry to specify the full path to the desired directory.

The batch handler will move input files from the $STAGING directory to the $PROCESSING directory, where the application will read them.
(Optional) To configure the application to get input files from a directory other than $pinUELDir, edit the $PROCESSING entry to specify the full path to the desired directory. This must be the same directory that is specified as the application's input directory.

The batch handler will move input files from the $PROCESSING directory to the $ARCHIVE or $REJECT directory, depending on the application's exit code. Successfully processed files go into the $ARCHIVE directory, and files with problems go into the $REJECT directory.
(Optional) To store the application's processed files somewhere other than $HANDLER_DIR, edit the $ARCHIVE and $REJECT entries to specify the full paths to the desired directories.
Save and close the batch handler's .values file.

Configuring the Batch Controller

You must identify your new batch handler to the Batch Controller. Edit the Infranet.properties file of the Batch Controller, as described in "Handler Identification".

Starting the New Batch Handler

As with any batch handler, you start this one by starting or restarting the Batch Controller. The Batch Controller monitors the newly specified file location for the arrival of files and, when a file appears, starts the new batch handler. For more information, see "About the Batch Controller".

Before using the new batch handler for your production system, you should try it on a development system.