8 Controlling Batch Operations

This chapter explains how to configure the Batch Controller to launch batch handlers to check or process data for your Oracle Communications Billing and Revenue Management (BRM) system.

About the Batch Controller

The BRM Batch Controller lets you specify when to run programs or scripts automatically, either at timed intervals or upon creation of certain files, such as log files.

The Batch Controller configuration file (BRM_home/apps/batch_controller/Infranet.properties) has entries that tell the Batch Controller when to run the specified batch handlers. These programs or scripts can be triggered at specified times or by specified kinds of occurrences, such as creation of a new log file. You can specify scheduled execution, starting the handler at fixed times of the day only, or metronomic execution, starting the handler repeatedly at a fixed interval.

A batch handler can be any executable program or script that can be run from a command line. For more information about the requirements for writing batch handlers, see "Writing Custom Batch Handlers" in BRM Developer's Guide.

Only one Batch Controller can run on a single computer, but it can control many batch handlers to launch various applications.

BRM installation includes a generic batch handler, called SampleHandler, which you can use with most applications. See "About SampleHandler".

Setting Activity Times and Triggers

Certain general parameters apply to all batch activity. Other parameters identify the batch handlers and regulate when those handlers are to be run.

General Batch Controller Parameters

The Batch Controller's Infranet properties file specifies how to connect the Batch Controller to the Connection Manager (CM), when to allow high batch traffic, how much logging to record, and how long to wait for handlers.

Connection Parameters

Batch Controller requires the following parameters to connect to a CM: a user ID and password (specified in the infranet.connection parameter), the CM's port number, and the address of the CM's host computer.

Tip:

To only use a user ID, set the infranet.login.type parameter to 0; to use both a user ID and a password, set infranet.login.type to 1.

In the batch.lock.socket.addr parameter, specify the socket number to lock on. If in doubt, check with your system administrator for the number of an unused socket that can be used exclusively by the Batch Controller. You can use the default value, 11971, unless some other application is using that socket number.

To write a stack trace of any runtime exceptions in the log file, set infranet.log.logallebuf to True. To disable this feature, set it to False.

Time-to-Run Parameters

When your system's heaviest load typically occurs, you might want some handlers to run less often than they do when the load is lighter. The Batch Controller divides the day into high-load and low-load periods to balance the demand for system resources.

Specify the starting time of your system's busiest period in the batch.start.highload.time parameter. Specify the starting time of your system's slowest period in the batch.end.highload.time parameter. For each of these times, specify the hour, minute, and second, in hhmmss, using the 24-hour clock. For each handler, you can specify the maximum number of simultaneous actions during each of these two periods.

The end parameter must be greater than the start parameter. If you do specify start, it must be greater than end. Specifying the same value for both parameters causes an error.

In the batch.check.interval parameter, specify the time, in seconds, between checks for occurrences of the type specified in batch.timed.events or batch.random.events. Omitting batch.check.interval causes an error.

Log-File Parameter

For logging, you can specify the level of information reported, the full path of the log file, and whether to print a stack trace of runtime exceptions. Set infranet.log.level to 0 for no logging; to 1 for error messages only; to 2 for error messages and warnings; or to 3 for error messages, warnings, and debug messages. Set infranet.log.file to the path and file name for the Batch Controller log file.

If you omit infranet.log.file, BRM uses default.pinlog in the current directory. Omitting infranet.log.level causes an error.

Timeout-Limit Parameters

You can also set timeout limits for handlers to start their objects and to complete their execution. Set batch.handler.start.wait to the number of seconds allowed for the handler to update its own object status from STARTING to STARTED, and set batch.handler.end.wait to the number of seconds allowed for updating from STARTED to some other state, such as COMPLETED. See "Writing Custom Batch Handlers" in BRM Developer's Guide for descriptions of all of the handler states.

Note:

Omitting either batch.handler.start.wait or batch.handler.end.wait causes an error.

Example of Parameters

This example demonstrates the general parameters:

infranet.connection       pcp://root.0.0.0.1:mypass@myserver:11960/service/pcm_client
infranet.login.type        1
infranet.log.logallebuf    true
batch.lock.socket.addr     11971
batch.start.highload.time  083000
batch.end.highload.time    211500
infranet.log.file 
       BRM_home/apps/batch_controller/batch.pinlog
infranet.log.level         2
batch.handler.start.wait   600
batch.handler.end.wait     43200

In this example, the Batch Controller logs on to the CM host on myserver, port 11960, as user root.0.0.0.1, using password mypass. High usage is expected between 8:30 a.m. and 9:15 p.m. Logging writes a normal level of information to file batch.pinlog and prints a stack trace if any program errors are found. Timeouts for updating object status are 10 minutes (600 seconds) for going to STARTED and 12 hours (43,200 seconds) for going to COMPLETED or some other state.

Handler Identification

To identify each of the batch handlers:

  1. In the handler_name.name parameter, enter a descriptive label for the handler. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.

  2. In the handler_name.max.at.highload.time parameter, specify the highest number of instances of this batch handler that are permitted to run simultaneously during the time from batch.start.highload.time to batch.end.highload.time.

  3. In the handler_name.max.at.lowload.time parameter, specify the highest number of instances of this batch handler that are permitted to run simultaneously during the low-usage time; the time from batch.end.highload.time to batch.start.highload.time.

  4. In the handler_name.start.string parameter, specify the command line to start this batch handler.

    Note:

    When the Batch Controller issues this command, it appends -p handler_poid -d failed_handler_poid to the command, as described in "Writing Custom Batch Handlers" in BRM Developer's Guide. If you are not using custom batch handlers, you can ignore these options.

This example demonstrates the identification parameters:

handler1.name                     Visa Handler #1
. . .
handler1.max.at.lowload.time      4
handler1.max.at.highload.time     2
handler1.start.string    BRM_home/apps/visa-handler/visa.pl

handler2.name                     Discover Handler #1
. . .
handler2.max.at.lowload.time      5
handler2.max.at.highload.time     3
handler2.start.string      BRM_home/apps/discover-handler/discover -y 2001

In this example, the internal name Visa Handler #1 applies to the program started by the command string BRM_home/apps/visa-handler/visa.pl, which runs a Perl script. The parameters in the above example limit this program to one or two concurrent actions during the specified high-load period or as many as four during the low-load period.

The other batch handler in.1 this example, Discover Handler #1, runs the application

BRM_home/apps/discover-handler/discover with the additional option -y 2001.

Occurrence-Driven Execution

To trigger execution based on specified occurrences:

  1. In the batch.random.events parameter, specify the event identifiers. If you have two or more event identifiers, separate each with a comma, but no blank space.

  2. In the event_name.name parameter, enter a descriptive label for the event. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.

  3. In the event_name.handlers parameter, specify the identifiers of one or more handlers to trigger when the event occurs. You must specify at least one handler. If you have two or more handlers, separate each with a comma; no blank spaces are allowed.

  4. In the event_name.file.location parameter, specify the full path name of the directory to monitor for the arrival of new files that match the pattern in event_name.file.pattern.

  5. In the event_name.file.pattern parameter, specify the file name pattern to look for. You can use an asterisk (*) to represent zero or more characters in the file name. No other wild cards (metacharacters) are supported.

    Important:

    Depending on your configuration, the file pattern might conflict with a file pattern used by another component, such as Rated Event Loader. To prevent conflicts, use a specific pattern, for example, test*.out.

    You can also check the Pipeline Manager registry to see if any temporary file output patterns conflict with this entry.

    Caution:

    You must specify batch.timed.events or batch.random.events or both. Specifying neither causes the Batch Controller to shut down just after it starts because it has no tasks to perform.

When the Batch Controller starts, it tests the file name pattern against every file name in the specified directory and runs the batch handler for each file where the name matches the pattern. It then monitors the files entering that directory and runs the batch handler whenever it finds a match.

Note:

By default, the Batch Controller appends .bc to the file name of each file it processes. This prevents batch handlers from retrieving files that the Batch Controller has yet to process. You can change the default .bc extension by editing the batch.file.rename.extension entry in the Batch Controller Infranet.properties file.

This example demonstrates the occurrence parameters:

batch.random.events               event1,event3
. . .
event1.name                       Random Discover file arrival 
event1.handlers                   handler1
event1.file.location              /apps/discover/stagingDir
event1.file.pattern               *.*.6011.*
. . .
event3.name                       Random Visa file arrival 
event3.handlers                   handler3
event3.file.location              /apps/visa/stagingDir
event3.file.pattern               *.dat
. . .
handler1.name                     Discover UEL Handler
handler1.max.at.lowload.time      6
handler1.max.at.highload.time     3
handler1.start.string             /apps/discover -uel
. . .
handler3.name                     Visa UEL Handler
handler3.max.at.lowload.time      8
handler3.max.at.highload.time     4
handler3.start.string             /apps/visa -uel

In this example, event1 is triggered when any file name in the /apps/discover/stagingDir directory matches the pattern *.*.6011.*. This match causes the Batch Controller to issue the command /apps/discover -uel, which runs handler1, the Discover UEL Handler program. Based on the parameters shown here, the Batch Controller starts no more than six concurrent instances of this program during the specified period of low expected load, or three during the high-load period.

Timed Execution

The Batch Controller provides two time-based scheduling options: metronomic execution and scheduled execution.

Metronomic Execution

To set up metronomic execution:

  1. In the batch.timed.events parameter, specify the event identifiers. If you have two or more event identifiers, separate each with a comma; blank spaces are not allowed.

  2. In the event_name.name parameter, enter a descriptive label for the event. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.

  3. In the event_name.handlers parameter, specify identifiers for one or more handlers to trigger when the event occurs. If you have two or more handlers, separate each with a comma; blank spaces are not allowed.

  4. (Optional) In the event_name.start parameter, specify when you want the first execution to occur, in hhmmss, using the 24-hour clock. If you omit this parameter, the first execution occurs immediately after the Batch Controller starts.

  5. In the event_name.interval parameter, specify the frequency, in seconds, for triggering the associated handler. Failing to specify the interval causes an error.

  6. (Optional) In the event_name.count parameter, specify how many times to execute this batch handler. If you do not specify this limit, batch handlers run repeatedly at the fixed interval for as long as the Batch Controller is running.

    Caution:

    You must specify batch.timed.events or batch.random.events or both. Specifying neither causes the Batch Controller to shut down just after it starts because it has no tasks to perform.

This example demonstrates the metronomic parameters:

batch.timed.events                event4
. . .
event4.name                       Hourly tally
event4.handlers                   handler4
event4.start                      000000
event4.interval                   3600
event4.count                      4

In this example, the occurrence specified as event4 is named Hourly tally. It runs handler4 for the first time at midnight (000000), and then runs it again every hour (3600 seconds) after that until it has run four times. If it cannot run at a scheduled time because previous executions are not finished, it runs again immediately as soon as possible. For example, consider the timeline in Figure 8-1 for event4, above:

Figure 8-1 Hourly Tally Run 1 Exceeds 1 Hour

Description of Figure 8-1 follows
Description of ''Figure 8-1 Hourly Tally Run 1 Exceeds 1 Hour''

In this example, the first run of handler4 continues for more than an hour, taking it past the time when the second run is supposed to begin. The second scheduled run cannot start at the one-hour interval, so it starts as soon as possible after that (1:20 a.m.). The third and fourth scheduled executions start at regular multiples of the interval, measured from the first run.

If the overly long run continues past two scheduled run start times (occurrences), only one run starts on the delayed basis. For example, suppose the midnight run lasts until 2:25 a.m., as shown in Figure 8-2:

Figure 8-2 Hourly Tally Run1 Exceeds 2 Hours

Description of Figure 8-2 follows
Description of ''Figure 8-2 Hourly Tally Run1 Exceeds 2 Hours''

In this case, the run scheduled for 2:00 begins immediately at 2:25. The fourth run, scheduled for 3:00 begins on time. The second run, scheduled for 1:00 is skipped.

Scheduled Execution

To set up scheduled execution:

  1. In the batch.timed.events parameter, specify the event identifiers. If you have two or more event identifiers, separate each with a comma; blank spaces are not allowed.

  2. In the event_name.name parameter, enter a descriptive label for the event. This name can include spaces or any other characters. It can be of any length, but short names are easier to read.

  3. In the event_name.handlers parameter, specify identifiers for one or more handlers that are to be triggered when the event occurs. If you have two or more handlers, separate each with a comma; blank spaces are not allowed.

  4. In the event_name.at parameter, specify each time when you want execution to occur, in hhmmss, using a 24-hour clock.

  5. In the event_name.file.location parameter, specify the full path name of the directory to monitor for the arrival of new files that match the pattern in event_name.file.pattern.

  6. In the event_name.file.pattern parameter, specify the file name pattern to look for. You can use an asterisk (*) to represent zero or more characters in the file name. No other wild cards (metacharacters) are supported.

    Important:

    Depending on your configuration, the file pattern might conflict with a file pattern used by another component, such as Rated Event Loader. To prevent conflicts, use a specific pattern, for example, test*.out.

    You can also check the Pipeline Manager registry to see if any temporary file output patterns conflict with this entry.

This example demonstrates the schedule parameters:

batch.timed.events                event5
. . .
event5.name                       Sporadic tally
event5.handlers                   handler5
event5.at                         004500, 022500, 045500
event5.file.location              /apps/discover/stagingDir
event5.file.pattern               *.*.6011.*
. . .
handler5.name                     Sporadic Handler
handler5.max.at.lowload.time      8
handler5.max.at.highload.time     4
handler5.start.string             /apps/sporadic-uel

In this example, the program specified as event5 is named Sporadic tally. It runs only at 12:45 a.m., 2:25 a.m., and 4:55 a.m. If a program cannot run at a scheduled time because previous executions are not finished, it runs as soon as possible.

Starting the Batch Controller

Use this command to start the Batch Controller:

start_batch_controller

About SampleHandler

BRM software includes a generic batch handler, called SampleHandler, which you can use with any batch application that processes data from files. The Batch Controller can call this batch handler whenever a specified directory receives a file whose name matches a specified pattern. The input and output files are then moved to directories that you specify.

Preparing SampleHandler for use involves:

  1. Copying SampleHandler

  2. Customizing SampleHandler

  3. Configuring the Batch Controller

  4. Starting the New Batch Handler

If SampleHandler does not meet your needs, you can write your own batch handler, as described in "Writing Custom Batch Handlers" in BRM Developer's Guide.

Copying SampleHandler

The directory BRM_home/apps/sample_handler contains these files:

  • pin.conf: the batch handler's configuration file for BRM-related parameters.

  • sample_template.xml: used by Universal Event (UE) Loader only.

  • SampleHandler.pl: the actual code of the batch handler.

  • SampleHandler_config.values: the batch handler's configuration file for application-specific parameters.

  • SampleReload.pl: used by UE Loader only.

Copy the entire directory and name the copy appropriately. For example, if your new handler is for the Widget application, then you might name the new directory BRM_home/apps/<widget_handler>.

In the new directory, you can rename the SampleHandler_config.values file to include the application's name, such as <widget_handler>_config.values. If you do so, you must also edit the SampleHandler.pl file to change SampleHandler_config.values to the new name.

Customizing SampleHandler

To configure the new batch handler for the desired application:

  1. Open the pin.conf file for the batch handler (BRM_home/apps/<widget_handler>/pin.conf, for example).

  2. Edit the BRM connection parameters. For information, see "Using Configuration Files to Connect and Configure Components".

  3. Save and close the batch handler's pin.conf file.

  4. Open the .values file for the batch handler (BRM_home/apps/<widget_handler>/SampleHandler_config.values, unless you have renamed the file).

  5. Ensure that the $HANDLER_DIR entry specifies the full path to the directory containing the batch application's log, input, output, and other files.

  6. Edit the $FILETYPE entry to specify the file name pattern to look for.

    The batch handler retrieves all files with this file name pattern from the specified directory.

    Important:

    The file name pattern must have the .bc file extension. The Batch Controller automatically appends .bc to each file name before it runs a batch handler.

    Tip:

    You can use an asterisk (*) to represent zero or more characters in the file name. No other wild cards (metacharacters) are supported.

  7. (Optional) To change the batch handler's log file to a directory other than $HANDLER_DIR, edit the $LOGFILE entry to specify the full path to the desired directory.

  8. Edit the $pinUEL entry to specify the name of the application to run.

    Ensure that the $pinUELDir entry specifies the full path to the directory containing the application to run.

  9. (Optional) To configure the batch handler to get input files from a directory other than $HANDLER_DIR, edit the $STAGING entry to specify the full path to the desired directory.

    The batch handler will move input files from the $STAGING directory to the $PROCESSING directory, where the application will read them.

  10. (Optional) To configure the application to get input files from a directory other than $pinUELDir, edit the $PROCESSING entry to specify the full path to the desired directory. This must be the same directory that is specified as the application's input directory.

    The batch handler will move input files from the $PROCESSING directory to the $ARCHIVE or $REJECT directory, depending on the application's exit code. Successfully processed files go into the $ARCHIVE directory, and files with problems go into the $REJECT directory.

  11. (Optional) To store the application's processed files somewhere other than $HANDLER_DIR, edit the $ARCHIVE and $REJECT entries to specify the full paths to the desired directories.

  12. Save and close the batch handler's .values file.

Configuring the Batch Controller

You must identify your new batch handler to the Batch Controller. Edit the Infranet.properties file of the Batch Controller, as described in "Handler Identification".

Starting the New Batch Handler

As with any batch handler, you start this one by starting or restarting the Batch Controller. The Batch Controller monitors the newly specified file location for the arrival of files and, when a file appears, starts the new batch handler. For more information, see "About the Batch Controller".

Before using the new batch handler for your production system, you should try it on a test system.