Setting Up Batches

7.4.1.2 Setting Up Batches

OFSBD delivers with a default batch called DLY. The KDD_PRCSNG_BATCH table includes this batch and must contain all batches in the system.

When a batch starts as part of an automated process, it uses the batch names and other start-up information in this table. The DLY processing batch with ALL as the source origin is reserved for instances where one batch load is required, ignoring source systems. If you wish to associate specific source systems to DLY, then the DLY/ALL record must be deleted from the KDD_PRCSNG_BATCH_SRC table.

The following table provides the contents of the KDD_PRCSNG_BATCH table.

Table 7-7 KDD_PRCSNG_BATCH Table Contents

Column Name	Description
PRCSNG_BATCH_NM	Name of the batch , such as DLY.
PRCSNG_BATCH_DSPLY_ NM	Readable name for the batch, such as Daily.
PRCSNG_ORDER	Relative order of a batch run within processing.
EOD_BATCH_NM	Name of the batch that is this batch’s end-of-day. This name is the same as the name for PRCSNG_BATCH_NM if the row represents an end-of-day batch.
PRCSNG_BATCH_NM	Description of this processing batch.

Each row in the KDD_PRCSNG_BATCH table represents a batch. Each batch identifies the batch that is the corresponding end-of day batch. The following examples illustrate this concept.

Single Batch

In this example, the KDD_PRCSNG_BATCH table contains a single batch per day. This is typical of deployment of a single geography for which a solution set does not require detection more than once daily. The KDD_PRCSNG_BATCH table may look similar to the example below.

Table 7-8 Sample KDD_PRCSNG_BATCH Table with Single Batch

PRCSNG_BATCH_NM	PRCSNG_BATCH_DSPLY_NM	PRCSNG_ORDER	EOD_BATCH_NM
DLY	Daily Batch	1	DLY

Single Site Intra-day Processing

In this intra-day batch example, the system is servicing a single time zone but runs an additional batch during the day to identify behaviors related to overnight trading, as shown in the following example.

Table 7-9 Sample KDD_PRCSNG_BATCH Table with Intra-day Processing

PRCSNG_BATCH_NM	PRCSNG_BATCH_DSPLY_NM	PRCSNG_ORDER	EOD_BATCH_NM
MAIN	Main Evening Batch	2	MAIN
MORN	Morning Batch	1	MORN

In this configuration, run the Calendar Manager Utility only during the MORN batch. Refer to Managing Calendar Manager Utility for more information. You can run the Data Retention Manager either in the MORN or MAIN batch. If you run it in the MAIN batch, define at least one buffer partition so that the MORN batch does not fail due to inadequate partitions. Refer to Managing Data Retention Manager, for more information.

Multiple Countries

As an Oracle client loading data through CSA, the system groups various source systems into one processing batch, so that it can call upon a specific batch and load data from specific source systems within that batch. This allows the handling of different batch loads from different countries running on the same staging instance. The association of the source systems to processing batch are captured in the KDD_PRCSNG_BATCH_SRC FSDM table. The following columns are available in this table:

Table 7-10 KDD_PRCSNG_BATCH_SRC FSDM Columns

Column	Data Type	Null	Primary Key	Default Value
PRCSNG_BAT CH_NM	VARCHAR2(20)	Not Null	Yes	DLY To load only the US source for a batch, for example, Batch1, another record, Batch1, needs to be added.
SRC_ORIGIN	VARCHAR2(3)	Not Null	Yes	ALL To load only the US source for a batch, for example, Batch1, another record, US, needs to be added.
SRC_DESC	VARCHAR2(25 5)	Null	No	Productized Daily Processing Batch for all Source Systems

If you want to load only the US source for a batch, for example, Batch1, then another record, US Source System Load, needs to be added.

A single deployment supports detection against data from New York, London, and Hong Kong. In this case, three batches are all end-of-day batches, as the following table describes.

Table 7-11 Sample KDD_PRCSNG_BATCH Table with Multiple Country Processing

PRCSNG_BATCH_NM	PRCSNG_BATCH_DSPLY_NM	PRCSNG_ORDER	EOD_BATCH_NM
HK	Hong Kong	1	HK
LND	London	2	LND
NY	New York	3	NY

Since Hong Kong’s markets open first, this is the first batch. You should run the Calendar Manager and Data Retention Manager at the start of the HK batch.

Upon setup of the batches, Behavior Detection processing begins with the start_mantas_batch.sh shell script. The final step in a batch is calling the end_mantas_batch.sh shell script.