E.3 Ingestion Timeline - Intra-Day Ingestion Processing

The following figure provides a high-level flow of the intra-day ingestion process of extracting, transforming, and loading data.

Figure E-1 Intra-Day Data Management Processing

Description of "Figure E-1 Intra-Day Data Management Processing"

Intra-day processing references different processing groups as Figure 58 illustrates , such as beginning-of-day processing and intra-day processing. Multiple batches run throughout the day. As in Figure 58, you configure batch ONE, load and extract data, and then start processing. (Data for OpenOrder and CorporateAction is not included.) When batch ONE processing is complete, batch TWO processing begins. The same occurs for all other batches until all batch processing is complete.

You can run intra-day processing and add or omit detection runs at the end of (non end-of-day) ingestion batch runs. These cycles of detection should only run BEX and some TC scenarios. They detect only against that day’s data and/or data for open batches, dependent on each scenario against which each batch is running. The last intra-day batch should be configured as the end-of-day batch.

You must run a final end-of-day batch that detects on all data loaded into the database for that day, not only looking at the batch that was last loaded. The system can display these events on the next day.

If you want to use either types of intra-day ingestion, you must set up intra-day batches and one endof- day batch. If you do not, the FDT processes more market data than necessary and runs for a long period.

Table E-2 Processing Batch Table Set-up

ONE	Intra-Daybatch 1	1	NNN
TWO	Intra-Daybatch 2	2	NNN
NNN	Intra-Daybatch N+ end of day	3	NNN