8 Understanding Batch Processing

Oracle Application Integration Architecture (AIA) uses an extract, transform, and load (ETL) tool called Oracle Data Integrator (Oracle ODI) for high-performance movement and transformation of very large volumes of data between heterogeneous systems. Batch, real time, synchronous, and asynchronous modes are supported. Based on the relevant RDBMS engines and SQL, Oracle AIA can transform data on the target server at a set-based level. This chapter discusses use cases for batch processing and discusses bulk data integration patterns.

For more information about batch processing, see "Using Oracle Data Integrator for Bulk Processing" in the Oracle Fusion Middleware Developer's Guide for Oracle Application Integration Architecture Foundation Pack.

This chapter includes the following sections:

• Section 8.1, "Batch Processing Use Cases"

• Section 8.2, "Bulk Data Integration Patterns"

8.1 Batch Processing Use Cases

Oracle AIA uses batch processing technology for these use cases:

• To perform an initial synchronization of reference data across disparate applications.

• To load an Operational Data Store to provide fresh, integrated information.

• To load production databases from data entered over the Internet (by sales forces, agencies, suppliers, customers, and third parties) that strictly respects security constraints.

• To use the Cross Reference and Domain Value Map when the data transferred runs services from an Integration Scenario.

8.2 Bulk Data Integration Patterns

Oracle AIA supports the following bulk data integration patterns:

• Initial data loads

• High volume transactions with XREF table

• High volume transactions without XREF

• Intermittent high volume transactions

8.2.1 Initial Data Loads and High Volume Transactions with XREF Table

When you implement a new integration, you may need to synchronize some records. Using Oracle ODI, you can handle a large data-set and maintain the cross-reference table if necessary.

8.2.2 High Volume Transactions Without XREF

If synchronized records require no further DML operations, maintaining a cross-reference (XREF) record is unnecessary. For example, if retail stores transmit daily sales transactions to HQ, and if HQ requires no DML operations (as in a data-warehouse), then maintaining the cross-reference table is unnecessary.

8.2.3 Intermittent High Volume Transactions

Use the Intermittent High Volume Transactions pattern when both Oracle ODI and AIA EBS manage data integration. AIA EBS handles normal instance-based synchronization. Oracle ODI handles synchronization for batch transactions. Oracle ODI can also handle intermittent transactions containing very large objects, such as orders with many lines.