Consideration for ETL Performance

The ETL process for Primavera Data Warehouse was designed for multi-core processor systems. Instead of using a serialized process, Java is used to create a multi-threaded process to run concurrent threads of SQL*Plus and SQL*Loader. This can result in multiple run threads on the various servers. This also means that the process can be adversely affected by having to compete with other applications sharing the same resources. Therefore, an ideal configuration would have dedicated cores available for each component in the process and a dedicated database instance with sufficient SGA allocation and high throughput IO system.

This is an ideal configuration that is meant to minimize contention. By dedicating resources to each of the physical components, concurrent performance is maximized during peak usage. Different steps in the ETL process put a variety of load on each component.

Related Topics

Network

Memory

Extract and Load

Oracle Data Integrator (ODI)



Legal Notices | Your Privacy Rights
Copyright © 1999, 2020

Last Published Monday, December 14, 2020