Data transformation tasks:
Database-level integration (OLEDB/ADO cursor)
File-based import (bulk insert or SQL Insert)
Note: | Data transformation tasks, the most resource-intensive tasks executed during the load process, are the most likely processes to cause resource problems. |
| Sequence | Task | I/O Location | Active Server |
|---|---|---|---|
1 | A file is transferred from the Web server to the application server. | Inbox directory | Application |
2 | The transformation engine stages the source file into a clean, delimited text file which is then copied to the Inbox directory. | Application server Temp directory and Inbox directory | Application |
3 | The source file is added to the document archive directory. | Data directory | Application |
4 | A user-specific temporary table is created. | Data server (work area) | Data |
5 | For bulk insert, a SQL Server Bulk Insert statement is called, and Oracle SQL Loader is launched on the application server. | Inbox directory | Data (for the statement ) and application and data (for Oracle SQL Loader) |
For SQL insert, the clean, delimited text file runs SQL Insert statements in batches of 100 statements. | Inbox directory | Data | |
6 | Indexes are added to the user-specific temporary table. | Data server (work area) | Data |
7 | The transformation engine executes all calculations and data transformation rules. | Data server (work area) | Data, application, or data and application |
8 | If data is replacing data, a delete action is executed against the active data mart data-segment table. | Data server (data mart) | Data |
9 | The cleaned and transformed data from the user-specific temporary table is posted into the data mart data-segment table. | Data server (work area and data mart) | Data |
10 | The user-specific temporary table is deleted. | Data server (work area) | Data |