As mentioned above, the loader component initiates the data loading process, but the actual processing of the data is performed by a processor pipeline. The pipeline is invoked by a pipeline driver component to which the loader passes batches of log entries. The pipeline driver calls a pipeline manager component, which controls the pipeline. The processors in the pipeline perform such tasks as looking up dimensional data in the warehouse; looking up profile, catalog, and order data in repositories on the production site; and writing data about each item in an order to the warehouse.
The pipeline driver components, like the data loader components, are located in the /atg/reporting/datawarehouse/loaders/ Nucleus folder. Each loader component has a pipelineDriver property that points to the pipeline driver it invokes. For example, the OrderSubmitLoader.properties file includes:
pipelineDriver=OrderSubmitPipelineDriver
All of the pipeline drivers point to the same pipeline manager component, /atg/reporting/datawarehouse/process/PipelineManager. This pipeline manager’s definition file, /atg/reporting/datawarehouse/process/pipeline.xml, defines several data loading processor chains. The specific chain invoked by an individual pipeline driver is specified by its pipelineChainId property. For example, the OrderSubmitPipelineDriver.properties file includes:
pipelineManager=../process/pipelineManager
pipelineChainId=submitOrder
The following table summarizes the data loaders, their pipeline drivers, and the processor chains they invoke:
Data Loader | Pipeline Driver | Processor Chain |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
