As discussed in the previous section, data logging for Commerce Reporting occurs in the Oracle ATG Web Commerce production environment on which the Commerce site runs. Much of the logging occurs in real time as customers use the site. Loading data into the Data Warehouse is performed in a separate Oracle ATG Web Commerce environment, typically (but not necessarily) running on a different group of physical servers. These data loader environment instances do not handle customer requests; their primary purpose is to process data from the log files and load it into the Data Warehouse. Unlike logging, the loading process is not performed in real time. It is run on a regular schedule, typically once a day.

For each type of log file, the data loader environment has a data loader component. When one of these loader components is run, it claims a log file from a queue, then parses the entries in the file in batches, using the Data Warehouse Loader Repository to track the progress of the load operation. The loader component hands off the batches to a pipeline driver component, which parses the data and passes it to a processor pipeline. The pipeline processes the data and stores it in the Data Warehouse.

Copyright © 1997, 2013 Oracle and/or its affiliates. All rights reserved. Legal Notices