The EndecaIndexingOutputConfig
components specify how to generate records from items in the catalog repository, but the generation itself is performed by data loader components. Depending on your environment, data loading may be an operation that is performed occasionally (if the content rarely changes) or frequently (if the content changes often). To be as flexible as possible, the Guided Search integration provides two approaches to loading the data:
Bulk loading generates the complete set of records for the catalog. Bulk loading is performed by the
atg.endeca.index.RecordStoreBulkLoaderImpl
class. The Guided Search integration includes a component of this class,/atg/search/repository/BulkLoader
.Incremental loading generates only the records that have changed since the last load. The incremental loader records which repository items have changed since the last incremental or bulk load. It deletes the records that represent items that have been deleted, and creates records for any items that are new or have been modified.
Incremental loading is performed by the
atg.endeca.index.RecordStoreIncrementalLoaderImpl
class. The Guided Search integration includes a component of this class,/atg/search/repository/IncrementalLoader
.
Bulk loading and incremental loading are not mutually exclusive. For some environments, only bulk loading will be necessary, especially if content is updated only occasionally. For other environments, incremental loading will be needed to keep the search content up to date, but even in that case, you should perform a bulk load occasionally to ensure the integrity of the indexed data.
Note that Guided Search always does a baseline update after the Oracle Commerce Platform performs bulk loading, and typically does a partial update after incremental loading. In some cases, however, a baseline update may be triggered after incremental loading. For example, if incremental loading adds a new category dimension value, a baseline update must be performed. See EndecaIndexingOutputConfig Components for information about how to configure this.
The IncrementalLoader
component uses an implementation of the PropertiesChangedListener
interface to monitor the repository for add, update, and delete events. It then analyzes these events to determine which ones necessitate updating records, and creates a queue of the affected repository items. When a new incremental update is triggered, the IncrementalLoader
processes the items in the queue, generating and loading a new record for each changed repository item.