Once you have created your XML definition file, the Oracle ATG Web Commerce platform can generate the XHTML documents from the items in the repository. To make them available for searching, these documents must be loaded into Oracle ATG Web Commerce Search and indexed. Depending on your ATG environment, data loading may be an operation that is performed occasionally (if the content rarely changes) or frequently (if the content changes often). To be as flexible as possible, the ATG platform provides two approaches to loading the data:
Bulk loading loads all of the data generated from a repository into Search. Each time a bulk load is performed, the ATG platform creates and loads the complete set of XHTML documents for the repository, and Search builds a complete new index.
Incremental loading loads only the data that has changed since the last load. The incremental loader records which repository items have changed since the last incremental or bulk load. It deletes the XHTML documents that represent items that have been deleted, and creates and loads XHTML documents for any items that are new or have been modified. Search then incrementally indexes those documents.
Bulk loading and incremental loading are not mutually exclusive. For some environments, only bulk loading will be necessary, especially if content is updated only occasionally. For other environments, incremental loading will be needed to keep the search content up to date, but even in that case it is important to perform a bulk load occasionally to ensure the integrity of the indexed data. In particular, if you make any structural changes to an indexed repository, you should run a bulk load so Search builds a complete index to replace the existing index. Note that modifying a product catalog often results in structural changes to the catalog repository.
In addition to configuring the Nucleus components described in this section, you must also create a project in Search Administration that manages the indexing process. You configure the project by specifying the content to be indexed and various additional parameters, such as text processing options or a schedule for performing the indexing. For example, you might schedule incremental indexing to take place every four hours, and schedule bulk indexing to be performed once a week. For more information, see the Managing Search Projects chapter in this guide.
