Choosing a data loading strategy

Strategies for loading data into Endeca Server fall into two general categories: baseline updates and incremental updates.

A baseline update loads data into an empty data domain. Baseline updates include full initial load of data and subsequent baseline updates. An incremental update adds records to an existing data domain, while an incremental update with a delta also modifies existing records in a loaded data domain.

The following table describes these strategies in detail.

Table 1. Data loading strategies
Data loading strategy	Type	Description	When to use this strategy
Full initial load	Baseline update	A full initial load is a simple load of data into an empty data domain. The data domain may have been configured, and a schema may have been loaded. If no schema configurations have been loaded, a default schema is created when the data is loaded. The Baseline graph in the Getting Started project performs an initial load, after loading a basic attribute schema.	Use this strategy the first time you load data into a new data domain. This strategy typically uses the Bulk Add/Update Records component to load new records quickly.
Subsequent baseline	Baseline update	Also known as a re-baseline, a subsequent baseline completely replaces the existing contents of an existing data domain while preserving the configuration and schema. A typical subsequent baseline includes the following graphs: An export configuration graph to preserve the existing configuration of the data domain. This graph uses the Export Config component. A reset graph to reset the data domain to a pristine state. This graph uses the Reset component. An import configuration graph that imports the previously exported configuration into the data domain after reset. This graph uses the Import Config component. A set of graphs to reload the data to the reconfigured data domain. Typical practice is to run these graphs within an outer transaction using the Transaction RunGraph component.	Use this strategy to refresh the data domain periodically to ensure data is up to date and not corrupt.
Incremental update	Incremental update	An incremental update adds new records to a data domain, and new properties to existing records.	Use this strategy when you want to add new records to a data domain, or to add new assignments to existing records. This strategy typically uses the Merge Records component. Note that this component adds or modifies only one Endeca record per input record.
Incremental update with delta	Incremental update	An incremental update with delta: Adds new assignments to existing records Modifies existing assignments on existing records Deletes assignments from a record Deletes complete records	Use this strategy when you want to modify or remove specific records. This strategy uses either the Modify Records component (to add, modify, or remove assignments) or the Delete Records component (to delete complete records). Note that both components can modify or delete multiple Endeca records per input record.