Using a merge policy for incremental index updates

A merge policy determines how frequently the Dgraph process of the Endeca data domain merges incremental update generations in its index files, for a specific data domain.

The data layer stores the index files of an Endeca data domain as a series of internal files with versions. As a result:

Old versions can be accessed while new versions are created.
Old versions are garbage-collected when no longer needed.

A version is persisted as a sequence of generation files. A new version appends a new generation file to the sequence. Query latency depends, in part, on the number and size of generation files used to store the index files.

Generation files are combined through a process called merging. Merging is a background task that does not affect the Endeca data domain processing, but may affect its performance. Because of this, you can set a merge policy that dictates the aggressiveness of the merges. In a clustered environment, a request to set the merge policy is routed to the leader Dgraph node.

You can set the merge policy by sending a request to the Configuration Web Service to change the settings for the Global Configuration Record(GCR), utilizing any Web services tool, such as soapUI. Specifically, the mdex-config_MergePolicy attribute in the Global Configuration Record sets the merge policy for the Endeca data domain.

Alternatively, you can use the URL /admin/dataDomainName?op=merge command to change the merge policy of a running data domain, or to force a merge.

If you are running a cluster of nodes, this request is routed to the leader node and processed by it for the data domain's index.