The Endeca data domain and the Dgraph process

The Endeca Server application services requests for one or more Endeca data domains.

An Endeca data domain is a set of one or more Dgraph processes that together handle end-user query requests. One of the Dgraph processes in a data domain is responsible for handling all write operations (updates, configuration changes), while the remaining Dgraph processes serve as read-only. All Dgraph nodes in a given data domain typically utilize the same index residing on shared storage. Each data domain node is represented by the Dgraph process and is typically hosted on a separate Endeca Server instance application running in an Endeca Server cluster.

The Dgraph process for the data domain is referred to as the data domain node.

When you first install the Endeca Server software, it does not contain Endeca data domains. You create data domain profiles and use them to create named data domains.

If you configure a data domain to run on a single Endeca Server instance, it typically also requires a single node that is started once you start the Endeca Server application. If you configure a data domain to run multiple Dgraph processes (such data domains are described as "data domains with multiple nodes"), then the Endeca Server cluster locates ports on which to start the Dgraph processes for the data domain and identifies which Dgraph configuration flags to use.

When you create a data domain, Endeca Server creates an internal .profile file associated with each new data domain and stores it in the index of the data domain. The file contains the configuration of the profile with which the data domain was created. If you later update the data domain, its .profile file is updated. When you export the data domain, the .profile file is exported, and when you import the data domain, this file is used as the data domain's profile for the new data domain.

The .profile file contains the data domain profile info, which is applied to the data domain. Its contents is similar to the following:

computeCacheSizeMB=0
sessionIdType=header
numFollowers=0
numComputeThreads=4
allowOversubscribe=true
shutdownTimeoutSeconds=30
startupTimeoutSeconds=600
args=
idleTimeoutMinutes=15
autoIdle=false
readOnly=false
description=this a customized data domain profile
allowQueriesOnLeader=true
sessionIdKey=X-Endeca-Session-ID

Once an Endeca data domain is created, you only need to use the name of the data domain to manage it. You do not need to know which port the Dgraph processes for the data domain are running on, as the Endeca Server keeps track of that information using its Cluster Coordinator services. This name-only reference to the data domains makes it much easier to enable and disable them and perform other data domain management operations.

Note: A data domain configured to run several Dgraph processes is typically deployed in an Endeca Server cluster. Such a data domain is also known as a data domain cluster. Both types of data domains — the one that runs a single Dgraph process, and the one that runs multiple processes — are managed by the Endeca Server application. Thus, in cases when a multi-process data domain is implied, the term "data domain" may be used interchangeably with the term "data domain cluster".

For information on data domain clusters and the Endeca Server cluster, see the Oracle Endeca Server Cluster Guide.

About the Dgraph process

The Dgraph uses proprietary data structures and algorithms that allow it to provide real-time responses to client requests. It stores the index created from loading the data into it. After the index is stored, the Dgraph receives client requests via the application tier, queries the index, and returns the results. The communication between the Endeca Server and the Dgraph is secure by default.

The Dgraph is designed to be stateless. This design requires that a complete query be sent to it for each request. The stateless design facilitates the addition of Dgraph processes for load balancing and redundancy — any replica of a Dgraph can reply to queries independently of other replicas.