Startup of Dgraph nodes

Once the Big Data Discovery cluster is started, it activates the Dgraph processes on a subset of the nodes that are hosting the Dgraph instances. This topic discusses the behavior of the Dgraph nodes at startup.

The startup behavior of a Dgraph is as follows:

A Dgraph starts up without any Dgraph databases mounted.
If a Dgraph gets a Web service request involving a database that it has not mounted, it tries to mount it. The Dgraph mounts the database as a follower node by default or as a leader node if it has already been appointed leader by the Dgraph Gateway.
Follower Dgraphs do not alter the Dgraph database in any way; they continue answering queries based on the version of the database to which they have access at startup, even if the leader Dgraph node is in the process of updating, merging, or deleting the database. Follower Dgraph nodes do not receive update requests; they acquire access to the new database once the updates complete.
The Dgraph can start up without ZooKeeper being up. In this case, the Dgraph comes up in a running but not ready-for-requests state, which means that the Dgraph will not be able to service any requests involving accessing data. The Dgraph continues to wait and connects with ZooKeeper when ZooKeeper comes up on one of the Hadoop nodes in the BDD cluster.
If the Dgraph databases are on HDFS, the Dgraph can start up if HDFS is down. In this scenario, the Dgraph will come up in a running but not ready-for-requests state. A background thread will try to connect to HDFS once a second indefinitely, until the connection to HDFS is successful.

The startup behavior of the Dgraph HDFS Agent is as follows:

Unlike the Dgraph, the Dgraph HDFS Agent will not fully start up until it successfully connects to ZooKeeper. If ZooKeeper is down when the Dgraph HDFS Agent starts, the Dgraph HDFS Agent will report that it failed to start. However, when ZooKeeper comes back up, the Dgraph HDFS Agent immediately continues its initialization without user intervention.
When ZooKeeper is stopped, the Dgraph HDFS Agent is expected to hang. If the Dgraph HDFS Agent is started or re-started in this scenario, the timeout mechanism in the bdd-admin script correctly ends the hang and reports failure.