Increasing the numbers of open file descriptors and processes

Regardless of where you put your Dgraph databases, you must increase the maximum numbers of open file descriptors and processes, or the Dgraph may crash during processing.

The number of open file descriptors should have hard and soft limits of 65536, at a minimum. The number of open processes should have a soft limit of 65536 and an unlimited hard limit.

To set these, do the following on each node in your cluster (including Hadoop nodes):

  1. Create a process limit configuration file for the bdd user named /etc/security/limits.d/<bdd>.conf, where <bdd> is the name of the bdd user.
  2. Open <bdd>.confand add the following: 
    <bdd>      soft      nofile      65536
<bdd>      hard      nofile      65536
<bdd>      soft      nproc      65536
<bdd>      hard      nproc      unlimited
    Where <bdd> is the name of the bdd user.
  3. Save and close the file.
  4. Log out and then log back in so that your changes will take effect.
  5. Run the following to verify your changes: 
    ulimit -n
    The above command should output 65536.