Installing FUSE

Filesystem in Userspace (FUSE) enables unprivileged users to access filesystems without having to make changes to the kernel. In the context of BDD, it enables the Dgraph to read and write data to HDFS by making HDFS behave like a mountable local disk. The Dgraph supports FUSE 2.8+.

Note: FUSE isn't supported for Hadoop clusters that have MapR, CDH 5.7.x or higher, or HDFS data at rest encryption.

If you're not using the HDFS NFS Gateway service, FUSE must be installed on all HDFS DataNodes that will host the Dgraph. Additionally, the bdd user requires extra permissions to enable the Dgraph process to integrate with FUSE, and socket timeouts in HDFS must be increased to prevent FUSE and the Dgraph from crashing during parallel ingests.

To install FUSE:

  1. Download the FUSE client from https://github.com/libfuse/libfuse/releases.
    The fuse-<version>.tar.gz file is downloaded to your machine.
  2. Extract fuse-<version>.tar.gz:
    tar xvf fuse-<version>.tar.gz
    This creates a directory called /fuse-<version>.
  3. Copy /fuse-<version> to all nodes that will host the Dgraph.
  4. On each node, install FUSE by going to /fuse-<version> and running:
    ./configure
    make -j8
    make install
  5. On each Dgraph node:
    1. Add the bdd user to the fuse group.
    2. Give the bdd user read and execute permissions for fusermount.
    3. Give the bdd user read and write permissions for /dev/fuse.
  6. Update your HDFS configuration:
    1. Open hdfs-site.xml in a text editor and add the following lines:
      <property>
           <name>dfs.client.socket-timeout</name>
           <value>600000</value>
      </property>
      <property>
           <name>dfs.socket.timeout</name>
           <value>600000</value>
      </property>
      <property>
           <name>dfs.datanode.socket.write.timeout</name>
           <value>600000</value>
      </property>
      
    2. Make the following changes in your Hadoop manager.
      If you have CDH, open Cloudera Manager and add the above lines to the following properties:
      • HDFS Service Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml
      • DataNode Advanced Configuration Snippet (Safely Valve) for hdfs-site.xml
      • HDFS Client Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml
      If you Have HDP, open Ambari and set the following properties to 600000:
      • dfs.client.socket-timeout
      • dfs.datanode.socket.write.timeout
      • dfs.socket.timeout
    3. Restart HDFS to make your changes take effect.