Required Hadoop client libraries

BDD requires a number of client libraries to interact with Hadoop. When the installer runs, it adds these libraries to a single JAR, called the Hadoop fat JAR, which is distributed to all BDD nodes.

How you obtain the client libraries depends on your Hadoop distribution:

• CDH: The installer will download the required libraries automatically. Note that this requires an internet connection on the install machine. If the script can't download all of the client libraries, it will fail and you will have to download them manually. See Failure to download the Hadoop client libraries for more information.
• HDP: Locate the following directories on your Hadoop nodes and copy them to the install machine. Note that they might not all be on the same node.
  • /usr/hdp/<version>/hive/lib/
  • /usr/hdp/<version>/spark/lib/
  • /usr/hdp/<version>/hadoop/
  • /usr/hdp/<version>/hadoop/lib/
  • /usr/hdp/<version>/hadoop-hdfs/
  • /usr/hdp/<version>/hadoop-hdfs/lib/
  • /usr/hdp/<version>/hadoop-yarn/
  • /usr/hdp/<version>/hadoop-yarn/lib/
  • /usr/hdp/<version>/hadoop-mapreduce/
  • /usr/hdp/<version>/hadoop-mapreduce/lib/
• MapR: Locate the following directories on your Hadoop nodes and copy them to the install machine. Note that they might not all be on the same node.
  • /opt/mapr/spark/spark-1.6.1/lib
  • /opt/mapr/hive/hive-1.2/lib
  • /opt/mapr/zookeeper/zookeeper-3.4.5
  • /opt/mapr/zookeeper/zookeeper-3.4.5/lib
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/tools/lib
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn
  • /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib