hadoop

The hadoop configuration type makes changes to BDD's Hadoop configuration.

Depending on the specified options, hadoop can:

Publish new or updated Hadoop client configuration files to your BDD cluster.
Reset the HUE_URI property in $BDD_HOME/BDD_manager/conf/bdd.conf (HDP only).
Switch to a different version of your Hadoop distribution without reinstalling BDD. Note, however, that it can't be used to switch to a different Hadoop distribution.

Note: The script requires the username and password for your Hadoop cluster manager if the BDD_HADOOP_UI_USERNAME and BDD_HADOOP_UI_PASSWORD environment variables aren't set.

To update BDD's Hadoop configuration, run the following from the Admin Server:

./bdd-admin.sh publish-config hadoop [option <arg>]

hadoop supports the following options.

Option	Description
`-u, --hueuri <host:port>`	HDP clusters only. Sets the `HUE_URI` property in bdd.conf to the specified URI.
`-l, --clientlibs <path[,path]>`	Regenerates the Hadoop fat jar from a comma-separated list of client libraries. `<path[,path]>` must be a comma-separated list of the new libraries. This can be used to switch to a different version of your Hadoop distribution. This must be run with `--sparkjar`.
`-j, --sparkjar <file>`	Sets the location of the Spark on YARN jar in all BDD configuration files to the specified path. `<file>` must be the absolute path to the Spark on YARN jar on the Hadoop nodes. This can be used to switch to a different version of your Hadoop distribution. Note that unless the location of your Hadoop installation has changed, you can use the value of `SPARK_ON_YARN_JAR` in bdd.conf. Be sure to double-check the path, just in case. This must be run with `--clientlibs`.

Option

Description

-u, --hueuri <host:port>

HDP clusters only. Sets the HUE_URI property in bdd.conf to the specified URI.

-l, --clientlibs <path[,path]>

Regenerates the Hadoop fat jar from a comma-separated list of client libraries. <path[,path]> must be a comma-separated list of the new libraries. This can be used to switch to a different version of your Hadoop distribution.

This must be run with --sparkjar.

-j, --sparkjar <file>

Sets the location of the Spark on YARN jar in all BDD configuration files to the specified path. <file> must be the absolute path to the Spark on YARN jar on the Hadoop nodes. This can be used to switch to a different version of your Hadoop distribution.

Note that unless the location of your Hadoop installation has changed, you can use the value of SPARK_ON_YARN_JAR in bdd.conf. Be sure to double-check the path, just in case.

This must be run with --clientlibs.

If no options are specified, the script publishes the Hadoop client configuration files to all BDD nodes and updates the Hadoop-related properties in all BDD configuration files.

For more information on the actions performed by this configuration type, see: