Adding new Data Processing nodes

You can add new Data Processing nodes to your BDD cluster to increase your processing power.

Note: You can also add more Dgraph nodes; for more information, see Adding new Dgraph nodes. You can't add more WebLogic Server nodes without reinstalling.

To do this, you add one or more qualified YARN NodeManager nodes to your Hadoop cluster, then run the bdd-admin script with the reshape-nodes command. The script queries your Hadoop cluster manager (Cloudera Manager, Ambari, or MCS) for the newly-added nodes and automatically installs Data Processing on them. When the script completes, the new nodes are up and ready to accept new jobs.

Note: The bdd-admin script requires the username and password for the Hadoop cluster manager to query it. It will prompt you for this information if the BDD_HADOOP_UI_USERNAME and BDD_HADOOP_UI_PASSWORD environment variables aren't set.

To add a new Data Processing node:

  1. Add one or more YARN NodeManager nodes to your Hadoop cluster. To support Data Processing, the following Hadoop components must be installed on each:
    • Spark on YARN
    • YARN
    • HDFS/MapR-FS
    For instructions on adding new YARN NodeManager nodes, refer to the documentation for your Hadoop distribution.
  2. If you have TLS/SSL enabled, export the public key certificates for the new YARN node(s), then copy them to the directory on the Admin Server defined by HADOOP_CERTIFICATES_PATH in $BDD_HOME/BDD_manager/conf/bdd.conf.
    You can export the certificates by running the following from the new YARN node(s):
    keytool -exportcert -alias <alias> -keystore <keystore_filename> -file <export_filename>
    Where:
    • <alias> is the certificate's alias.
    • <keystore_filename> is the absolute path to your keystore file. You can find this in your Hadoop manager.
    • <export_filename> is the name of the file you want to export the keystore to.
  3. On the Admin Server, go to $BDD_HOME/BDD_manager/bin and run:
    ./bdd-admin.sh reshape-nodes
  4. Enter the username and password for your Hadoop cluster manager, if prompted.