Configuring KMS high availability

BDD Shell can be configured to run in a Hadoop environment that has been set up for Key Trustee KMS high availability (HA).

This procedure assumes that you have already set up Key Trustee KMS (Key Management Server) high availability and that you have already installed BDD.

The Apache Spark that is used by BDD Shell does not support the semi-colon delimited list of KMS nodes that are set up for the HA purpose. Apache Spark only supports one KMS node. The workaround is to modify the core-sites.xml and hdfs-sites.xml configuration files used by BDD Shell and replace the list of KMS nodes with only one KMS node.

To modify the BDD Shell Hadoop configuration files for KMS high availability:

  1. Install BDD Shell.

    The installation creates a directory named hadoop-conf in the BDD Shell root directory, into which the Hadoop configuration files are copied from the $BDD_HOME/common/hadoop/conf directory.

  2. In the hadoop-conf directory, use a text editor to open the core-site.xml file and adjust the hadoop.security.key.provider.path property to refer to only one machine:
    Before: 
    ...
   <property>
      <name>hadoop.security.key.provider.path</name>
      <value>kms://http@kms01.example.com;kms02.example.com:16000/kms</value>
   </property>
...
    After: 
    ...
   <property>
      <name>hadoop.security.key.provider.path</name>
      <value>kms://http@kms01.example.com:16000/kms</value>
   </property>
...
  3. In the same hadoop-conf directory, open the hdfs-site.xml file and make a similar adjustment to the hadoop.security.key.provider.path property to refer to only one machine:
    Before: 
    ...
   <property>
      <name>dfs.encryption.key.provider.uri</name>
      <value>kms://http@kms01.example.com;kms02.example.com:16000/kms</value>
   </property>
...
    After: 
    ...
   <property>
      <name>dfs.encryption.key.provider.uri</name>
      <value>kms://http@kms01.example.com:16000/kms</value>
   </property>
...