Additional Configurations for ORAAH Executions

The following configurations are mandatory for model executions using ORAAH.

Set the following environment variables in $R_HOME/etc/Renviron.site file:

  • HIVE_HOME, SPARK_HOME, HADOOP_HOME with the respective paths
  • HIVE_CONF_DIR, HADOOP_CONF_DIR, YARN_CONF_DIR, SPARK_CONF_DIR with their respective configuration directory paths
  • CLASSPATH and HADOOP_CLASSPATH with all the hadoop/hdfs/yarn/hive jars, Hadoop configuration directory (HADOOP_CONF_DIR) and spark configuration directory (SPARK_CONF_DIR)

    For example,

    CLASSPATH=$HADOOP_CONF_DIR:$SPARK_CONF_DIR:All_hadoop_jars

  • SPARK_JAVA_OPTS variable with $R_HOME/lib

    For example, SPARK_JAVA_OPTS="-Djava.library.path=/usr/lib64/R/lib"

  • For Kerberos enabled cluster, initializing the ticket should be done in Renviron/Renviron.site file.