A.26.2 Sample spark-default.conf Configuration File

Here is the sample code block for creating spark-default.conf file.

spark.driver.port 30303
spark.blockManager.port 31313
spark.driver.bindAddress 0.0.0.0
spark.yarn.dist.files <COMPLIANCE STUDIO INSTALLTION PATH>/deployed/mmg-home/
mmg-studio/interpreter-server/spark-interpreter-<version>/extralibs/spark-
<version>-bin-hadoop<version>/python/lib/pyspark.zip,<COMPLIANCE STUDIO
INSTALLTION PATH>/deployed/mmg-home/mmg-studio/interpreter-server/sparkinterpreter-<
version>/extralibs/spark-<version>-bin-hadoop<version>/python/
lib/py4j-0.10.7-src.zip
spark.executorEnv.PYTHONPATH pyspark.zip:py4j-0.10.7-src.zip
spark.driver.defaultJavaOptions "-Dsun.security.krb5.debug=false -
Djavax.security.auth.useSubjectCredsOnly=false -
Djava.security.krb5.conf=<COMPLIANCE_STUDIO_INSTALLATION_PATH>/deployed/
batchservice/user/conf/krb5.conf"
spark.driver.host <FQDN_HOSTNAME>
spark.yarn.keytab <COMPLIANCE_STUDIO_INSTALLATION_PATH>/deployed/
batchservice/user/conf/fccstudio.keytab
spark.yarn.principal <KRBS_PRINCIPAL>
spark.yarn.kerberos.relogin.period 1m

Note:

• FQDN_HOSTNAME stands for compliance Studio Fully Qualified hostname, and KRBS_PRINCIPAL stands for Kerberos principal.
• For example, the Spark version is spark-2.4.0-bin-hadoop2.7.