3.1.6.2 Configuration

The PySpark interpreter can be configured through the Spark interpreter, with the only exception being the Python version used. By default, the Python version is set to 3 that can be changed either in the interpreter JSON files before the startup or from the Interpreters page of the Compliance Studio application UI during runtime by changing the following properties:

To change the value of the spark.pyspark.python property before installing the Compliance Studio, follow these steps:

1. Navigate to <COMPLIANCE_STUDIO_INSTALLATION_PATH>/deployed/mmg-home/mmgstudio/ server/builtin/interpreters/spark.json directory.
2. Update the value in spark.pyspark.python property of the spark.json file.

To change the value of the spark.pyspark.python property after installing the Compliance Studio, follow these steps:

1. Login to the Compliance Studio application.
2. Launch the CS Production Workspace.
3. Click the User Profile drop-down list and select Data Studio Options.
4. Click Interpreters.
   By default, the Interpreters page lists all the available interpreters on the LHS.

   Figure 3-28 Spark Interpreter

   
   

   
   Description of "Figure 3-28 Spark Interpreter"

   
   
5. Click spark interpreter on the LHS and then click Plain Configuration on the RHS.
6. Update the value in the spark.pyspark.python property and click Update.
   In the Spark Interpreter Settings page of the Compliance Studio application UI (or spark.json file), change the value of the spark.pyspark.python property to the Python executable path that is to be used by the Spark executors.
   In the PySpark Interpreter Settings page of the Compliance Studio application UI (or pyspark.json file), change the value of the zeppelin.pyspark.python property to the Python executable path that is to be used by the Spark driver.