7.2 GGSA Pipeline

Pipelines are deployed to Spark cluster and you can retrieve logs from the Spark console or from the command line. Log into Spark console at https://<IP_of_Instance/spark. You are prompted for username and password which you can obtain from the README.txt file. Once you provide these credentials, you should see a list of Running Applications. Click the Application ID for which you want to retrieve logs.

Click on stderr or stdout links in the Log column.

When running in the local Spark cluster, logs can also be accessed by navigating to /u01/app/spark/work folder. Run ls -ltr to view both application and driver logs. For example, you can navigate to Driver logs, as shown in the screenshot below.

Similarly you can navigate to application or pipeline logs, as shown in the screenshot below.

For troubleshooting specific UI issues, see Troubleshoot.