Transform logging

If your transformation script fails to commit, you can learn more about the cause of the failure by looking through the Data Processing logs.

Data Processing writes its logs to a user-specified directory on each Data Processing node in the cluster. The default location is the $BDD_HOME/logs/edp directory.

Each transformation is identified within the logs by the name of the data set it was applied to and the name of the project it originated from. You can use this information locate the messages related to your script and determine which functions caused the failure.