Oracle Cloud Infrastructure Documentation

Viewing Run Logs

View the logs for a Run in Data Flow.

Each run of an application generates a set of output logs:
  • Application logs, such as stdout and stderr. These logs are available immediately after the Run completes.
  • Diagnostics logs, such as Spark driver and executor logs. These logs are uploaded every 10 or 12 minutes into the Run execution. So, depending on the time the Run run takes, these logs aren't always uploaded before it finishes, and possibly more than one set of diagnostic logs is uploaded.
Note

The value of Data Read in a Run might be different to the amount of source data, as it represents the total amount of data read across all stages of the Run. Similarly the value of Data Written in a Run might be different to the amount of source data, as is represents the total amount of data written across all stages of the Run.
  • To view the Spark UI or Data Flow logs, you must allow cookies in your browser for oraclecloud.com. If they're not allowed, you might see an "HTTP Error 401 - Unauthorized Request" error message.
    If you have enabled Oracle Cloud Infrastructure Logging, you see Logs in the Run details page after the Run goes to In Progress state.
    1. On the Runs list page, select the name of the Run you want to view the logs for. If you need help finding the Runs list page, see Listing Runs in Data Flow.
    2. On the Run's details page, under Resources, select Logs.
      Links to the stdout, application stderr, Spark driver, and one for each of the Spark executors. If there several executors, the logs might be paginated.
    3. Select the log link to take you to Oracle Cloud Infrastructure Logging page to view the log.
      For information on viewing the logs, see the Oracle Cloud Infrastructure Logging documentation.

  • Use the get-log command and required parameters to view the logs for a run in a specific compartment:

    oci data-flow run get-log [OPTIONS]

    For a complete list of flags and variable options for CLI commands, see the CLI Command Reference.

  • Run the GetRunLog operation to get the logs of a run of an application in a specific compartment.