9.1 Accessing Data Ingestion Errors Dashboard via DFCS

Data ingestion pipelines receive files submitted by users and load the data into target database tables. The ingestion process runs through multiple stages. At each stage, the data is validated and filtered before it moves to the next stage.

  1. Log in to the DFCS application.
  2. Navigate to Data Ingestion Errors under Data Services.

    The Data Errors window appears.

  3. Select the following.
    Action Description
    Select a Pipeline The dropdown shows all pipelines available in the tenant. Select the pipeline you want to check.

    Note:

    Only pipelines that have been executed will appear in this list.
    Select the As-of Date The As-of Date dropdown shows the date of the source file used in the execution. Select the date you want to review.
    Select the Run SKey A pipeline can be executed multiple times. Each execution gets a unique number called a Run SKey. Use this dropdown to select the specific execution you want to check.
    Example: If you ran the same pipeline three times on the same day:
    Run SK Execution Time Use This To
    96 Morning run Check errors from the morning run
    97 Afternoon run Check errors from the afternoon run
    98 Night run Check errors from the night run
  4. Click Submit.
  5. The Summary Page loads automatically, the page shows a table of application connectors and EDD for that execution.
  6. Use search bar to find connectors (case-insensitive).

    You will see:

    What You See What It Means
    Red error icon next to EDD Records failed at the EDD stage.
    Red error icon next to Connector Records failed at the connector stage.
    No icon (icon is greyed out) No errors at this stage
  7. Click the red icon to view the failed records for that stage. The table shows each failed record and the reason it failed.
    Column Description
    Connector Name Name of the connector
    Run SKey Execution ID
    Status SUCCESS / FAILED
    No of EDD Number of EDDs
    No of Error EDD Number of EDDs with errors
    Total Number of Records Total records processed
    Number of Records Loaded Successfully loaded records
    No of Errors Failed records
  8. You can download the error records to a CSV file by clicking the Export.
    Export Detail Information
    File format CSV file
    File name Error Type_EDD/ Connector Name_Timestamp format
    Maximum rows 1,000 records per export

    Note:

    If you have more than 1,000 error records, only the first 1,000 will be exported. Contact your administrator for more details.

    Searching Within a Pipeline

    If a pipeline has many connectors, you can use the search box on the page to filter the connector list. Type part of the connector name to narrow down the results.

    Example: A pipeline may have 184 connectors. Typing part of the connector name in the search box will show only matching connectors.