3.1 Use and Execute the Source Data Quality Check Process

The Run Pipeline (Process) is used to perform Data Quality Checks on source records for various data elements. Follow the steps below to access, execute, and verify the Source Data Quality Check Process in Process Orchestration.

Steps to Use and Execute the Source Data Quality Check Process

1. On the home page, navigate to Data Operations > Process Orchestration.
   The Process Modeller page will be displayed.
2. Search for Source Data Quality Check Data Foundation Process on the Process Modeller page.
   The Process Flow Page will be displaying the process flow created on a drawing canvas. This canvas uses Transition, Activity, and Widgets components (from the floating toolbar).

   Note:

   This workflow consists of:

   • RUN DQ RULE Widgets which represent Data Quality Groups) are arranged in parallel.).
   • A Data Service Widget called the Data Quality Reporting Engine is added at the end of the process for reporting purposes.
3. View Widget Details (Optional)
   • To see details of a widget, double-click on it.
   • You can view details related to Activity, Transition, and Notification.
   • On the drawing canvas, you can check the Definition, Data Fields, and Application Rule details.
4. You can start execution from the Process Flow Page or the Process Modeller Page.
5. On the Process Modeller Page, locate the Source Data Quality Check Data Foundation Process.
6. Click the Menu button, then select Execute Run.

   The Execution Page will be displayed.

7. On the Execution Page, choose With Parameters from the Execution Type List.
8. Select the required As of Date to process Data Quality Checks.
9. Click Apply to start the Run Pipeline execution.

   Note:

   The Run Pipeline is triggered using the selected Extraction Date. For additional details, refer to the Process Orchestration framework documentation.
10. Steps to Verify the Run Execution
    1. Open the Process Monitor Page by clicking Process Monitor on the Process Modeller Page.
    2. The Process Monitor Page lists all Run Instances for the Source Data Quality Check Data Foundation Process.
    3. Use Process ID or Process Name to search for the executed process.
    4. Select the required Process Instance to view the Run Execution Status.

    The Process Flow Page will display the Run Execution Status at each node of the workflow.

11. On the Process Monitor Page, click the required Process Instance to review execution logs.
    The Process Flow Page will display the Run Execution Status for each node.
12. To check the execution details of a specific Node, double-click on it.
13. Click Execution Logs to open the Log Viewer Page
    The Log Viewer Page displays all logs related to the Process Instance.
14. Click Show More to expand log entries.
15. Click outside the Log Viewer to close it.
    The Source Data Quality Check Process has been successfully executed and verified.