9.5.1 Pipeline

A pipeline is an embedded data processing engine that runs inside the application to filter, transform, and migrate data on-the-fly. Pipelines are a set of data processing elements called widgets connected in series, where the output of one widget is the input to the next element. Use the Pipeline canvas to create the model and execute the pipeline using widgets.

To create a paragraph using pipeline, follow these steps:

1. Navigate to the Pipeline Designer page. Pipeline Canvas is displayed.
2. Click on the Connector to display the widgets.
3. Select Customer Screening from the list.
4. Select a widget and add the widget to the pipeline canvas. For information on widgets, see the CS Pipeline Widgets table.
5. From the pipeline canvas double click on the widget to open the widget details screen on the right side.
6. In the widget screen under the Custom Parameters tab, click Copy to open the Clone Objects Window.
7. Select the source model ID from the Clone Objects Window and select the version from which you want to clone the widget.

   Note:

   Select the model ID as PROD. For the Batch EDQ widget, the model ID is PRODi (Example: PROD0, PROD1) based on the input in the PROD_edq_project_name Key.
8. Click Copy. The CS Widget clone process begins. Once the cloning is completed, the current model ID and version will automatically be populated in the widget screen.

   Note:

   Cloning of Watchlist-Management and Customer-Screening project should not be configured in the same pipeline.
9. Click Save to save the widget.
10. Click Add the next widget and repeat from step 2 to step 8.

    Figure 9-22 CS Simulation Pipeline

    
    

    

    
    

    Figure 9-23 CS Watchlist Screening Pipeline

    
    

    

    
    

    Figure 9-24 Customer Screening Staged Data Simulation Pipeline

    
    

    

    
    

    Table 9-7 CS Pipeline Widgets

    Widget	Description
    Customer Screening Batch EDQ	This widget will allow you to access the EDQ Project for modifying the matching configurations.
    Data Load Event Generation Batch	This batch will call the EDQ and perform the screening between Customer and Watchlist data to generate the matches in simulation schema.
    L1Alert Generation Batch	This batch will generate the alerts for investigation.
    CS Simulation Statistics	This report shows comparison between Production Results and Simulation Results for the same input
    CS Simulation Results	This report shows the list of all L1 alerts generated along with the events for a Simulation Run.
    CS Production Results	This report shows the list of all L1 alerts generated along with the events in Production Environment for the same input.
    Watchlist Management Batch EDQ	This widget will allow you to access the EDQ Project for modifying the watchlist configurations.
    Watchlist Management Batch	This batch will call the EDQ and load the watchlist data.
    Business Data Load Batch	This batch will move the data from Staging tables to Business tables.
    Customer Screening Batch EDQ	This widget will allow you to access the EDQ Project for modifying the matching configurations.
    Data Load Event Generation Batch	This batch will call the EDQ and perform the screening between Customer and Watchlist data to generate the matches in simulation schema.
    L1Alert Generation Batch	This batch will generate the alerts for investigation.
    CS Simulation Statistics	This report shows the Simulation Result Statistics.
    CS Simulation Results	This report will show the list of all L1 alerts generated along with the events for a Simulation Run.

To execute the pipeline follow the subsequent steps:

1. Click execute . Execute Pipeline window is displayed.

   Figure 9-25 Execute Pipeline

   
   

   

   
   
2. Click Open from saved Parameter set? to import the template.
3. If your not importing the template enter Enter the execution Key and Value manually.

   Note:

   Select the flow, which you want to execute Scoring, Training, and Experimentation. It displays all the keys defined for all the paragraphs in the notebook with a placeholder for providing the values.
4. You can add new parameters using Add .

   Note:

   If the parameter is not defined in the notebook, it will not be used for the execution. In case of multi select, if there are common parameters among the chosen scenarios, it will take the value based on the order of selection. that is first chosen scenario parameter will be taken.
5. But if open from saved scenarios again (not on single go), then already added will get replaced by the newly added (same as what existed)
6. Execution is performed based on selected link types. It filters out all the not required/unused parameters. And, all the unused parameters for the current execution are displayed with a warning . To view the only required parameters, click Show only required link.
7. Click Reset to reset the entered data.
8. Click Delete to delete the entered Key and Value.
9. Click Execute to initiate the execution. The widgets in pipelines are executed sequentially and you can see icon on each widget for a successful execution. For individual widget execution details click the widget and click View Details.