9.10.1 Widgets in Data Pipelines

Depending on the pipeline type, specific widgets are available in the widgets pane of the pipeline.

Table 9-16 Widgets and Descriptions - Data Pipelines

Widget Name Description

Dataset

Dataset Use this widget to add a Dataset. Datasets correspond to the contents of a single database table which can be a staging table, business table, or a table that a data pipeline has created. A data pipeline must always begin with a Dataset widget.

Filter

Filter Use this widget to filter the data in the pipeline to use a subset of the available data records. On applying a filter, all data matching the filter conditions are obtained. This allows you to search and analyze behaviors of interest.

Join

Join Use this widget to combine or group multiple tables using various join operators.

Persist

Persist Use this widget to write data to database tables to be used in other pipelines.

External Service

External Service Use this widget to add an external service. External Services perform actions on the data, such as loading or moving the data or performing a virus scan.