Persist Widget with SCD Modes
The Persist widget is used to write processed data into database tables for reuse in other pipelines. It must be connected to a Data Forge widget to function.
The Persist Widget offers the following new mode:
- SCD (Slowly Changing Dimension) is used to map data from source datasets to the target table with both current and historical data stored in the target table.
This enhancement provides greater control over data ingestion strategies by allowing users to choose between incremental updates and complete data reloads, improving efficiency, accuracy, and operational flexibility in data processing workflows.
Steps to enable and configure
You don't need to do anything to enable this feature.