Enable data lineage tracing in a custom program

Custom program code must, if possible, enable data lineage tracing so that the system can:

• For each data point, display information about contributing data points in source models and resulting data points in target models, if any.

• Display each discrepancy on contributing data points on the preferred path in source models and resulting data points in target models.

To enable data lineage tracing in a custom program:

1. Map the CDR$SKEY surrogate key column in the source table to the corresponding auxiliary column in the target table. (The system creates the auxiliary columns in target tables. You do not need to do that.) See How the system tracks data lineage for more information.
2. The program must populate the target table's auxiliary column for each source table with the value of the CDR$SKEY column for each source record, as part of the INSERT INTO statement.

See Sample programs that populate auxiliary columns with the source surrogate key for examples of the SAS and PL/SQL code required to do this.

Note:

• Data lineage tracing cannot work when it is not possible to trace all contributing data points, as in aggregations. The system displays discrepancies only in the model in which they were created.

• If a custom program performs a pivot or unpivot, the source columns' surrogate key values must be modified before writing to the auxiliary column in the target table or data lineage tracing will not work. However, you can create an intermediate model containing a table and create a pivot or unpivot transformation to it in the user interface, then create a custom program from the intermediate table to the target table.