Checking for Null Data

A data set may sometimes contain empty records or null values. Verify whether your data set contains such null values by including a validation in your transform, and prompt error alerts if a specific column has null values above a specified percentage threshold.

To perform a null data check:

1. On the main authoring page, identify the transform column that you want check for null values.
2. In the Column cell, click the More Actions icon.
   A drop-down list appears.
3. Select Null Check.
   The Null Data Check dialog box appears.
4. In the Max Null Percent field, enter a maximum percentage value for null values in the selected column. If the percentage of null values in this column is higher than this value, then an error alert appears in the job log.
   Even if a transform produces an error alert for a null check threshold that’s been exceeded, the system treats it only as a warning. The processing of the data file does not fail.
5. Click Apply.

   Note:

   A new action in the Transform Script pane is added for every null data threshold level that you’re setting on your data columns. In this figure, threshold percentages are set for three data columns named gender, first_name, and name_title.
   
   Description of the illustration nullcheck-transformscript.png
   
6. Publish your transform interactively or by running a policy. For more information, see Publishing Transforms or Understanding Policies and Scheduling.
7. To see data validation errors from your publish job:
   • If you published your transform interactively, then see the job log on the Publish page. For more information, see Understanding a Publishing Log.
   • If you published your transform with a policy, then see the Job Details page for your publish job. For more information, see Viewing Details for a Specific Job.