1.3.7 Profilers

Profilers are used to understand data; that is, to discover the technical characteristics of the data, and to help you find issues in your data, and identify data that is not fit for its business purposes.

Profilers are different from audit processors in that they do not use business rules that are specific to your data. Rather, they are used for the discovery of data characteristics that can be used to form business rules that may be used when auditing the data.

Profilers therefore do not 'check' data, and do not have Output Filters (such as Valid or Invalid Records). They are intended purely to analyze data, and not to siphon off records with different characteristics.

Data Profiling is done by analyzing data from scratch; that is, without any preconceptions of what the business rules for the data are or should be.

Note:

The use of profilers is not recommended in production environments. Also, if your dataset has over 500,000 rows and 50 columns, profilers may be used after sampling the dataset.