5.1 DQ Checks
Data Quality Framework consists of a scalable rule-based engine that uses a single-pass integration process to standardize, match, and duplicate information across global data. This framework within the infrastructure system facilitates you to define rules and execute them to query, validate, and correct the transformed data existing in an Information Domain.
Data Catalog Contents include Data Quality Check Rules and DQ Groups (logical grouping of DQ rules). These Rules are defined at the Business Term and Entity Level, and seeded as a part of the Data Catalog Content.
For instructions on how to create/edit/delete DQ rule, refer to Oracle® Data Foundation Cloud Service Data Platform.
The following is a list of pre-configured Data Quality rules included in the offering:
In the banking domain, data quality is essential for ensuring the integrity, accuracy, and reliability of financial information that supports decision-making, compliance, and operational efficiency. To maintain high standards of data quality, various validation rules are applied to the data. These rules help identify and prevent errors, inconsistencies, and incomplete data entries that could lead to incorrect business processes or regulatory violations. Common data quality rules used in banking include Prebuilt Business check, list of values, referential integrity checks, range checks, and more. Below is an overview of these rules, along with examples specific to the banking domain, which highlight their importance in managing data effectively across banking systems.
| Data Quality Rules | Definition | Example | Objective |
|---|---|---|---|
| Prebuilt Business Checks | Prebuilt Business check is a rule based on business specific or unique conditions that don’t fall under standard data quality rule categories. | A customer's Account Balance must not be negative if the account type is 'Savings'. | To apply business-specific logic that isn't captured by standard validation rules. |
| List of Values or Code Check | This rule ensures that a field contains a value from a predefined list of valid values, such as bank codes, account types, or country codes. | The Account Type must be one of the following: 'Checking', 'Savings', 'Credit". | To ensure only valid and standardized data entries are used in the system. |
| Referential Integrity Check | Ensures that relationships between different tables or data entities are correct, typically by checking foreign keys against primary keys. | The Loan ID in the Payment History table must exist in the Loan Account table. | To prevent orphaned records and ensure that linked data entities are consistent across the system. |
| Column Reference or Specific Value Check | Ensures that the value in one column is consistent with values in another column or a related data set. | If the Account Status is 'Closed', then the AccountBalance must be zero. | To ensure that business logic between columns is followed correctly. |
|
Generic Check |
A general validation check applied across various data sets, such as length checks or format checks, which doesn’t belong to a specific category. | The IBAN number should be 22 characters long and contain only alphanumeric characters. |
To apply basic checks that are common across different data sets. |
| NULL Value Check | Verifies that required fields are not missing, meaning they are not NULL. | The Customer Name field must not be NULL for all active customer records. |
To ensure that essential customer or transaction information is always captured. |
| Range Check | Verifies that data values fall within a valid range, which is commonly used for numeric fields or dates in banking systems. | The Credit Score must be between 300 and 850. | To ensure that data, such as credit scores or loan amounts, falls within realistic and acceptable ranges. |
| BLANK Value Check | Ensures that fields do not contain blank or empty string values, which may represent missing or incomplete data. | The Loan Amount field must not be blank or contain only spaces in a loan application. |
To prevent incomplete data from being entered into critical banking systems, ensuring full and valid records. |
These banking-specific examples help ensure that critical financial data is correct, consistent, and adheres to industry standards for data integrity and compliance.
In DFCS the following number of rules are bundled /autogenerated in each category.
Table 5-1 Number of Tools that are bundled
| Check Type | No of Rules |
|---|---|
| Prebuilt Business Checks | 801 |
| List of Values or Code Check | 3266 |
| Referential Integrity Check | 7610 |
| Column Reference or Specific Value Check | 338 |
| Generic Check | 262 |
| NULL Value Check | 279 |
| Range Check | 11 |
| BLANK Value Check | 6 |