This chapter provides information about Data Quality Rules Execution in the Oracle Insurance Data Foundation application and step-by-step instructions to use this section.
Topics:
· Data Quality Rules For Staging Tables
· Data Quality Groups for Staging Tables
Data Quality Framework consists of a scalable rule-based engine which uses a single-pass integration process to standardize, match, and duplicate information across global data. Data Quality Framework within the Infrastructure system facilitates you to define rules and execute them to query, validate, and correct the transformed data existing in an Information Domain.
You can access Data Quality Framework by expanding the Data Integrator Framework within the Unified Metadata Manager section in tree structure of LHS menu.
Data Quality Framework consists of the following sections:
NOTE:
For the latest information about Data Quality Framework, see the Data Quality Framework section in the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.
Data Quality Rules allows you to create a DQ (Data Quality) definition and define specific validation checks based on Range, Data Length, Column Reference/Specific Value, List of Value/Code, Null Value, Blank Value, Referential Integrity, Duplicity, and Custom Check/Business. You can also correct data for range, column reference, list of values, null value, and blank value parameters. The defined Data Quality Rule checks can be logically grouped and executed together.
For the roles mapped to the Data Quality Rules, see the Data Quality Rules section in the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.
To navigate to the Data Quality Rules page, from OFSAA Home, select Oracle Insurance Data Foundation, select Data Management Framework, select Data Quality Framework, and then select Data Quality Rules. The Data Quality Rules page is displayed.
Figure 168: Data Quality Rules page
A defined rule is displayed in Saved status until it is Approved or Rejected by the approver. The approved rules can be grouped further for execution and the rejected rules are sent back to the user with the Approver comments.
You can Add, View, Modify, Copy, Approve or Reject, Resave, or Delete the Data Quality Rules within the Data Quality Rules page. You can make any version of a Data Quality Rule as latest. You can also search for a Data Quality Rule based on Name, Folder, Check Type, On Source, Source, Table, or Record Status (Active, Inactive, and All).
To perform different actions related to the Data Quality Rules, see the following sub-sections in the Data Quality Rules section of the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0:
· Creating a Data Quality Rule section to create a Data Quality Rule
· Viewing Data Quality Rule section to view an existing Data Quality Rule
· Modifying Data Quality Rule section to edit an existing Data Quality Rule
· Copying Data Quality Rule section to copy an existing Data Quality Rule details
· Deleting Data Quality Rule section to delete a Data Quality Rule
· Approving/ Rejecting Data Quality Rule to approve a pre-defined DQ Rule or reject a DQ Rule
· Versioning and Make Latest Feature section to make any older version as latest
· Resaving Data Quality Rule section to save multiple Data Quality Rules simultaneously
To execute any Data Quality Rule, a Data Quality Group must be created and the rules for execution has to be mapped with this Data Quality Group. For more information, see the Executing Data Quality Group section in the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.
For the latest information about Data Quality Rules, see the Data Quality Rules section in the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.
Data Quality Groups facilitates you to logically group the defined DQ definitions and schedule for execution. DQ definitions can be executed either through Data Quality Groups page of Data Management Tools framework or in Batch Execution page of Operations module.
When a Data Quality Group is executed for processing, the details of the execution are captured in a log file.
For the roles mapped to the Data Quality Groups, see the Data Quality Groups section in the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.
To navigate to the Data Quality Groups page, from OFSAA Home, select Oracle Insurance Data Foundation, select Data Management Framework, select Data Quality Framework, and then select Data Quality Groups.
Figure 169: Data Quality Groups Page
The Data Quality Groups Page displays the list of pre-defined Data Quality Groups with the other details such as Name, Folder, Creation Date, Created By, Last Modification Date, Last Modified By, Last Run Date, and Last Run Status.
You can create and execute DQ Group definitions and View, Modify, Copy, Refresh, or Delete DQ Group definitions within the Data Quality Groups Page.
To perform different actions related to the Data Quality Groups, see the following sub-sections in the Data Quality Groups section of the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0:
· Creating Data Quality Group section to create a Data Quality Group
· Viewing Data Quality Group section to view an existing Data Quality Group
· Modifying Data Quality Group section to edit an existing Data Quality Group
· Copying Data Quality Group section to copy an existing Data Quality Group details
· Deleting Data Quality Group section to delete a Data Quality Group
· Executing Data Quality Group section to execute a Data Quality Group definition
· Viewing Data Quality Group Summary Log section to view the execution log details of Data Quality Rules
· Viewing Data Quality Report section to view the execution summary report of Data Quality Rules
For the latest information about Data Quality Groups, see the Data Quality Groups section in the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.
Data Quality (DQ) Rules are framed and created based on Staging Tables. Each rule is based on specified staging table column Specific Check or table Generic Check. The rules created for each of the tables are detailed in the DQ_Check_Rules spreadsheet.
In Specific Check, a particular column is checked based on rule's predefined checks, whereas in Generic Check any columns are not specified. Generic Check is useful if you have a check which is not Specific or you use IF-ELSE conditions or CASE statements.
Figure 170: Specified Data Quality Rules for the Staging tables
To execute any DQ Rule, a group must be created and the rules for execution has to be mapped with this group. For each staging table, a group has to be created and rules for these particular table have to be mapped under this group.
Figure 171: Specified Data Quality Groups for the Staging tables
For more information regarding Data Quality Group execution, see the Executing Data Quality Group section in the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.
The DQ Group Mapping sheet of the Oracle Insurance Data Foundation Application Pack Runchart document of this release, which displays the total groups and corresponding Rules mapped to that group.
Create a Batch for executing DQ Rules and add a Task to the selected Batch. Add component as RUN DQ RULE and in Dynamic Parameter List, add a DQ Group.
A single Batch can have multiple number of Tasks and each Task is executing the DQ Group. Batch execution facilitates the execution of multiple DQ Groups, simultaneously.
You can also use the Include or Exclude functionality to determine which all groups have to be executed. The following batch needs to be executed in OIDF:
<INFODOM>_DATA_QUALITY_BATCH_OIDF
See the RUN_CHART_SUMMARY sheet of this release Runchart document of this release.
For more information about the Batch Execution, see the Operations section of the Oracle Financial Services Advanced Analytical Applications Infrastructure User Guide Release 8.1.2.0.0.