Data Quality Guide for Oracle Customer Hub > Configuring Data Quality with Universal Connector >
Optimizing Data Cleansing Performance
The following are recommendations for achieving good performance with data cleansing when working with large volumes of data:
- Include only new or recently modified records in the batch data cleansing process.
- Cleansing all records in the Siebel database each time a data cleansing is performed can cause performance issues. Include an Object WHERE clause when you submit your batch job, as shown in Table 20. Split the tasks into smaller tasks and run them concurrently.
Table 20. Recommended Data Cleansing Object WHERE Clause Solutions
|
Use This in Your Object WHERE Clause |
Updated records |
[Last Clnse Date] < [Updated] |
New records |
[Last Clnse Date] IS NULL |
Updated and new records |
[Last Clnse Date] < [Updated] OR [Last Clnse Date] IS NULL |
To speed up the data cleansing task for large databases, run batch jobs to cleanse a smaller number of records at a time using an Object WHERE clause. For more information about data cleansing for large batches, see Cleansing Data Using Batch Jobs.
|