Run concurrent Data Quality Manager server tasks for data matching.
Use different, mutually-exclusive object WHERE clauses to separate the data matching into smaller batches (not more than 50,000 to 75,000 records at a time). For example, you might run separate tasks for each first letter (or letters) of a contact record's Last Name or Name fields as in the following example:
run task for component DQMgr with BObjName="Contact", BCName="Contact", OpType="Key Generate", ObjWhereClause="[Last Name] like 'A*'"
The object WHERE clauses to process all records are as follows:
ObjWhereClause="[Last Name] < 'A'"
ObjWhereClause="[Last Name] like 'A*'"
ObjWhereClause="[Last Name] like 'a*'"
ObjWhereClause="[Last Name] like 'B*'"
ObjWhereClause="[Last Name] like 'b*'"
...
ObjWhereClause="[Last Name] like 'Y*'"
ObjWhereClause="[Last Name] like 'y*'"
ObjWhereClause="[Last Name] like 'Z*'"
ObjWhereClause="[Last Name] > 'z'"
NOTE: When you run a batch task with an object WHERE clause, only records specified by the object WHERE clause are read into memory. However, depending on the number of records and customization, a single task can still consume a large amount of memory. To limit the total amount of memory used by the Data Quality Object Manager for concurrent tasks, you can reduce the value of the MaxTasks server component parameter setting so that fewer concurrent tasks run. For more information about setting the MaxTasks parameter, see Applications Administration Guide.
Set the DQSetting parameter to Delete to improve the performance of batch data matching and key generation processing.
By default, when you run data matching using SSA-NAME3, existing duplicate records are not removed from the S_DEDUP_RESULT table. Likewise when you run key generation batch jobs, existing keys are not removed.
To remove all keys in the key tables or all duplicate records in the S_DEDUP_RESULT table, run the appropriate batch job with DQSetting = Delete.
NOTE: The Delete setting is an optional Data Quality Setting parameter, whereas BCName, BObjName, and OpType are required.