Siebel Data Quality Administration Guide > Optimizing Siebel Data Quality Performance > Optimizing Matching Server Performance >

Data Quality Manager Server Tasks

The following are recommendations for achieving good performance with batch data matching:

• Run concurrent Data Quality Manager server tasks for data matching.

  Use different, mutually-exclusive object WHERE clauses to separate the data matching into smaller batches (not more than 50,000 to 75,000 records at a time). For example, you might run separate tasks for each first letter (or letters) of a contact record's Last Name or Name fields as in the following example:

  run task for component DQMgr with BObjName="Contact", BCName="Contact", OpType="Key Generate", ObjWhereClause="[Last Name] like 'A*'"

  The object WHERE clauses to process all records are as follows:

  ObjWhereClause="[Last Name] < 'A'"

  ObjWhereClause="[Last Name] like 'A*'"

  ObjWhereClause="[Last Name] like 'a*'"

  ObjWhereClause="[Last Name] like 'B*'"

  ObjWhereClause="[Last Name] like 'b*'"

  ...

  ObjWhereClause="[Last Name] like 'Y*'"

  ObjWhereClause="[Last Name] like 'y*'"

  ObjWhereClause="[Last Name] like 'Z*'"

  ObjWhereClause="[Last Name] > 'z'"

  NOTE:  When you run a batch task with an object WHERE clause, only records specified by the object WHERE clause are read into memory. However, depending on the number of records and customization, a single task can still consume a large amount of memory. To limit the total amount of memory used by the Data Quality Object Manager for concurrent tasks, you can reduce the value of the MaxTasks server component parameter setting so that fewer concurrent tasks run. For more information about setting the MaxTasks parameter, see Applications Administration Guide.

• After your initial data matching or key generation, include only new and updated records in your key generation and data matching processes because reprocessing all records is too time consuming.

  You use the DeDup Key Modification Date and DeDup Last Match Date business component fields in your search specifications to exclude records. For example, the following table shows the Object WHERE Clause to run key generation or data matching.

  To Query for...	Key Generation Example	Data matching Example
  Updated records	([DeDup Key Modification Date]<[Updated])	([DeDup Last Match Date]<[Updated])
  New records	([DeDup Key Modification Date] IS NULL)	([DeDup Last Match Date] IS NULL)
  Updated and new records	([DeDup Key Modification Date]<[Updated]) OR ([DeDup Key Modification Date] IS NULL)	(([DeDup Last Match Date]<[Updated]) OR ([DeDup Last Match Date] IS NULL))

• Set the object sort clause based on the fields that are used to generate match keys:
  • For person (contact or prospect), use Last Name, First Name, Middle Name.
  • For company (account), use Name or Name, Location.
• Set the DQSetting parameter to Delete to improve the performance of batch data matching and key generation processing.

  By default, when you run data matching using SSA-NAME3, existing duplicate records are not removed from the S_DEDUP_RESULT table. Likewise when you run key generation batch jobs, existing keys are not removed.

  To remove all keys in the key tables or all duplicate records in the S_DEDUP_RESULT table, run the appropriate batch job with DQSetting = Delete.

  NOTE:  The Delete setting is an optional Data Quality Setting parameter, whereas BCName, BObjName, and OpType are required.

CAUTION:  Do not attempt to use the Delete option if you are not an expert user of SQL as you run the risk of corrupting your database.

For more information about running batch key generation jobs, see Generating or Refreshing Keys Using Batch Jobs.

For more information about running batch data matching jobs, see Customizing Data Quality Server Component Jobs for Batch Mode.