10Universal Connector API
Universal Connector API
This chapter describes the application programming interface (API) functions that third-party software vendors must implement in the dynamic link libraries (DLL) or shared libraries that they provide for use with the Universal Connector. It includes the following topics:
This chapter discusses the Universal Connector API. It includes the following topics:
About the Terminology in This Chapter
The following terms are used in this chapter:
Driver record. The record the user just entered in real time or the record for which duplicates have to be found.
Candidate records. The records that potentially match the driver record.
Duplicate records. The subset of candidate records that actually match the driver record after the matching process.
Master record. The record for which data matching was performed.
About the Universal Connector API
The following topics describe the application programming interface (API) functions that third-party software vendors must implement in the dynamic link libraries (DLL) or shared libraries that they provide for use with the Universal Connector. This topic includes the following information:
About Universal Connector API Terminology
The following terms are used in descriptions of the Universal Connector API:
Driver record. The record the user just entered in real time or the record for which duplicates have to be found.
Candidate records. The records that potentially match the driver record.
Duplicate records. The subset of candidate records that actually match the driver record after the matching process.
Master record. The record for which data matching was performed.
Vendor Libraries
Vendors must follow these rules for their DLLs or shared libraries:
The libraries must be thread-safe. A library can support multiple sessions by using different unique session IDs.
The libraries must support UTF-16 (UCS2) as the default Unicode encoding.
If there is a single library for all supported languages, the libraries must be named as follows:
BASE.dll (on Windows)
libBASE.so (on AIX and Oracle Solaris)
libBASE.sl (on HP-UX)
where BASE is a name chosen by the vendor. If a vendor has many solutions for different types of data, then the vendor can use different base names for different libraries.
If there are separate libraries for different languages, the library name must include the appropriate language code. For example, for Japanese (JPN), the libraries must be named as follows:
BASEjpn.dll (on Windows)
libBASEjpn.so (on AIX and Oracle Solaris)
libBASEjpn.sl (on HP-UX)
The Siebel application loads the libraries from the locations described in Levels of Enabling and Disabling Data Cleansing and Data Matching.
The mapping of Siebel application field names to vendor field names is stored as values of the relevant Business Component user properties in the Siebel repository. Storage of these field values is mandatory.
Any other vendor-specific parameter required (for example, port number) for the vendor’s library must be stored outside of Siebel CRM.
Connector Initialization and Termination Functions
This topic describes functions that are called when the vendor library is loaded or when the Siebel Server shuts down:
sdq_init_connector Function
This function is called using the absolute installation path of the SDQConnector directory (./siebsrvr/SDQConnector) when the vendor library is first loaded to facilitate any initialization tasks. It can be used by the vendor to read any configuration files it might choose to use.
| Syntax | int sdq_init_connector (const SSchar * path) |
| Parameters | path: The absolute path of the Siebel Server installation. Vendors can use this path to locate any required parameter file for loading the necessary parameters (like port number and so on). This is a Unicode string because the Siebel Server can be installed for languages other than English. |
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
sdq_shutdown_connector Function
This function is called when the Siebel Server is shutting down to perform any necessary cleanup tasks.
| Syntax | int sdq_shutdown_connector (void) |
| Parameters | This function does not have any parameters. |
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
Session Initialization and Termination Functions
The Siebel Server works in multi-threaded mode to serve multiple users. To allow for user and invocation-specific parameters, there is the concept of a session context where such values can be stored. The session ID is supplied for all data matching or data cleansing functions. Upon completion of data cleansing or data matching the session is closed.
This topic describes the functions that are used for session initialization and termination:
sdq_init_session Function
This function is called when the current session is initialized. This allows the vendor to initialize the parameters of a session or perform any other initialization tasks required.
| Syntax | int sdq_init_session (int * session_id) |
| Parameters | session_id: A unique value provided by the vendor that is used in function calls while the session is active. The value 0 is reserved as an invalid session ID. The Siebel CRM code calls this function with a session ID of 0, so the session ID must be initialized to a nonzero value. |
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
sdq_close_session Function
This function is called when a particular data cleansing or data matching operation is finished and it is required to close the session. Any necessary cleanup tasks are performed.
| Syntax | int sdq_close_session (int * session_id) |
| Parameters | session_id: The session ID obtained by initializing the session. |
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
Parameter Setting Functions
Most third party software vendors provide lists of parameters to customers so that the customers can configure the vendor library’s behavior to suit their business needs.
This topic describes the functions that set parameters at both the global context and at the session context (that is, specific to a session):
sdq_set_global_parameter Function
This function is called to set global parameters. This function call is made after the call to sdq_init_connector.
The vendor must put the configuration file, if using one, in ./siebsrvr/SDQConnectorpath. When the vendor DLL is loaded, it calls the sdq_init_connector API function (if it is exposed by the vendor) with the absolute path to the SDQConnector directory. It is then up to the vendor to read the appropriate configuration file. The configuration file name is dependent on vendor specifications.
An XML character string is used to specify the parameters. This provides an extensible way of providing parameters with each function call.
Using the sdq_set_global_parameter API, any global parameters specific to the vendor can be put as a user property to DeDuplication business service, where the format of the business service user property is as follows:
"Global", "Parameter Name", "Parameter Value"
These global parameters are set to the vendor only after the vendor DLL loads. You can define user properties for the DeDuplication business service as follows:
Property: My Connector 1
Value: MyDQMatch
Property: MyDQMatch Parameter 1
Value: "Global", "zGlobalParam1", "zGlobalParam1Val"
| Syntax | int sdq_set_global_parameter (const SSchar* parameterList) |
| Parameters | parameterList: An XML character string that contains the list of parameters and values specific to this function call. An example of the XML is as follows: <Data>
<Parameter>
<GlobalParam1>GlobalParam1Val</GlobalParam1>
</Parameter>
</Data>
|
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
sdq_set_parameter Function
This function is called, after the call to sdq_init_session, to set parameters that are applicable at the session context.
The vendor must put the configuration file, if using one, in ./siebsrvr/SDQConnectorpath. When the vendor DLL is loaded, it calls the sdq_init_connector API function (if it is exposed by the vendor) with the absolute path to the SDQConnector directory. It is then up to vendor to read the appropriate configuration file. The configuration file name is dependent on vendor specifications.
Using the sdq_set_parameter API, any session parameters specific to the vendor can be put as a user property to the DeDuplication business service, where the format of the business service user property is as follows:
"Session", "Parameter Name", "Parameter Value"
These session parameters are set to the vendor, after each session opens with the vendor. You can define user properties for the DeDuplication business service as follows:
Property: My Connector 1 Value: MyDQMatch
Property: MyDQMatch Parameter 2
Value: "Session", "zSessParam2", "zSessParam2Val"
| Syntax | int sdq_set_parameter (int session_id, const SSchar* parameterList) |
| Parameters |
|
|
|
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
External Key Generation Function
This function allows you to enable third party vendors to generate deduplication keys in Siebel CRM. This topic describes the functions that set parameters for external deduplication key.
sdq_get_dedup_keys Function
This function allows you to invoke a third party vendor to generate the deduplication keys.
| Syntax | int sdq_get_dedup_keys (int session_id, SSchar* parameterList, SSchar* inputRecordSet, SSchar* outputRecordSet) |
| Parameters |
|
Note: The parameterList can be set to NULL if all required parameters are already set at the session level.
|
|
Here is an example of the XML in hierarchical data format (to support multiple child entities deduplication): <Data>
<DriverRecord>
<Account.Id>12-ABCC</Account.Id>
<Name>Hibbing Manufacturing</Name>
<Location>San Jose</Location>
<ListOfCUTAddress>
<CUTAddress>
<City>York1</City>
<Country>USA</Country>
<Zip>78201</Zip>
<State>AL</State>
<Street> 1200 Davisville Road</Street
</CUTAddress>
<CUTAddress>
<City>York2</City>
<Country>USA</Country>
<Zip>78202</Zip>
<State>AL</State>
<Street> 1201 Davisville Road</Street
</CUTAddress>
</ListOfCUTAddress>
</DriverRecord>
</Data>
|
|
|
|
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
Error Message Function
This topic describes the function associated with error messages: sdq_get_error_message.
sdq_get_error_message Function
This function is called if any of the Universal Connector functions return a code other than 0, which indicates an error. This function performs a message lookup and gets the summary and details for the error that just occurred for display to the user or writing to the log.
Syntax |
void sdq_get_error_message (int error_code, SSchar * error_summary, SSchar * error_details) |
| Parameters |
|
|
|
|
|
| Return Value | This function does not have a return value. |
Real-Time Data Matching Functions
This topic describes the different functions that are called for real-time data matching when match candidate acquisition takes place in Siebel CRM and in Oracle Data Quality Matching Server.
sdq_dedup_realtime Function is used when match candidate acquisition takes place in Siebel CRM.
sdq_dedup_realtime_nomemory Function is used when match candidate acquisition takes place in Oracle Data Quality Matching Server.
sdq_dedup_realtime Function
This function is called to perform real-time data matching when match candidate acquisition takes place in Siebel CRM.
This function sends the data for each record as driver records and their candidate records. The function is called only once; multiple calls to the vendor library are not made even when the set of potential candidate records is huge. As all the candidate records are sent at once, all the duplicates for a given record are returned.
| Syntax | int sdq_dedup_realtime (int session_id, SSchar* parameterList, SSchar* inputRecordSet, SSchar* outputRecordSet) |
| Parameters |
|
|
|
|
|
|
|
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
sdq_dedup_realtime_nomemory Function
This function is called to perform real-time data matching when match candidate acquisition takes place in Oracle Data Quality Matching Server.
| Syntax | int sdq_dedup_realtime_nomemory (int session_id, SSchar* parameterList, SSchar* inputRecordSet, SSchar* outputRecordSet) |
| Parameters | session_id: The session ID obtained by initializing the session. |
|
|
|
|
|
|
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
Batch Mode Data Matching Functions
This topic describes the functions that are called for batch mode data matching:
sdq_set_dedup_candidates Function
This function is called to provide the list of candidate records in batch mode. The number of records sent during each invocation of this function is a customer-configurable deployment-time parameter. However, this is not communicated to the vendor at run time.
| Syntax | int sdq_set_dedup_candidates (int session_id, SSchar* parameterList, SSchar* xmlRecordSet) |
| Parameters |
|
|
|
|
Note: The parameterList parameter is set to NULL as all required parameters are already set at the session level.
|
|
|
|
|
|
|
Note: The order of the driver records and candidate records is not significant. If a candidate has already been sent, it is not necessary to send it again even though it is a candidates associated with multiple driver records.
|
|
An example of the XML is as follows: <Data>
<DriverRecord>
<DUNSNumber>123456789</DUNSNumber>
<Name>Siebel</Name>
<RowId>1-X40</RowId>
</DriverRecord>
<DriverRecord>
<DUNSNumber>987654321</DUNSNumber>
<Name>Oracle</Name>
<RowId>1-X50</RowId>
</DriverRecord>
<DriverRecord>
<DUNSNumber>123123123</DUNSNumber>
<Name>IBM</Name>
<RowId>1-X60</RowId>
</DriverRecord>
</Data>
|
|
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
sdq_start_dedup Function
This function is called to start the data matching process in batch mode, and essentially signals that all the records to be used for data matching have been sent to the vendor’s application.
| Syntax | int sdq_start_dedup (int session_id) |
| Parameters | session_id: The session ID obtained by initializing the session. |
| Return Value | This function does not have a return value. |
sdq_get_duplicates Function
This function is called to get the master record with the list of its duplicate records along with their match scores. This is done in batch mode. The number of records received for each call to this function is set in the BATCH_MATCH_MAX_NUM_OF_RECORDS session parameter before the function is called.
| Syntax | int sdq_get_duplicates (int session_id, SSchar* xmlResultSet) |
Parameters |
session_id: The session ID obtained by initializing the session. |
xmlRecordSet: An XML character string that the vendor library populates with a master record and a list of its duplicate records along with their match scores. If the number of duplicates is more than the value of the parameter BATCH_MATCH_MAX_NUM_OF_RECORDS, the results can be split across multiple function calls with each function call including the master record as well. The XML is in the following format: <Data>
<ParentRecord>
<DQ.MasterRecordsRowID>2-24-E</DQ.MasterRecordsRowID>
<DuplicateRecord>
<Account.Id>2-24-E</Account.Id>
<DQ.MatchScore>92</DQ.MatchScore>
</DuplicateRecord>
<DuplicateRecord>
<Account.Id>2-23-F</Account.Id>
<DQ.MatchScore>88</DQ.MatchScore>
</DuplicateRecord>
</ParentRecord>
</Data>
|
|
| Return Value | A return value of 0 (zero) indicates successful execution, while a return value of 1 indicates that there are no duplicate records. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function.
Note: Data quality code only processes the returned XML character string while the return value is 0. Even if there are fewer records to return than the value of the BATCH_MATCH_MAX_NUM_OF_RECORDS parameter, the vendor driver sends a return value of 0 and then return a value of 1 in the next call.
|
Real-Time Data Cleansing Function
This topic describes the functions that are called for real-time data matching: sdq_datacleanse.
sdq_datacleanse Function
This function is called to perform real-time data cleansing. The function is called for only one record at a time.
| Syntax | int sdq_datacleanse (int session_id, SSchar* parameterList, SSchar* inputRecordSet, SSchar* outputRecordSet) |
| Parameters |
Note: This parameter is set to NULL as all required parameters are already set at the session level.
|
|
|
|
|
| Return Value | A return value of 0 indicates successful execution. Any other value is a vendor error code. The error message details from the vendor are obtained by calling the sdq_get_error_message function. |
Batch Mode Data Cleansing Function
This topic describes the functions that are called for batch mode data cleansing: sdq_data_cleanse.
sdq_data_cleanse Function
The same function, is called by data quality code for both real-time and batch data cleansing. For batch data cleansing, the call is made with one record at a time.
Data Matching and Data Cleansing Algorithms
This topic describes the algorithms used in Siebel CRM code for both real-time and batch data cleansing and data matching:
Batch Data Matching Algorithm
The algorithm is as follows:
Load the vendor library.
Call sdq_init_connector.
Call sdq_set_global_parameter.
Call sdq_init_session.
Call sdq_set_parameter (RECORD_TYPE – Account/Contact/List Mgmt Prospective Contact, BATCH_DATAFLOW_NAME, BATCH_MATCH_MAX_NUM_OF_RECORDS)
Query the Siebel database to get the candidate records.
To get the candidate records, a query against the match key is executed. The match key itself is generated when a record is created, or key fields are updated. Universal Connector supports multiple key generation. For more information about match key generation, see Match Key Generation.
Call sdq_set_dedup_candidates. This function is called multiple times to send the list of all the candidate records.
Call sdq_start_dedup to start the data matching process.
Call sdq_getduplicate. This function is called multiple times to get all the master records and their duplicate records and until the function returns -1 indicating that there are no more records.
Call sdq_close_session (int * session_id) while logging out of the current session.
Call sdq_close_connector.
Real-Time Data Matching Algorithm
The algorithm is as follows:
Load the vendor library.
Call sdq_init_connector.
Call sdq_set_global_parameter.
Call sdq_init_session.
Call sdq_set_parameter (RECORD_TYPE – Account/Contact/List Mgmt Prospective Contact).
Call sdq_get_dedup_keys. This fuction calls a third party vendor to generate the dedup keys, then EDQ invokes the function sdq_get_dedup_keys to get the dedup keys for driver record. The keys are then used to query for candidate records to send to then third party vendor in the dedup call which follows.
Call sdq_dedup_realtime.
Call sdq_close_session while logging out of current session.
Call sdq_close_connector.
Batch Data Cleansing Algorithm
The algorithm is as follows:
Load the vendor library.
Call sdq_init_connector.
Call sdq_set_global_parameter.
Call sdq_init_session.
Call sdq_set_parameter (RECORD_TYPE – Account/Business Address/Contact/List Mgmt Prospective Contact, BATCH_DATAFLOW_NAME).
Query the Siebel database to get the set of records to be cleansed.
Call sdq_datacleanse. This function is called for each record in the result set of the query. It sends the driver record as XML and the output from the function has the cleansed driver record.
After cleansing each record, save the record into the Siebel repository.
Call sdq_close_session while logging out of current session.
Call sdq_close_connector.
Real-Time Data Cleansing Algorithm
The algorithm is as follows:
Load the vendor library.
Call sdq_init_connector.
Call sdq_set_global_parameter.
Call sdq_init_session.
Call sdq_set_parameter (RECORD_TYPE – Account/Business Address/Contact/List Mgmt Prospective Contact).
Query the Siebel database to get the Driver Record.
Call sdq_datacleanse. This function sends the driver record as XML and the output from the function will have the cleansed driver record.
Save the record into the Siebel repository.
Call sdq_close_session while logging out of current session.
Call sdq_close_connector.
New Key Generation Algorithm
The algorithm is as follows:
Call sdq_init_session.
Call sdq_set_parameter (RECORD_TYPE) - Account/Contact/Prospect).
Call sdq_get_dedup_keys. This function sends the driver record as XML and the output from this call is the list of dedup keys generated by third party vendor. EDQ then stores the keys in its dedup key table.
Call sql_close_session.