1. Marketing Installation and Administration Guide
  2. Data Loading Process

Data Loading Process

The Siebel CRM integration with Oracle Unity supports data loading as follows:

  • Bulk loading

  • Incremental loading

Bulk Loading

The following summarizes the bulk loading process for a configured deployment:

  1. In the Data Export Settings view, the administrator clicks the Export or Export All button to send the request to the Search Index Manager Service.

    Where multiple administrators might be working with the data in this view, the administrator can also click Refresh to refresh the displayed data prior to exporting.

  2. The result set from the business component query is split into multiple batches, based on the configuration.

  3. Jobs are submitted to the Search Data Exporter component.

  4. Data is sent to the CSV Export Adapter Service to write into a CSV file.

  5. The OOS File Upload Service is invoked to upload the CSV file into OCI Object Storage.

Incremental Loading

The following summarizes the incremental loading process, where the initial bulk loading has been done:

  1. The Transaction Processor component reads the data from the S_DOCK_TXN_LOG table and creates the DX file.

  2. A recurring job processes the DX file, fetches the data from the respective business component based on the row ID, and sends it to the configured engine.

  3. Data is sent to the CSV Export Adapter Service to write into a CSV file.

  4. The OOS File Upload Service is invoked to upload the CSV file into OCI Object Storage.

Additional Notes

The following notes provide more detailed information about how the bulk load and incremental load processes work. Refer to other topics in this chapter for related information.

  • In the Data Export Administration view, the Active flag is for incremental data load only. If you set the Active flag to N, the incremental load job creates no CSV file. The Active flag has no relevance to the initial (bulk) data load.

  • If you set the Active flag to N for all Data Export engines, including the Search engines, and the incremental load job is running, then the DX file is created continuously and the incremental load job also runs on the scheduled interval. The incremental load job processes the DX file, but because there is no active engine to send the data, it then deletes the DX file.

  • When multiple engines are configured, if one of the engines fails to send the data successfully, then it does not delete the DX file. It continues to send the data to all of the engines until it sends to all of the engines successfully.

  • You might choose to increase the JVM memory size of the Tomcat container that executes the JBS.

  • If you have included an MVG field in your category, then the export writes only the primary record's value in the CSV file and does not write the values of nonprimary records.

  • When the user exports a category and no record is returned, the status is updated to Completed. No records are created in the Export Status applet and no CSV file is created.

  • When a filter specification is configured for a category, all of the data that match this filter specification are exported. In this case, if you change the value of a field that is part of the filter specification but that that does not meet the filter specification, then the modified data is not exported. This data becomes out of sync in the third-party system.

  • Import the OCI root certificate to the internal Tomcat that runs the OOS File Upload Service JBS.