Bookshelf v7.5.2: Data Loading Issues

Siebel Data Warehouse Installation and Administration Guide > Administering and Troubleshooting the Siebel Data Warehouse > Siebel Analytics for Life Sciences Data Considerations >

Data Loading Issues

Siebel Analytics for Life Sciences updates the Siebel Data Warehouse with a full refresh or using incremental updates. In Informatica, the Full_Load_Siebel_DW batch does a complete refresh and is usually only used for the initial build. Refresh_Siebel_DW incrementally updates the Siebel Data Warehouse.

Running Full_Load_Siebel_DW deletes all existing information stored in the fact and dimension tables. The following types of incremental updates rules are supported on the fact tables.

Activity Product Fact. Add new records (call activity) to the fact table.

Rx Sales (syndicated data) Fact. ETL does not support incremental updates. When running Full_Load_Siebel_DW, all records in the fact and dimension tables are deleted. To maintain a history in the dimension tables (such as multiple alignments), use Refresh_Siebel_DW. If you need to incrementally update the RxSales fact table for incremental syndicated data loading, use one of the following strategies:

For incremental insert. Load the data directly into staging and modify the session not to truncate the fact table. Then use the existing ETL to load it to the fact table.

For incremental update. Create a new mapping that does a lookup against the fact table and run the update.

Objective. Add new records as per the new ePharma Objective. Update existing records with a changed ePharma Objective to the fact table.

Profile Rank. Add new records to the fact table and update existing records in the fact table.

Med Ed. Add new MedEd events to the fact table. A MedEd event is assumed to be locked in the Siebel ePharma MedEd. If the information is changed in an existing MedEd event, the ETL creates a new record in the fact table. If an existing MedEd event is changed, the ETL process does not load the data. The ETL always skips already loaded MedEd events.

Known Issues with the RxSales Fact Table

The following are known issues with creation of aggregate measure columns in the RxSales fact table.

With large volumes of syndicated data, the creation of aggregate measures in the Siebel Data Warehouse can take four times the amount of time needed to load the fact table. This may be unacceptable under circumstances when there are large amounts of syndicated data.

All aggregate measures have been recreated at the metadata level except for MAT. MAT can be entered as a formula for ad-hoc analysis in Siebel ePharma Answers using the function msum(measure, window). For more information, read Siebel Analytics User Guide.

The LS syndicated fact table contains following aggregate measure columns to speed enterprise wide reporting: TTD, TTDLY, TTDPT, MAT, YTD. To update these aggregate columns ETL process requires following steps:

Upgrading aggregate columns

Join the syndicated data to the day dimension to denormalize by time.

Create a second set of time denormalized syndicated data.

Join these two data sets to obtain appropriate measure for TTD, TTDLY, TTDPT, MAT, and YTD aggregation.

Aggregate the measures.

Update the syndicated data aggregate columns.