Processing Staged Data
Certain types of data that you import into Enterprise Learning Management through EIPs goes into intermediary staging tables before it is loaded into the system. A control value assigned to each row of incoming data identifies whether the data is from a full sync or an incremental sync EIP.
After an EIP imports data into the staging tables, you run a validation process to ensure that there are no data errors. Valid data can then be quickly loaded without the risk of errors. You can delete data in the staging tables as often as needed. The system uses a truncate command on supported platforms, and uses a delete command otherwise.
Staging tables are used for data that undergoes frequent changes or for which there is a high volume in Enterprise Learning Management. Steps for running the validation and load processes depend on whether you are importing data from a full sync EIP message or an incremental sync EIP message:
Full Sync Processing
The typical sequence for processing data from a full sync EIP is as follows:
Run the delete process to delete any existing full sync data from the staging tables.
Run the full sync EIP data publish message from the external application, such as PeopleSoft HR.
Run the full sync EIP data subscribe message from Enterprise Learning Management to import the data into the staging tables.
Ensure the full sync message has completed processing in PeopleSoft HCM and Enterprise Learning Management, and that no errors exist. If errors exist, resolve them before you continue.
Use the Process FullSync Data component to run the validation process.
View the log file for errors and ensure the data is corrected.
If you are satisfied, continue to the next step. Otherwise, correct the errors and return to step one.
Note: Make the corrections in the system of record. For example, if you are importing person data and information is missing, you may need to contact the HR administrator.
Run the upload process to load the validated data into the application tables.
Run the delete process to remove the full sync data from the staging tables.
Note: Rerunning a full sync EIP that uses a staging table does not delete data that resides only in Enterprise Learning Management, such as learner IDs, job codes, and learner department assignments.
Incremental Sync Processing
When incremental sync EIP messages import data into the staging tables in Enterprise Learning Management, a sequence log table captures the messages in sequence. When you initiate the validation process, data is validated and loaded in the order in which it was received.
When you process asynchronous data, the system automatically validates all incremental sync data in the staging table and loads the valid data. After the process is complete, you can check the message log files for errors. The system provides an option to automatically delete incremental sync data in the staging tables after the upload.
To ensure consistency between the data imported into Enterprise Learning Management and data in the source database, schedule the validation and load process for asynchronous data to run at a regular interval, such as every hour or every day. A minimum frequency of daily is recommended.
Pages Used to Process Staged Data
Page Name |
Definition Name |
Navigation |
Usage |
---|---|---|---|
Process FullSync Data |
LM_RNCTL_FULL_LOAD |
|
Run the Process FullSync Data (LM_LD_STGDAT ) Application Engine process to validate full sync data in the staging table, load validated data into Enterprise Learning Management, and delete data from the staging tables. |
Process Asynchronous Data |
LM_RNCTL_ASYNC |
|
Run the Process Asynchronous Data (LM_LD_ASYNC) Application Engine process to validate asynchronous data in the staging tables, load validated data into Enterprise Learning Management, and delete data from the staging tables. |
Prerequisite for Processing Staged Data
Run the full sync EIP for the data you want to import before running any processes for staged data.
Process FullSync Data Page
Use the Process FullSync Data page (LM_RNCTL_FULL_LOAD) to run the Process FullSync Data (LM_LD_STGDAT ) Application Engine process to validate full sync data in the staging table, load validated data into Enterprise Learning Management, and delete data from the staging tables.
Image: Process FullSync Data page
This example illustrates the fields and controls on the Process FullSync Data page.
![Process FullSync Data page](img/i1be2c247n-7f62.png)
You can validate data, load validated data into Enterprise Learning Management, and delete data in the staging tables in separate processes or at the same time. The process runs in this order, for all selected check boxes: validate, upload, and delete.
Process Asynchronous Data Page
Use the Process Asynchronous Data page (LM_RNCTL_ASYNC) to run the Process Asynchronous Data (LM_LD_ASYNC) Application Engine process to validate asynchronous data in the staging tables, load validated data into Enterprise Learning Management, and delete data from the staging tables.
Image: Process Asynchronous Data page
This example illustrates the fields and controls on the Process Asynchronous Data page.
![Process Asynchronous Data page](img/i1be2c247n-7f7b.png)
When you process asynchronous data, the system automatically validates all data in the staging table and loads the validated data. To have the system delete the incremental sync data in the staging table after it loads the data, select the check box labeled Automatically delete completed process.