Data warehouse job
The Data warehouse job is a job that runs in Oracle Unity to copy ingested data from staging to the data warehouse and complete data validation.
When importing data with an ingest job or with the the Streaming API, you must run the Data warehouse job to validate and process the imported data. If you import data with the Near real-time API, you don't need to run the Data warehouse job.
The following values are calculated when the Data warehouse job runs:
-
Data density for attributes belonging to data objects that are not Master entities. For attributes belonging to master entities, data density is calculated by the Identity resolution job.
-
Lookup values for attributes.
After the Data warehouse job runs, you can unify the validated data and create master entities by running the Identity resolution job. Learn more about Master entities.
Sequence of the data warehouse job
You can customize the data warehouse job by using the Oracle Unity API to create scripts. Learn more about DW Mapping from the Oracle Unity Developer Help Center.
The flow of the data warehouse job is the following:
-
Initialization of the data warehouse job.
-
Validation and mapping (occurs in parallel).
Validated entities: Entities that go through address validation.
-
Data warehouse mapping (occurs in parallel for each entity).
-
Optional data warehouse pre-scripts.
-
Default standard data warehouse scripts.
-
Non-Validated entities: Entities that go through data warehouse mapping directly without going through address validation.
- Optional data warehouse pre-scripts.
-
Default standard data warehouse scripts.
-
Refresh values for Data density based on the refresh schedule (see the Additional information for calculating data density section).
-
Refresh values for Attribute lookups and Segmentation attribute lookups.