Metadata Model

This page describes the metadata model used by the implementation of Oracle Utilities Analytics Warehouse (OUAW).

On this page:

Overview

This metadata model comprises a set of tables and views that help achieve the following objectives:

Maintain the configuration details of each source system.
Maintain the configuration of each instance of the source system when multiple instances of the same source system have been implemented.
Maintain the configuration of each of the target entities, by reusing interfaces and packages for multiple instances.
Manage dependencies.
Manage multiple Data Source Indicators (DSIs) allowing external references at the instance level and also at granularity of individual rows of the source data.

Note: You should not change any of the metadata entries that are delivered by default in these metadata tables. Wherever your inputs are needed or allowed, the Oracle Utilities Analytics Administration Tool supplied with the OUAW product should be used to provide inputs. Other than this method, any other means to change the metadata is strictly prohibited.

Metadata Tables

The tables used in the metadata model are listed as below:

Products: This table is used to store a list of the products supported by the implementations, to identify which products are supported, and to determine the Oracle Data Integrator scenario to be used to import source metadata and configurations.

Instances: You may have multiple instances of the same product that you want to integrate with OUAW. The instance object represents each instance of the same product that can be used as a source. The objective is to enable the development of a single interface that can be used across multiple instances.

The instances differ by the source database connections and possibly by differing configurations. The Data Source Indicator (DSI) is unique across instances and products, as it provides the ability to identify the source of the data in the data warehouse.

Global Configuration: This table is used to store the configuration name-value pairs that are applicable. These configurations are used to control the execution modes for interfaces.

Entities: This table holds the configurations for the target entities in the data warehouse. The target entities can be dimensions, facts, or materialized views. The same target entity may be loaded from multiple source instances. The table stores the configuration that is common across multiple instances.

Job Configuration: This table is used to provide configuration for the package (the executable logic) for populating the target entities for each instance. This table stores the current state information and an override package for use when the logic for one instance differs from the logic used for other instances. The override package name can be used to provide a custom scenario in case you want to deviate from the logic provided.

Schedule: This table is used to provide configuration for scheduling jobs at regular intervals. This table also stores the start and end time of the job within a day to enable the job to run during a specific time period.

Objects: This table is used to store all the objects and their common configurations. The objects can be the source tables that need to be replicated, the database objects that need to be generated, the source Maintenance Objects (MOs), or the Business Objects (BOs). Any object that is created during installation or when a source is attached is listed in this table along with configuration associated with the object.

Mapped Objects: This table contains a list of all the source objects that are mapped to a target object. It provides the primary data entry point used to map views to be generated for the target entities, which are identified by an object type flag.

The table below lists the object type flags and their usage.

Object Type Flag	Description
MO	Maintenance Object: The value in the source object field is identified as OUAF Maintenance object based on this flag.
TBL	Table: The value in the source object field is identified as an NMS source table based on this flag.
PRVW	Primary View: The value in the source object field is identified as primary view to load the data into the target object based on this flag.
MVDP	Materialized View Dependency: Based on this flag, it is assumed that the value in the source object column is a dimension or fact entity and the value in the target object is a materialized view. This also indicates that the materialized view load should depend on the load of the entity specified in the source object column.
DMDP	Dimension Dependency: Based on this flag, it is assumed that the value in the source object column is a dimension entity and the value in the target object is a fact entity. This also indicates that the fact load should depend on the dimension load.

Source Tables: This table contains configurations that control which tables are replicated and the replication mode.

Job Executions: This table is used for tracking the execution of the ELT processes. An entry is created for each execution. Some attributes are populated from the SNP_SESSION table which is used by Oracle Data Integrator to track sessions.

Dependencies: This table is used to map a job to the jobs it is dependent on. The dependent objects are only executed up to the minimum synchronization timestamp of all of the dependencies.

DSI Mapping: A source system may be integrated with other source systems. When pulling information from multiple source instances, it is possible that the data in one instance refers to the master data from another instance. This table allows such cross references to be configurable. By default, the configuration is at an instance level that is a product and instance number level. However, more fine-grained control is possible by specifying the entity for which the configuration should be applied.

Server Configuration: This table is used to generate the Oracle GoldenGate scripts for ongoing synchronization. It stores database, GoldenGate, and JAgent specific information of the target and source system.

GoldenGate Checkpoint: This table is used by Oracle GoldenGate replication process to track its processing activities. This table is used by the scheduler process to ensure that the warehouse loading tasks only process data that has been synchronized. The structure is controlled by Oracle GoldenGate.

Extract Parameters: This table stores configurations that control the extraction of data. Each source system may have different codes used for the same purpose. It allows you to configure the codes so that the extraction routine can use it appropriately.

Storage Configuration: This table is used to store configuration for allocation of table spaces while creating objects within the knowledge models. It has the default configurations at the pre-loaded Global context level.

Metadata Views

The views used in the metadata model are listed as below:

Context Configuration: The view associates the product instance with the logical and physical schemas defined in Oracle Data Integrator.

Replication Configuration: The replication configuration view is a summary of the settings that are used to control the tables to be replicated.

Job Configuration: The job configuration view shows all the jobs based on the configuration in the metadata table. The schedule date and the slice start and end date are calculated along with the maximum retry interval, retry interval, and staging retention period.

Storage Configuration: The storage configuration view shows the tablespaces configured for different object types for each logical schema. The configuration can be common or can be granular at logical schema level. This view breaks down the configurations at the lowest granularity and provides easy access to all attributes of storage configuration handling the complexities of identifying context and logical schema specific configurations that override the global configurations. All the attributes of storage configuration handle the complexities of identifying context and logical schema specific configurations that override the global configurations.

Dependencies: The Oracle GoldenGate processes can be tracked via the Oracle GoldenGate Checkpoint table. A source instance can have multiple Oracle GoldenGate processes and each process consists of multiple tables. A target entity can be loaded only up to the maximum synchronization timestamp of all the Oracle GoldenGate processes associated with the context in which it is executing. A fact load is dependent on the load of Type 2 dimensions associated with the fact. The fact associated with a Type 2 dimension cannot be loaded beyond the time that the dimension has been synchronized. A materialized view becomes stale if any of the underlying tables is modified. Hence, the materialized view depends on the execution of any of its underlying tables. The view identifies the base object and the timestamp associated with its dependencies so that appropriate actions can be taken to avoid any data loss.

Schedule: This is a consolidated list of the start time of the job for a given schedule. Based on the start time at regular intervals the jobs are picked for execution.

Jobs: This is a consolidated way of viewing all jobs that are being executed or that have completed. The view shows the slicing units, duration of execution, and volumes being processed. This can be used for health checks and for monitoring the load processes.

Ready for Load: This view governs the selection of execution jobs and control its pattern. Note the applicable rules:

The entity should not be disabled.
The configured scenario should exist.
All dependencies should have run.
The number of running or error instances of the job should be less than the maximum parallel executions allowed.
If the instance has been configured to run once, then it should not execute again once it has been successfully executed.
If a job fails, it should be retried until the maximum retries per day is reached. The interval between successive retries should be based on the configuration.
The Oracle GoldenGate models comprising of the source tables used in the entity should have been synchronized. In case the synchronization timestamps vary across multiple models, then the least common synchronization timestamp is used.
The snapshot entities are executed during or after the snapshot end period.
The schedule time can be used to stagger loads and distribute processing. A job is not executed until the current time crosses the scheduled time.

Wait Reasons: This view can be used to diagnose the reasons causing that a job is not being executed. This view applies the rules for scheduling and reverses the logic to obtain those causes. The following issues can be identified using this view.

An entity is deactivated.
The Oracle GoldenGate replication is not configured for model {model name}.
The configured scenario does not exist.
This job is scheduled to run at {timestamp}.
This one time load interface has already executed successfully.
Reached max retries allowed per day.
The approximately {X} minutes until next retry.
The maximum number of parallel materialized views execution reached.
Waiting for a change to occur in dependent entity.
Waiting on a dependent entity to be loaded.
Waiting for at least one minute of incremental data to arrive.
The cut-off time has been enabled. All the entities are loaded only up to {timestamp}.
The temporary table is not reversed in the staging model.
The staging table is not reversed in the staging model.
The flex fields are not set for table.