Get Started
Oracle Utilities Analytics Warehouse (OUAW), previously known as Oracle Utilities Analytics (OUA), is a suite of pre-built analytics products for the utilities industry that supports near real-time data extraction from a utility's source application systems using Oracle GoldenGate Microservices (OGG MS), as well as data transformation using Oracle Data Integrator (ODI). Also, this product provides actionable intelligence in its pre-built analytics through Oracle Analytics Server (OAS), which enables business users to make informed decisions.
On this page:
Product Overview
OUAW consists of a single data warehouse with available data integrations to the following Oracle Utilities source applications (also known as edge applications):
- Oracle Utilities Customer Care and Billing
- Oracle Utilities Customer to Meter
- Oracle Utilities Meter Data Management
- Oracle Utilities Mobile Workforce Management
- Oracle Utilities Work and Asset Management
- Oracle Utilities Network Management System
OUAW also supports end-to-end analytic workflows including the ability to drill back to the source application data. The extractors and schemas are designed with pre-built mapping between the source and the target. Its warehouse is a separate database from the operational database. The data warehousing involves large volumes of data used primarily for analysis. The data warehouse has the following features:
- Includes pre-built star schemas, pre-built metrics, pre-built reports, and pre-built dashboards.
- Makes data structures easily accessible for end user reporting needs.
- Retrieves large volumes of data quickly, which allows for fast rendering of the graphics that showcase key performance indicators (KPIs).
- Contains star schemas and graphics suited for the data retrieved from various Oracle Utilities edge applications.
- Provides you with the ability to add star schemas and graphics per your requirements using the required development tools.
- Supports schema extensibility with built-in user-defined fields, dimensions, and measures.
All tables configured to be replicated from edge application databases are replicated to the corresponding replication schema using Oracle GoldenGate. The data in the replication area is transformed into appropriate star schemas using Oracle Data Integrator. Oracle Data Integrator generates the aggregated materialized views based on the star schemas. Oracle Analytics Server is used for presenting the data in the star schemas and the associated materialized views on dashboards for each subject area.
Audience
The intended audience for the documentation on this site is administrators and specialists who must install and configure the OUAW version 2.8.0.2 or higher.
Solution Components
The diagrams below illustrate the solution components of OUAW, and the roadmap for Analytics Deployment and ETL.
Acronyms
The list below defines the acronyms most commonly used in this document.
- APEX: Oracle Application Express
- CC&B: Oracle Utilities Customer Care and Billing
- CDC: Changed Data Capture
- ELT: Extraction, Loading, and Transformation
- ETL: Extraction, Transformation, and Loading
- MDM: Oracle Utilities Meter Data Management
- MWM: Oracle Utilities Mobile Workforce Management
- NMS: Oracle Utilities Network Management System
- OAS: Oracle Analytics Server
- ODI: Oracle Data Integrator
- ODM: Oracle Utilities Operational Device Management
- OGG: Oracle GoldenGate
- OUAW: Oracle Utilities Analytics Warehouse
- OUAF: Oracle Utilities Application Framework
Terminology
This section covers basic terminology related to OUAW and the main products needed for its implementation.
Oracle Utilities Analytics Warehouse
Data Source Indicator
OUAW receives data from multiple source applications. The Data Source Indicator (DSI) is an identifier for the originating source application or system-of-record for the data. Both fact and dimension tables have a DSI value for each record.
Entity
The data warehouse consists of star schemas, which in turn consist of facts and dimensions. Each fact and dimension is considered an entity. We also use materialized views to aggregate data in the facts for improving the analytics performance. These are also considered entities.
Instance
Each deployment of the source product is considered to be an instance of that product. A product instance may include multiple servers in a clustered deployment. Instances differ by the source database connections and possibly by the differing configurations. The Data Source Indicator is unique across instances and products, as this allows you to identify the source of the data in the data warehouse.
Late Arriving Dimension
The late arriving dimensions are the dimensions where the fact (measurable quantities) table records come early when compared to the dimension table records.
Product
A product is the source system or edge application that acts as the source for the transactional data that is loaded in the data warehouse. The following scenarios are expected depending on your global presence and the implementation of your products:
- The edge applications may be deployed in different locations across the globe, and so transactions may occur in multiple time zones.
- There may be multiple instances of each of the source systems that you have implemented.
- Oracle Utilities products support cross-product integration. It is possible that some of the master data is integrated across multiple systems.
Oracle Data Integrator
Agent
An agent is a component that executes the Oracle Data Integrator jobs. Multiple agents can be created and utilized for load balancing.
Change Data Capture
This is the process of identifying and tracking data that has changed since the last known state. The change data capture mechanisms are implemented by the Journalizing Knowledge Modules (JKM).
Context
A context is a collection of logical schemas mapped to physical schemas. All logical schemas may not be mapped to the physical schemas in a context. A context identifies the physical schemas and connections that are utilized for the data access. The interfaces can be executed in any context as this allows reusing the same code for multiple instances.
Interface
An interface is a declarative mapping between the source to the target. It utilizes Loading Knowledge Modules (LKM), Integration Knowledge Module (IKM), and Check Knowledge Module (CKM), as required, for execution. The joins, aggregations, filters, and data transformations are performed in an interface.
Journalization
This is the process of keeping track of data changes, enabling incremental data access, transport, and transformation.
Knowledge Modules
Oracle Data Integrator Knowledge Modules (KMs) are components that implement reusable transformation and ELT (Extraction, Loading, and Transformation) strategies across different technologies.
- Knowledge modules are code templates. Each KM is dedicated to an individual task in the overall data integration process.
- Knowledge modules may be reused across several interfaces or models so a change made to a KM will be automatically propagated to all of the transformations that include it.
- Knowledge modules are based on logical tasks that are performed. They do not contain references to the physical objects (data stores, columns, physical paths, and so on.).
- Knowledge modules may be written with different programming languages, types, and styles (for example, native RDBMS SQL, scripting languages such as Jython or JavaScript, or even Java).
The are six types of knowledge modules mentioned below. Refer to Oracle Data Integrator documents for more details. Oracle Utilities Analytics Warehouse uses the first four knowledge modules.
- Reverse-engineering Knowledge Modules (RKM)
- Loading Knowledge Modules (LKM)
- Integration Knowledge Modules (IKM)
- Journalizing Knowledge Modules (JKM)
- Check Knowledge Modules (CKM)
- Service Knowledge Modules (SKM)
Logical Schema
A logical schema is a logical representation of a storage location. The same physical schemas can be associated with different logical schemas.
Model
A model is a set of data stores corresponding to data structures contained in a physical schema. The tables, files, JMS messages, and elements from an XML file are represented as data stores.
Package
A package is made up of a sequence of steps organized into an execution diagram.
Physical Schema
A physical schema is the actual schema where data is retrieved or stored.
Reverse Engineering
Reverse engineering is the process of identifying the data structures, constraints, and relations from an existing data store. The data store can be any database management system, files, or any other Oracle Data Integrator supported data store.
Scenario
A scenario is a self-contained prepackaged executable version of a package, interface, or procedure in Oracle Data Integrator. The logic within a scenario cannot be viewed using the Oracle Data Integrator client.
Oracle GoldenGate
Distribution Service
Distribution Service functions as a networked data distribution agent in support of conveying and processing data and commands in a distributed deployment. It is a high-performance application that can handle multiple commands and data streams from multiple source trail files, concurrently. Distribution Service replaces the classic multiple source-side data pumps with a single instance service.
This service distributes one or more trails to one or more destinations and provides lightweight filtering only (no transformations).
Extract
The Extract process is the extraction (capture) mechanism of Oracle GoldenGate. The extract process runs on a source system or on a downstream database, or both, depending on the database and the implementation requirements. When configured for change synchronization, extract process captures the DML and DDL operations that are performed on objects in the Extract configuration. The Extract process stores these operations until it receives commit records or rollbacks for the transactions that contain them. When a rollback is received, the Extract discards the operations for that transaction. When a commit is received, the Extract persists the transaction to disk in a series of files called a trail, where it is queued for propagation to the target system. All of the operations in each transaction are written to the trail as a sequentially organized transaction unit. This design ensures both speed and data integrity.
Service Manager
The Service Manager is the primary watchdog service within Oracle GoldenGate MA that allows you to control and administer the deployments and associated services running on the host server. In the Services we can see how many services are running, failed and other. Service Manager is run as a system service and maintains inventory and configuration information about your deployments and allows you to maintain multiple local deployments. We can use Service Manager to start or stop various services like Administrator server, Distribution server and Receiver server.
Parameter Files
Most Oracle GoldenGate functionalities are controlled by means of parameters specified in the parameter files. A parameter file is a plain text file that is read by an associated Oracle GoldenGate process.
Replicat
The Replicat process runs on the target system, reads the trail on that system, and then reconstructs the DML or DDL operations and applies them to the target database. The Replicat uses the dynamic SQL to compile a SQL statement once, and then execute it many times with different bind variables.
Trail Files
To support the continuous extraction and replication of the database changes, Oracle GoldenGate stores records of the captured changes temporarily on the disk in a series of files called a trail. A trail can exist on the source system, an intermediary system, the target system, or any combination of those systems, depending on how you configure Oracle GoldenGate. On the local system, it is known as an extract trail (or local trail). On a remote system, it is known as a remote trail.
Oracle Analytics Server
Analysis/Report
This is a query that you create on the Criteria tab in Presentation Services. An analysis can optionally contain one or more filters or selection steps to restrict the results.
Business Model and Mapping Layer
The Business Model and Mapping Layer defines the business or logical model of the data and specifies the mapping between the business model and the physical layer schemas. This layer can contain one or more business models. The business model and mapping layer determines the analytic behavior that is seen by users, and defines the superset of objects available to you. It also hides the complexity of the source data models. Sometimes, this layer is also referred to as the logical layer.
Dashboard
An object that provides personalized views of corporate and external information. A dashboard consists of one or more pages. Pages can display anything that you can access or open with a web browser, such as results of analyses, images, alerts from agents, and so on.
Oracle Analytics Server Presentation Catalog
It stores business intelligence objects, such as analyses and dashboards, and provides an interface where you create, access, and manage objects, and perform specific object-based tasks (for example, export, print, and edit). The catalog is organized into folders that are either shared or personal.
Oracle Analytics Server Repository
It is a file that stores Oracle Analytics Server metadata. The metadata defines logical schemas, physical schemas, physical-to-logical mappings, aggregate table navigation, and other constructs. The repository file has an extension of .rpd. Oracle Analytics Server repositories can be edited using the Oracle Utilities Analytics Administration Tool.
Physical Layer
It is a layer of the Oracle Analytics Server repository containing objects that represent physical data constructs from back-end data sources. The physical layer defines the objects and relationships available for writing physical queries. This layer encapsulates source dependencies to enable portability and federation.
Presentation Layer
This layer provides a way to present customized, secure, role-based views of a business model. It adds a level of abstraction over the business model and the mapping layer in the Oracle Analytics Server repository. The presentation layer provides the view of the data as seen by the person who builds analytics in Presentation Services and other client tools and applications.
Subject Area
In an Oracle Analytics Server repository, a subject area is an object in the presentation layer that organizes and presents data about a business model. It is the highest-level object in the presentation layer and represents the view of the data that you see in Presentation Services. Oracle Analytics Server repository subject areas contain presentation tables, presentation columns, and presentation hierarchies. In Presentation Services, subject areas contain folders, measure columns, attribute columns, hierarchical columns, and levels.