Integrate with Oracle Utilities SaaS Applications and Export Data for Analytics

As a provider of utility services, you can manage your operations efficiently by taking advantage of the seamless integration between Oracle Utilities SaaS and all your applications deployed in Oracle Cloud and on-premises. Oracle offers multiple capabilities that leverage the power of Oracle Cloud for integrating your applications with Oracle Utilities SaaS and exporting data for analytics.

Architecture

This architecture shows the bulk data export, API integration, and data querying capabilities within Oracle Utilities SaaS along with the Oracle Cloud capabilities that you can use for SaaS workloads.

• Bulk data export

  Export large amounts of data from Oracle Utilities SaaS to downstream targets, such as data warehouses, data lakes, and other applications.

• API integration

  APIs are delivered as part of the base Oracle Utilities SaaS. A framework that allows you to create APIs against the SaaS data is also provided.

• Analytics, querying, and reporting

  You can query data, write and schedule reports, and manipulate analytical subject areas within the tools provided with Oracle Utilities SaaS applications.

The following diagram illustrates this reference architecture.

Description of the illustration utilities-integration.png

utilities-integration-oracle.zip

The architecture has the following components:

• General components
  • Oracle Cloud region

    An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).

    In this architecture, the Oracle Utilities SaaS applications and the Oracle Cloud Infrastructure resources are in the same region.

  • Oracle Utilities SaaS

    The Oracle Utilities SaaS applications related to this reference architecture include Oracle Utilities Customer Cloud, Oracle Utilities Meter Solution Cloud Service, and Oracle Utilities Work and Asset Cloud Service.

  • Oracle Cloud Infrastructure tenancy

    A tenancy is a secure and isolated partition that Oracle sets up within Oracle Cloud when you sign up for Oracle Cloud Infrastructure. You can create, organize, and administer your resources in Oracle Cloud within your tenancy. A tenancy is synonymous with a company or organization. Usually, a company will have a single tenancy and reflect its organizational structure within that tenancy. A single tenancy is usually associated with a single subscription, and a single subscription usually only has one tenancy.

  • On-premises data center

    The architecture shows integration between Oracle Utilities SaaS and any workloads deployed in your on-premises data center.

  • Oracle Cloud Infrastructure Identity and Access Management (IAM)

    Oracle Cloud Infrastructure Identity and Access Management (IAM) is the access control plane for Oracle Cloud Infrastructure (OCI) and Oracle Cloud Applications. The IAM API and the user interface enable you to manage identity domains and the resources within the identity domain. Each OCI IAM identity domain represents a standalone identity and access management solution or a different user population.

  • Oracle Identity Cloud Service

    Oracle Identity Cloud Service provides identity management, single sign-on (SSO), and identity governance for a wide range of SaaS and on-premises applications.

  • Cloud Guard

    You can use Oracle Cloud Guard to monitor and maintain the security of your resources in Oracle Cloud Infrastructure. Cloud Guard uses detector recipes that you can define to examine your resources for security weaknesses and to monitor operators and users for risky activities. When any misconfiguration or insecure activity is detected, Cloud Guard recommends corrective actions and assists with taking those actions, based on responder recipes that you can define.

• Bulk data export components
  • SaaS batch framework

    The batch framework allows you to run batch jobs that perform logical work against the Oracle Utilities SaaS applications. You can configure the batch jobs to run on a schedule and write results to Oracle Cloud Infrastructure Object Storage.

  • Generalized data export batches

    Generalized Data Export batches export data to messages in JSON format, which is compressed before delivery. The JSON format of these messages is closely linked to the underlying data model of the applications to provide a raw image of any of the core data from SaaS. This process uses change data-capture mechanisms to ensure that you only get the latest snapshot of data when the initial export is completed.

    Because utility meters send an extremely high volume of measurement data, a specialized data export is used, in tandem with Generalized Data Export for all other data.

  • DataConnect batches

    DataConnect batches export data to messages in a variety of delimiter-based formats. These batches are available to you in both Oracle Utilities Customer Cloud and Oracle Utilities Meter Solution Cloud Service. The data in these export messages focuses on the most crucial customer and metering data, and it's formatted to match the import format for Oracle Utilities Opower.

  • SQL-driven batches

    The plugin-driven batches within the Oracle Utilities SaaS applications allow you to define SQL to drive the execution of a batch.

  • Oracle Cloud Infrastructure Object Storage buckets

    Object storage provides quick access to large amounts of structured and unstructured data of any content type, including database backups, analytic data, and rich content such as images and videos. You can safely and securely store and then retrieve data directly from the internet or from within the cloud platform. You can seamlessly scale storage without experiencing any degradation in performance or service reliability. Use standard storage for "hot" storage that you need to access quickly, immediately, and frequently. Use archive storage for "cold" storage that you retain for long periods of time and seldom or rarely access.

  • Oracle Cloud Infrastructure functions

    Oracle Cloud Infrastructure Functions is a fully managed, multitenant, highly scalable, on-demand, Functions-as-a-Service (FaaS) platform. It is powered by the Fn Project open source engine. Functions enable you to deploy your code, and either call it directly or trigger it in response to events. Oracle Functions uses Docker containers hosted in Oracle Cloud Infrastructure Registry.

    Either use functions to push the data to your target data staging area based on events generated by Oracle Cloud Infrastructure Object Storage, or set up a data retrieval job to poll for data to pull. You need only one of these options to complete the architecture.

  • Data retrieval job

    This is a job that you set up on your target environment to poll Oracle Cloud Infrastructure Object Storage for new data to send to your target for data staging.

  • Target for data staging

    The data staging area on your target environment is the location where you stage data to be picked up for your downstream system.

• API integration components
  • SaaS messaging framework

    The Message Framework within the Oracle Utilities SaaS applications allows you to use existing APIs and configure new inbound APIs. This framework also allows you to send messages to external APIs.

  • Outbound message

    An Outbound Message is a method that sends a message to an external API from the Oracle Utilities SaaS applications.

  • API

    An API that's hosted within the Oracle Utilities SaaS applications. Both REST and SOAP protocols are available.

  • Oracle Integration

    Oracle Integration is a fully managed service that allows you to integrate your applications, automate processes, gain insight into your business processes, and create visual applications.

    Oracle Integration for SaaS, a streamlined version of Oracle Integration, gives you the features and benefits of Oracle Integration with a focus on SaaS.

    In this architecture, multiple integration flows are defined.

• Analytics, querying, and reporting components
  • Built-in SaaS reporting, using Business Intelligence (BI) Publisher

    The reporting tools included with the Oracle Utilities SaaS applications allow you to create reports with BI Publisher.

  • Built-in SaaS analytics, using Oracle Analytics

    The analytical tools included with the Oracle Utilities SaaS applications allow you to create new analytical visualizations with Oracle Analytics. Also, a library of predefined data subject areas and related visualizations are provided with the SaaS applications.

  • Built-in SaaS SQL querying, using SQL Developer Web

    SQL Developer Web is included with the Oracle Utilities SaaS applications, which allows you to query data using SQL.

Recommendations

Use the following recommendations as a starting point. Your requirements might differ from the architecture described here.

• Connectivity to Oracle Cloud Infrastructure Object Storage

  Connecting Oracle Utilities SaaS to Oracle Cloud Infrastructure Object Storage in the same Oracle Cloud region is preferred. If you need data in a different region, you can use the object storage replication capabilities.

• Network design

  When you create a virtual cloud network (VCN), determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.

  Select CIDR blocks that don't overlap with any other network (in Oracle Cloud Infrastructure, your on-premises data center, or another cloud provider) to which you intend to set up private connections.

  After you create a VCN, you can change, add, and remove its CIDR blocks.

  When you design the subnets, consider your traffic flow and security requirements. Attach all the resources within a specific tier or role to the same subnet, which can serve as a security boundary.

  Use regional subnets.

• Connectivity with the on-premises data center

  You can use Oracle Cloud Infrastructure VPN Connect for connecting a cloud VCN to your on-premises network, for an end-to-end encrypted connection. If you need dedicated and higher-bandwidth private connectivity, use FastConnect.

Considerations

When integrating with Oracle Utilities SaaS, consider the following factors:

• Service limits

  Consider the limits and quotas for the Oracle Cloud Infrastructure services used in the topology.

• Security

  Federating Oracle Cloud Infrastructure with Oracle Identity Cloud Service enables you to manage authentication and authorization for all the resources in the topology by using a centralized identity system. Note that depending on the date that your tenancy was created, it might be pre-federated with Oracle Identity Cloud Service.

• Cost considerations for Object Storage

  Use the low-cost Archive Storage service to store data that's rarely accessed but that must be retained for a long duration.

  Define lifecycle management policies to automatically move data to Archive Storage or delete data after a specified duration.

• Oracle Identity Cloud Service licensing

  A foundational license for Oracle Identity Cloud Service is provided with Oracle Utilities SaaS. You might need other features of Oracle Identity Cloud Service for more advanced identity management, such as federation, which requires additional licensing beyond what's provided with Oracle Utilities SaaS.

For real-time integrations, consider the following:

Real-Time Integration	Which Components to Leverage
You need the Oracle Utilities SaaS applications to read data from an external application in real time.	Use Outbound messages in Oracle Utilities SaaS applications to call external APIs. Turn data persistence off. Use Oracle Integration, a fully managed service to design integrations, monitor traffic, and manage connections between applications.
You need the Oracle Utilities SaaS application to write transactions to an external application in real time.	Use Outbound messages in Oracle Utilities SaaS applications to call external APIs. Data persistence is turned on by default. Use Oracle Integration.
You need an external system to read data from your Oracle Utilities SaaS application in real time.	Leverage REST APIS for GET in Oracle Utilities SaaS applications.
You need an external system to write transactions to your Oracle Utilities SaaS application in real time.	Leverage REST APIs for POST / PUT / PATCH in Oracle Utilities SaaS applications. Some of these REST APIs commit in real-time while others stage the request for batch data synchronization.

For data replication, consider the following:

Data Replication	Which Components to Leverage
You need a one-time copy of the entire database for your Oracle Utilities SaaS application sent to an external database.	Request a one-time Data Pump export through the Oracle Utilities support team.
You need an ongoing export of major data areas to be consumed by external applications from your Oracle Utilities SaaS application.	Run Generalized Data Export in Oracle Utilities SaaS applicationss. Note that some higher volume tables require a special optimized batch. External systems must consume export files from Oracle Object Storage.
You need to export core data for Oracle Utilities Opower or an external application from your Oracle Utilities SaaS application.	Run DataConnect in Oracle Utilities SaaS applications to send data to Oracle Utilities Opower or an external application such as a customer portal or self-service applications.
You need to build a custom export to be consumed by an external application from your Oracle Utilities SaaS application.	Plug-in Driven Batches in the Oracle Utilities SaaS application allow you to define SQL statements to drive the execution of a data retrieval batch.

For analytics reporting, consider the following:

Analytics Reporting	Which Components to Leverage
You need operational reporting for your Oracle Utilities SaaS application.	Oracle BI Publisher is included with your Oracle Utilities SaaS application to enable operational reporting.
You need to explore, visualize, and analyze data from your Oracle Utilities SaaS application.	Oracle Utilities Analytics Visualization provides prebuilt data areas and visualizations for data. The Oracle Analytics platform included within your Oracle Utilities SaaS app enables deeper data exploration as well as custom visualization.
You need to query data from your Oracle Utilities SaaS application.	SQL Developer Web is included with your Oracle Utilities SaaS application which allows you to query data using SQL.

Deploy

Oracle Cloud Infrastructure Object Storage and Oracle Identity Cloud Service are deployed in your tenancy alongside your Oracle Utilities SaaS application as prerequisites of the service-ordering process. Complete the following high-level steps to set up those services and deploy a serverless function.

1. Set up Oracle Cloud Infrastructure Object Storage.
   Follow the instructions in Object Storage Setup Overview in the Cloud Services Administration Guide.
2. Create users and set up access permissions in Oracle Identity Cloud Service.
   Follow the instructions in Setting Up a New User in the Cloud Services Administration Guide.
3. Deploy a serverless function to push data from object storage.
   The architecture diagram doesn't show the virtual cloud network (VCN), subnets, and other resources required to deploy serverless functions in Oracle Cloud Infrastructure resources.
   You can customize and use the automation provided in Deploy an event-triggered serverless application to deploy your function.
4. Use prebuilt Oracle Integration packages available in Oracle Cloud Marketplace to integrate your Oracle Utilities SaaS applications with other applications.

Explore More

Use the following resources to learn more about integrating with Oracle Utilities SaaS.

• For integration capabilities of Oracle Utilities SaaS applications, see the Administrative User Guide.

  For information about generalized data export, DataConnect, and plugin-driven batches, search for the topic you need. For example, search for "generalized data export," "DataConnect," or "plugin driven batch."

• For information about Oracle Utilities REST APIs, see Oracle Utilities REST API for Metering and Customer Information.
• Oracle Cloud Infrastructure features and best practices:

  • Overview of Archive Storage

  • Using Object Lifecycle Management

  • Federating Oracle Cloud Infrastructure Identity and Access Management with Oracle Identity Cloud Service

  • Service Limits

  • Best practices framework for Oracle Cloud Infrastructure

  • Oracle Cloud Infrastructure Documentation

  • Oracle Cloud Cost Estimator

Change Log

This log lists significant changes:

January 26, 2024	Updated product information in the diagram, content, and publication metadata. Revised the steps and support links in Deploy. Updated the resources listed in Explore More.
June 4, 2021	Added a decision table to the Considerations section for choosing methods based on typical use cases.
February 16, 2021	Added information about prebuilt integration flows in the Deploy section.