Integrate with Oracle Utilities SaaS Applications and Export Data for Analytics

As a provider of utility services, you can manage your operations efficiently by taking advantage of the seamless integration between Oracle Utilities SaaS and all your applications deployed in Oracle Cloud and on-premises. Oracle offers multiple capabilities that leverage the power of Oracle Cloud for integrating your applications with Oracle Utilities SaaS and exporting data for analytics.

Architecture

This architecture shows the bulk data export, API integration, and data querying capabilities within Oracle Utilities SaaS along with the Oracle Cloud capabilities that you can use for SaaS workloads.

  • Bulk data export

    Export large amounts of data from Oracle Utilities SaaS to downstream targets, such as data warehouses, data lakes, and other applications.

  • API integration

    APIs are delivered as part of the base Oracle Utilities SaaS. A framework that allows you to create APIs against the SaaS data is also provided.

  • Analytics, querying, and reporting

    You can query data, write and schedule reports, and manipulate analytical subject areas within the tools provided with Oracle Utilities SaaS applications.

The following diagram illustrates this reference architecture.

Description of utilities-integration.png follows
Description of the illustration utilities-integration.png
The architecture has the following components:
  • General components
    • Oracle Cloud region

      An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).

      In this architecture, the Oracle Utilities SaaS applications and the Oracle Cloud Infrastructure resources are in the same region.

    • Oracle Utilities SaaS

      The Oracle Utilities SaaS applications related to this reference architecture include Oracle Utilities Customer Cloud service, Oracle Utilities Meter Solution Cloud service, and Oracle Utilities Work and Asset Cloud service.

    • Oracle Cloud Infrastructure tenancy

      A tenancy is a secure and isolated partition that Oracle sets up within Oracle Cloud when you sign up for Oracle Cloud Infrastructure. You can create, organize, and administer your resources in Oracle Cloud within your tenancy.

    • On-premises data center

      The architecture shows integration between Oracle Utilities SaaS and any workloads deployed in your on-premises data center.

    • Oracle Cloud Infrastructure Identity and Access Management (IAM)

      Oracle Cloud Infrastructure Identity and Access Management (IAM) enables you to control who can access your resources in Oracle Cloud Infrastructure and the operations that they can perform on those resources.

    • Oracle Identity Cloud Service

      Oracle Identity Cloud Service provides identity management, single sign-on (SSO), and identity governance for a wide range of SaaS and on-premises applications.

    • Cloud Guard

      You can use Oracle Cloud Guard to monitor and maintain the security of your resources in Oracle Cloud Infrastructure. Cloud Guard uses detector recipes that you can define to examine your resources for security weaknesses and to monitor operators and users for risky activities. When any misconfiguration or insecure activity is detected, Cloud Guard recommends corrective actions and assists with taking those actions, based on responder recipes that you can define.

  • Bulk data export components
    • SaaS batch framework

      The batch framework allows you to run batch jobs that perform logical work against the Oracle Utilities SaaS applications. You can configure the batch jobs to run on a schedule and write results to Oracle Cloud Infrastructure Object Storage.

    • Generalized data export batches

      Generalized Data Export batches export data to messages in JSON format, which is compressed before delivery. The JSON format of these messages is closely linked to the underlying data model of the applications to provide a raw image of any of the core data from SaaS. This process uses change data-capture mechanisms to ensure that you only get the latest snapshot of data when the initial export is completed.

      Because utility meters send an extremely high volume of measurement data, a specialized data export is used, in tandem with Generalized Data Export for all other data.

    • DataConnect batches

      DataConnect batches export data to messages in a variety of delimiter-based formats. These batches are available to you in both Oracle Utilities Customer Cloud Service and Oracle Utilities Meter Solution Cloud Service. The data in these export messages focuses on the most crucial customer and metering data, and it's formatted to match the import format for Oracle Utilities Opower.

    • SQL-driven batches

      The plugin-driven batches within the Oracle Utilities SaaS applications allow you to define SQL to drive the execution of a batch.

    • Oracle Cloud Infrastructure Object Storage buckets

      Object storage provides quick access to large amounts of structured and unstructured data of any content type, including database backups, analytic data, and rich content such as images and videos. Use standard storage for "hot" storage that you need to access quickly, immediately, and frequently. Use archive storage for "cold" storage that you retain for long periods of time and seldom or rarely access.

    • Oracle Cloud Infrastructure functions

      Oracle Functions is a fully managed, multitenant, highly scalable, on-demand, Functions-as-a-Service (FaaS) platform. It is powered by the Fn Project open source engine. Functions enable you to deploy your code, and either call it directly or trigger it in response to events. Oracle Functions uses Docker containers hosted in Oracle Cloud Infrastructure Registry.

      Either use functions to push the data to your target data staging area based on events generated by Oracle Cloud Infrastructure Object Storage, or set up a data retrieval job to poll for data to pull. You need only one of these options to complete the architecture.

    • Data retrieval job

      This is a job that you set up on your target environment to poll Oracle Cloud Infrastructure Object Storage for new data to send to your target for data staging.

    • Target for data staging

      The data staging area on your target environment is the location where you stage data to be picked up for your downstream system.

  • API integration components
    • SaaS messaging framework

      The Message Framework within the Oracle Utilities SaaS applications allows you to use existing APIs and configure new inbound APIs. This framework also allows you to send messages to external APIs.

    • Outbound message

      An Outbound Message is a method that sends a message to an external API from the Oracle Utilities SaaS applications.

    • API

      An API that's hosted within the Oracle Utilities SaaS applications. Both REST and SOAP protocols are available.

    • Oracle Integration

      Oracle Integration is a fully managed service that allows you to integrate your applications, automate processes, gain insight into your business processes, and create visual applications.

      Oracle Integration for SaaS, a streamlined version of Oracle Integration, gives you the features and benefits of Oracle Integration with a focus on SaaS.

      In this architecture, multiple integration flows are defined.

  • Analytics, querying, and reporting components
    • Built-in SaaS reporting, using Business Intelligence (BI) Publisher

      The reporting tools included with the Oracle Utilities SaaS applications allow you to create reports with BI Publisher.

    • Built-in SaaS analytics, using Oracle Analytics

      The analytical tools included with the Oracle Utilities SaaS applications allow you to create new analytical visualizations with Oracle Analytics. Also, a library of predefined data subject areas and related visualizations are provided with the SaaS applications.

    • Built-in SaaS SQL querying, using SQL Developer Web

      SQL Developer Web is included with the Oracle Utilities SaaS applications, which allows you to query data using SQL.

Recommendations

Use the following recommendations as a starting point. Your requirements might differ from the architecture described here.

  • Connectivity to Oracle Cloud Infrastructure Object Storage

    Connecting Oracle Utilities SaaS to Object Storage in the same Oracle Cloud region is preferred. If you need data in a different region, you can use the Object Storage replication capabilities.

  • Network design

    When you create a virtual cloud network (VCN), determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.

    Select CIDR blocks that don't overlap with any other network (in Oracle Cloud Infrastructure, your on-premises data center, or another cloud provider) to which you intend to set up private connections.

    After you create a VCN, you can change, add, and remove its CIDR blocks.

    When you design the subnets, consider your traffic flow and security requirements. Attach all the resources within a specific tier or role to the same subnet, which can serve as a security boundary.

    Use regional subnets.

  • Connectivity with the on-premises data center

    You can use Oracle Cloud Infrastructure VPN Connect for connecting a cloud VCN to your on-premises network, for an end-to-end encrypted connection. If you need dedicated and higher-bandwidth private connectivity, use FastConnect.

Considerations

When integrating with Oracle Utilities SaaS, consider the following factors:

  • Service limits

    Consider the limits and quotas for the Oracle Cloud Infrastructure services used in the topology.

  • Security

    Federating Oracle Cloud Infrastructure with Oracle Identity Cloud Service enables you to manage authentication and authorization for all the resources in the topology by using a centralized identity system. Note that depending on the date that your tenancy was created, it might be pre-federated with Oracle Identity Cloud Service.

  • Cost considerations for Object Storage

    Use the low-cost Archive Storage service to store data that's rarely accessed but that must be retained for a long duration.

    Define lifecycle management policies to automatically move data to Archive Storage or delete data after a specified duration.

  • Oracle Identity Cloud Service licensing

    A foundational license for Oracle Identity Cloud Service is provided with Oracle Utilities SaaS. You might need other features of Oracle Identity Cloud Service for more advanced identity management, such as federation, which requires additional licensing beyond what's provided with Oracle Utilities SaaS.

For real-time integrations, consider the following:

Real-Time Integration Which Components to Leverage

You need the Oracle Utilities SaaS applications to read data from an external application in real time.

Use Outbound messages in Oracle Utilities SaaS applications to call external APIs. Turn data persistence off.

Use Oracle Integration, a fully managed service to design integrations, monitor traffic, and manage connections between applications.

You need the Oracle Utilities SaaS application to write transactions to an external application in real time.

Use Outbound messages in Oracle Utilities SaaS applications to call external APIs. Data persistence is turned on by default.

Use Oracle Integration.

You need an external system to read data from your Oracle Utilities SaaS application in real time.

Leverage REST APIS for GET in Oracle Utilities SaaS applications.

You need an external system to write transactions to your Oracle Utilities SaaS application in real time.

Leverage REST APIs for POST / PUT / PATCH in Oracle Utilities SaaS applications. Some of these REST APIs commit in real-time while others stage the request for batch data synchronization.

For data replication, consider the following:

Data Replication Which Components to Leverage

You need a one-time copy of the entire database for your Oracle Utilities SaaS application sent to an external database.

Request a one-time Data Pump export through the Oracle Utilities support team.

You need an ongoing export of major data areas to be consumed by external applications from your Oracle Utilities SaaS application.

Run Generalized Data Export in Oracle Utilities SaaS applicationss. Note that some higher volume tables require a special optimized batch.

External systems must consume export files from Oracle Object Storage.

You need to export core data for Oracle Utilities Opower or an external application from your Oracle Utilities SaaS application.

Run DataConnect in Oracle Utilities SaaS applications to send data to Oracle Utilities Opower or an external application such as a customer portal or self-service applications.

You need to build a custom export to be consumed by an external application from your Oracle Utilities SaaS application.

Plug-in Driven Batches in the Oracle Utilities SaaS application allow you to define SQL statements to drive the execution of a data retrieval batch.

For analytics reporting, consider the following:

Analytics Reporting Which Components to Leverage

You need operational reporting for your Oracle Utilities SaaS application.

Oracle BI Publisher is included with your Oracle Utilities SaaS application to enable operational reporting.

You need to explore, visualize, and analyze data from your Oracle Utilities SaaS application.

Oracle Utilities Analytics Visualization provides prebuilt data areas and visualizations for data. The Oracle Analytics platform included within your Oracle Utilities SaaS app enables deeper data exploration as well as custom visualization.

You need to query data from your Oracle Utilities SaaS application.

SQL Developer Web is included with your Oracle Utilities SaaS application which allows you to query data using SQL.

Deploy

Oracle Cloud Infrastructure Object Storage and Oracle Identity Cloud Service are deployed in your tenancy alongside your Oracle Utilities SaaS application as prerequisites of the service-ordering process. Complete the following steps to set up those services and deploy a serverless function.

  1. Download the required documentation:
    1. Go to the Oracle Utilities documentation site.
    2. Locate your Oracle Utilities SaaS application, and click Product Documentation.
    3. Download the following documentation:
      • Object Storage Setup Guide
      • End User Provisioning Guide
  2. Set up Oracle Cloud Infrastructure Object Storage.
    Follow the instructions in Object Storage Setup Guide.
  3. Create users and set up access permissions in Oracle Identity Cloud Service.
    Follow the instructions in End User Provisioning Guide.
  4. Deploy a serverless function to push data from Object Storage.
    The architecture diagram doesn't show the virtual cloud network (VCN), subnets, and other resources required to deploy serverless functions in Oracle Cloud Infrastructure resources.
    You can customize and use the automation provided in Deploy an event-triggered serverless application to deploy your function.
  5. Use prebuilt Oracle Integration packages available in Oracle Cloud Marketplace to integrate your Oracle Utilities SaaS applications with other applications.

Explore More

Use the following resources to learn more about integrating with Oracle Utilities SaaS.

Change Log

This log lists significant changes: