Monetize Data with Oracle Cloud Infrastructure

Oracle Cloud Infrastructure (OCI) allows you to manage the complete data lifecycle—from data ingestion, through storage, to usage—while providing strong security and governance. Some data usages is through data products, either data itself or artifacts derived from data, such as reports, analysis, and predictions. You might have a business case for monetizing these data products; for example, selling curated data, or selling reports or predictions based on data, to third parties who can make use of these data products for their own value-added activities.

This architecture presents a way to monetize data by setting up a payment framework that allows you to charge for access to it, making it a revenue generator.

Architecture

This architecture addresses the challenge of charging for data products exposed through an API. These products could be, for example, raw data, perhaps exposed by the Oracle Database through Oracle REST Data Services (ORDS), a system that mediates access to raw data (like an app that aggregates and filters data from one or more sources, presenting those data to clients through an API), or some functionality derived from data (for example, predictions or scores delivered by a machine learning runtime, trained on proprietary data).

This architecture allows client requests to be intercepted, validated, and charged for during access to such data products.

The key parts of an architecture to monetize data on OCI are:

• An API Gateway that provides access to data products and applies policies governing that access.
• An Identity Provider (here, OCI Identity and Access Management [AIM]) that authenticates data products’ customers.
• A function that authorizes access to products by checking access rights and also coordinates charging for data product access.
• A CRM system, or other database, for tracking customers’ rights to access specific data products and establishing the terms (for example, subscription or pay-per-use) that govern access.
• A means of charging customers, either through a third party service provider (for example, Stripe) or by recording a transaction (for example, as an Accounts Receivable entry in ERP).
• Services exposing data products, for example:
  • Autonomous Data Warehouse (ADW), which can expose REST interfaces to data, offers Delta Sharing APIs, and can offer SQL access to its own data and data in object storage.
  • Machine learning and AI runtimes.
  • Any other services, on-cloud or on-premises, that can make monetizable data available

The following diagram illustrates this reference architecture.

Description of the illustration data-monetization.png

data-monetization-oracle.zip

Data products can be data themselves (for example, historic financial figures) or can be derived from data (for example, KPIs, trends, predictions, scores) made accessible through APIs, downloads, and so on.

The above architecture works as follows.

1. The customer authenticates with identity provider.
2. The customer accesses the data product API through an API Gateway, which will later apply its own policies (for example, throttling) after authorizing the request.
3. The API Gateway invokes a function to authorize the request.
4. The function validates provided customer tokens with the identity provider.
5. The function then checks the customer’s access rights to data product in the CRM or other system and also checks whether subscription or per-use payment applies. If a subscription applies, the function checks whether it is valid.
6. The function records data product use for payment:
   1. Recording use in a ledger; and/or
   2. Executing an online payment through a payment provider.
7. Once authorized and monetized, the API Gateway provides access to data product.

Note that other functions might be involved in delivering data products; for example, making data available through an ephemeral file download.

Steps 4 and 5 above can be implemented as described OCI document, Passing Tokens to Authorizer Functions to Add Authentication and Authorization to API Deployments—which you can access from the Explore More topic, below—where your custom function checks subscriptions and/or performs charging as part of the authorization process. The API Gateway will cache the results of the authorization for a minimum of 60 seconds, so if you need to charge for individual customer requests that come more frequently, you can opt to deploy the authorization function as a proxy to the monetized API, as shown in the alternative architecture below:

Description of the illustration alternate-data-monetization.png

alternate-data-monetization-oracle.zip

In this alternative, the flow is slightly different from the one above.

1. The customer authenticates with identity provider.
2. The customer access data product API through API Gateway, which will later apply its own policies (for example, throttling) after authorizing the request.
3. The API Gateway invokes a function to authorize request.
4. The function validates provided customer tokens with identity provider
5. The function checks customer’s access rights to data product in CRM or other system, and also checks whether subscription or per-use payment applies. If a subscription applies, the functions checks whether that subscription is valid.
6. Once authorized, the API Gateway forwards the request to a proxy function.
7. On a per-request basis, the proxy function charges for the access to the data product. Note that this charging can also be done after a successful access to the data product, avoiding the situation that customers are charged if the access fails. Charging is done by either:
   1. Recording use in a ledger; and/or
   2. Executing an online payment through a payment provider.
8. The proxy function accesses the monetized data on behalf of the customer.

This architecture has the following components:

• Identity and access management (IAM)

  IAM enables you to control who can access your resources in OCI and the operations that they can perform on those resources.

• API gateway

  Oracle API Gateway service enables you to publish APIs with private endpoints that are accessible from within your network, and which you can expose to the public internet if required. The endpoints support API validation, request and response transformation, CORS, authentication and authorization, and request limiting.

• Functions

  Oracle Functions is a fully managed, multi-tenant, highly scalable, on-demand, Functions-as-a-Service (FaaS) platform. It is powered by the Fn Project open source engine. Functions enable you to deploy your code, and either call it directly or trigger it in response to events. Oracle Functions uses Docker containers hosted in the OCI Registry

These representative data product sources are also in this reference architecture:

• Autonomous Data Warehouse

  Oracle Autonomous Data Warehouse (ADW) is a self-driving, self-securing, self-repairing database service optimized for data warehousing workloads. You do not need to configure or manage any hardware, or install any software. OCI handles creating the database, as well as backing up, patching, upgrading, and tuning the database.

• Oracle Machine Learning

  Oracle Machine Learning, or Machine Learning in the Oracle Database, supports data exploration, preparation, and machine learning modeling at scale using SQL, R, Python, REST, automated machine learning (AutoML), and no-code interfaces, running directly on the data in the database. It allows the deployment and management of native in-database models and ONNX-format models in the Oracle Autonomous Database environment. Application developers use models through easy-to-integrate REST endpoints.

Recommendations

Use the following recommendations as a starting point when monetizing data with Oracle Cloud Infrastructure. Your requirements might differ from the architecture described here.

• Data Products

  Almost any body of data, or anything derived from data, can be a data product. Useful data products offer value by being comprehensive and trustworthy, and normally achieve that status through some sort of curation by a data product owner. Data governance forms a key part of data product management, covering data cataloging, access control and data quality management, and lineage tracking—all aspects of data products that are outside the narrow focus of this architecture and are therefore not shown. Oracle recommends, but not necessary, that you consider data product lifecycles and management when considering the deployment of a data monetization framework, such as the one presented here.

• Security

  Use Oracle Cloud Guard to monitor and maintain the security of your resources in Oracle Cloud Infrastructure proactively. Cloud Guard uses detector recipes that you can define to examine your resources for security weaknesses and to monitor operators and users for risky activities. When any misconfiguration or insecure activity is detected, Cloud Guard recommends corrective actions and assists with taking those actions, based on responder recipes that you can define.

  For resources that require maximum security, Oracle recommends that you use security zones. A security zone is a compartment associated with an Oracle-defined recipe of security policies that are based on best practices. For example, the resources in a security zone must not be accessible from the public internet and they must be encrypted using customer-managed keys. When you create and update resources in a security zone, Oracle Cloud Infrastructure validates the operations against the policies in the security-zone recipe, and denies operations that violate any of the policies.

• Cloud Guard

  Clone and customize the default recipes provided by Oracle to create custom detector and responder recipes. These recipes enable you to specify what type of security violations generate a warning and what actions are allowed to be performed on them. For example, you might want to detect Object Storage buckets that have visibility set to public.

  Apply Cloud Guard at the tenancy level to cover the broadest scope and to reduce the administrative burden of maintaining multiple configurations.

  You can also use the Managed List feature to apply certain configurations to detectors.

Considerations

Consider the following points when deploying this architecture.

• Monetization Approach

  Consider how customers should be charged for access to data products. Is a per-access payment approach (for example, using Stripe) suitable, or is a more comprehensive subscription-based approach more appropriate given the expected customer interactions with the data products?

• Customer and Subscription Management

  How are customers' rights to access specific data products recorded and how are subscriptions to data products managed, if required? This architecture shows how payments for access to data products can be triggered but, if more than a simple one-off payment per use is required, then some sort of record of the rights for customers to access specific data products is required, and some sort of subscription management framework might be required. Are these functions available through your CRM system? Are your requirements simple enough to roll your own custom solution or is a more capable subscription-management component required?

• User Provisioning and Access Control

  When customers are granted access to data products, how are they provisioned into OCI IAM (which authenticates them)? How are they de-provisioned when their access rights are to be revoked? These considerations arise a part of the wider discussion around how and to whom monetized data products are made available.

Explore More

Learn more about monetizing data with Oracle Cloud Infrastructure.

Review these additional resources:

• Best practices framework for Oracle Cloud Infrastructure

• Passing Tokens to Authorizer Functions to Add Authentication and Authorization to API Deployments

• OCI API Gateway

• OCI Functions

• OCI Data Catalog (part of a data product governance framework)

• Autonomous Data Warehouse (potential source of data products)

• IAM with Identity Domains (for identity and access management)

• Oracle Machine Learning (potential way to offer data products derived from data)

Acknowledgments

Author: Gareth Smith