Note:
- This tutorial requires access to Oracle Cloud. To sign up for a free account, see Get started with Oracle Cloud Infrastructure Free Tier.
- It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.
Consume a private OCI Streaming topic behind an API Gateway using OCI REST API call
Introduction
The Oracle Cloud Infrastructure (OCI) Streaming service provides a fully managed, scalable, and durable solution for ingesting and consuming high-volume data streams in real-time. Use OCI Streaming for any use case in which data is produced and processed continually and sequentially in a publish-subscribe messaging model.
In this scenario, we need to expose and consume a private OCI Streaming topic using a public API Gateway to control and secure the access.
Objective
This tutorial demonstrates how to consume OCI Streaming topic using API Gateway REST API calls.
Prerequisites
- Oracle account with admin level access permissions
- A compartment to create your resources
- Local OCI-CLI setup, if you need help, see Installing the CLI tutorial
- VCN with two subnets (Private & Public), see Creating Network tutorial
- Make sure your subnets have proper Ingress Rules for Service Gateway and port 443 for both private and public subnets.
- OCI Streaming topic Creating OCI Stream tutorial
- Local Python version 3 installed for testing
Task 1: Get details from your OCI Streaming
-
Go to your stream and note the OCID, click Analytics & AI under Messaging, click Streaming and select your stream.
-
Note the OCID and Message Endpoint, you’re going to need it for your API Gateway.
Task 2: Create a public API Gateway
-
Click Developer Services, under API Management, click Gateways and then click Create Gateway.
Task 3: Create a deployment for your API Gateway
-
Select the API Gateway created in Task 2, click Deployments, and then click Create Deployment.
-
Now, create the prefix by replacing your_ocid_for_stream with your stream OCID below, fill the name as publicTopic.
/20180418/streams/your_ocid_for_stream
-
Click Next.
-
Click Next.
-
Creating Route 1 for groupCursors: Create the URL using the endpoint and stream OCID you’ve gathered in Task 2 of this tutorial and build your URL.
<streaming_endpoint>/20180418/streams/<stream_ocid>/groupCursorsNote: To consume the stream, the OCI REST API for Streaming uses two endpoints groupCursors and messages, you need to create those two routes. Now provide the needed information such as Path, Methods, Backend type as below:
-
Click +Another route and add Route 2 for messages: Using the same URL but with the different endpoint at the end as /messages.
-
Go to your publicTopic and note the API Gateway endpoint.
Task 4: Generate test messages on your streaming
-
Go to your Streaming page, and click Produce Test Message.
-
These messages will be consumed in Task 5.
Task 5: Consume OCI Streaming via REST API behind the API Gateway
-
Download the sample Python consume code consume_rest_api_oci_lab.py.
-
Change the values for your API Gateway endpoint and stream OCID as indicated with the values you gathered earlier in this tutorial.
-
Run your Python code to verify it’s working.
python3 consume_rest_api_oci_lab.py
Related Links
Acknowledgments
- Author - Joao Tarla (Oracle LAD A-Team Solution Engineer)
More Learning Resources
Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.
For product documentation, visit Oracle Help Center.
Consume a private OCI Streaming topic behind an API Gateway using OCI REST API call
F80887-01
April 2023
Copyright © 2023, Oracle and/or its affiliates.