Creating Streams

Before publishing messages to a stream, or consuming messages from a stream, you must first create a stream. The following sections describe how to create a stream:

When creating a stream, consider the expected stream throughput, message retention period, partition key strategy, and how your stream will be consumed. Most configuration values cannot be changed after the stream has been created. For example, after a stream is created, you can't change the message retention time or number of partitions.

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  to work in.

For administrators: The policy in Let streaming users manage streams lets the specified group do everything with streaming and related Streaming service resources.

If you're new to policies, see Getting Started with Policies and Common Policies. If you want to dig deeper into writing policies for the Streaming service, see Details for the Streaming service in the IAM policy reference and Accessing Streaming Resources Across Tenancies.

Using the Console

  1. Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.

    A list of existing streams is displayed.

  2. Click Create Stream at the top of the list.
  3. Stream Name: Required. Specify a friendly name for the stream. It does not have to be unique within the compartment, but it must be unique to the stream pool. The stream name cannot be changed. Avoid entering confidential information.
  4. Compartment: Choose the compartment in which the stream will be created. To change the compartment, select a different compartment from the drop-down list.
  5. Stream Pool: Choose the stream pool that will contain your stream.
    1. If your chosen compartment has an existing stream pool, you can select it from the drop-down list or click Create new stream pool and configure the stream pool manually.
    2. If no stream pool exists in the chosen compartment, select Auto-create a default stream pool or click Create a new stream pool and configure the stream pool manually.
  6. In the Define Stream Settings panel:
    1. Retention (in Hours): Enter the number of hours (from 24 to 168) to retain messages. The default value is 24.
    2. Number of Partitions: Enter the number of partitions for the stream. The maximum number is based on the limits for your tenancy.

      Tip

      The maximum Total Write Rate and Total Read Rate of your stream are displayed as you adjust the number of partitions.
  7. Click Show Advanced Options to optionally define TagsIf you have permissions to create a resource, then you also have permissions to apply free-form tags to that resource. To apply a defined tag, you must have permissions to use the tag namespace. For more information about tagging, see Resource Tags. If you are not sure whether to apply tags, skip this option (you can apply tags later) or ask your administrator.
  8. Click Create.

Using the Command Line Interface (CLI)

For information about using the CLI, see Command Line Interface (CLI). For a complete list of flags and options available for CLI commands, see the Command Line Reference.

Note

The examples in this section use the full syntax for all parameters, for example --compartment-id. For some parameters, there are shortened versions that you can use instead, like -c. See the CLI online help for instances of a shortened parameter associated with a command.

You can create a stream in a compartment or a stream pool. The --compartment-id and --stream-pool-id parameters cannot be specified at the same time.

oci streaming admin stream create --name <stream_name> --partitions <number_of_partitions> --compartment-id <compartment_OCID>
oci streaming admin stream create --name <stream_name> --partitions <number_of_partitions> --stream-pool-id <stream_pool_OCID>

For example:

oci streaming admin stream create --name MyStream --partitions 5 --compartment-id ocid1.tenancy.oc1..exampleuniqueID
{
  "data": {
    "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
    "defined-tags": {},
    "freeform-tags": {},
    "id": "ocid1.stream.oc1.phx.exampleuniqueID",
    "lifecycle-state": "CREATING",
    "lifecycle-state-details": null,
    "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com",
    "name": "MyStream",
    "partitions": 5,
    "retention-in-hours": 24,
    "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID",
    "time-created": "2020-11-02T19:12:22.385000+00:00"
  },
  "etag": "\"d72d8103-f1ae-442a-822d-10f86cd097c5-25e61a9b-cc08-4fad-9908-40c9636d31d8\""
}

Using OCI SDKs

To create a stream, use the createStream method of StreamAdminClient.

See the Developer Guide to Streaming for detailed SDK examples.

Using Resource Manager and Terraform

Resource Manager is an Oracle Cloud Infrastructure (OCI) service that allows you to automate the process of provisioning your OCI resources. Using Terraform, Resource Manager helps you install, configure, and manage resources through the "infrastructure-as-code" model.

A Terraform configuration codifies your infrastructure in declarative configuration files. The configuration defines the resources you intend to provision, variables, and specific instructions for provisioning the resources

You can use Resource Manager or the Terraform CLI with the OCI Terraform provider to provision Streaming resources like streams and stream pools.

Stream Resource

You can use the oci_streaming_stream resource in Terraform configurations to create a stream in your compartment.

For example:

resource "oci_streaming_stream" "stream" {
  compartment_id     = var.compartment_ocid
  name               = "<stream_name>"
  partitions         = "<number_of_partitions>"
  retention_in_hours = "<retention_in_hours>"
}

For more information about writing configurations for use with Resource Manager, see Terraform Configurations for Resource Manager and Terraform Configuration.