Oracle Cloud Infrastructure Documentation

Connect to OCI Streaming

Learn to create a connection to OCI Streaming to use as a source or target with OCI GoldenGate

Before you begin

  1. Ensure that you review how OCI GoldenGate connects to your source and targets.
  2. Obtain the Stream Pool username:
    1. From the Oracle Cloud console navigation menu, select Streaming, and then Stream Pools.
    2. On the Stream Pools page, select your pool to view its details.
    3. On the Stream Pool details page, under Resources, click Kafka Connection Settings.
    4. Copy the username for SASL Connection Strings.
      Note

      If you're using Oracle Identity Cloud service, you must add OracleIdentityCloudService to your user name. For example, <tenancy-name>/OracleIdentityCloudService/<username>/<streampool-ocid>
  3. Create an Auth token:
    1. In the Oracle Cloud console global header, click Profile, and then select User settings.
    2. On the User Details page, under Resources, click Auth Tokens, and then click Generate Token.
    3. In the Generate Token dialog, enter a description, and then click Generate Token.
    4. Copy the auth token from the dialog to a secure location from where you can retrieve it later, and then click Close.

The Stream Pool username and Auth token will be entered for the Stream connection's username and password.

Create the source connection

To create a source OCI Streaming connection:
  1. From the OCI GoldenGate Overview page, click Connections.
    You can also click Create Connection under the Get started section and skip to step 3.
  2. On the Connections page, click Create Connection.
  3. In the Create Connection panel, complete the General Information fields as follows:
    1. For Name, enter a name for the connection.
    2. (Optional) For Description, enter a description that helps you distinguish this connection from others.
    3. For Compartment, select the compartment in which to create the connection.
    4. From the Type dropdown, under Big data, select OCI Streaming.
    5. (Optional) Click Show advanced options to manage keys or add tags.
      1. Under Security, select one of the following:
        • Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
        • Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
      2. Under Tags, add tags to organize your resources.
  4. Click Next.
  5. Complete the Connection Details fields as follows:
    1. Choose Select a stream pool to select an existing stream pool in the compartment. Click Change Compartment to select a stream pool in a different compartment.
    2. Choose Enter stream pool information to manually enter bootstrap server connection details:
      1. Under Bootstrap servers, select a Traffic routing method:
        • Shared endpoint, to share an endpoint with the assigned deployment. You must allow connectivity from the deployment's ingress IP.
        • Dedicated endpoint, for network traffic through a dedicated endpoint in the assigned subnet in your VCN. You must allow connectivity from this connection's ingress IPs.
      2. Enter the Bootstrap server's Host and Port.
        Note

        If you enter a private IP, then OCI GoldenGate rewrites the private IP in the format, ip-10-0-0-0.ociggsvc.oracle.vcn.com.
    3. For Username, enter the Stream Pool username copied from the SASL Connection Settings in the prerequisite steps above.
    4. For Password, enter the Auth token copied in prerequisite steps above.
    5. Click Show advanced options.
      To capture from OCI Streaming, create a consumer.properties file with one of the following deserializers or converters, and then drag and drop the file to Consumer properties:
      • Kafka Consumer properties for JSON deserializer:
        key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
      • Kafka Consumer properties for JSON converter:
        key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
      • Kafka Consumer properties for Avro converter:
        key.converter=io.confluent.connect.avro.AvroConverter
value.converter=io.confluent.connect.avro.AvroConverter
  6. Click Create.
After the connection is created, it appears in the Connections list where you can select it to view its details. Ensure that you assign the connection to a deployment to use it as a source or target in a replication.

Create the target connection

To create a target OCI Streaming connection:
  1. From the OCI GoldenGate Overview page, click Connections.
    You can also click Create Connection under the Get started section and skip to step 3.
  2. On the Connections page, click Create Connection.
  3. In the Create Connection panel, complete the General Information fields as follows:
    1. For Name, enter a name for the connection.
    2. (Optional) For Description, enter a description that helps you distinguish this connection from others.
    3. For Compartment, select the compartment in which to create the connection.
    4. From the Type dropdown, under Big data, select OCI Streaming.
    5. (Optional) Click Show advanced options to manage keys or add tags.
      1. Under Security, select one of the following:
        • Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
        • Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
      2. Under Tags, add tags to organize your resources.
  4. Click Next.
  5. Complete the Connection Details fields as follows:
    1. Choose Select a stream pool to select an existing stream pool in the compartment. Click Change Compartment to select a stream pool in a different compartment.
    2. Choose Enter stream pool information to manually enter bootstrap server connection details:
      1. Under Bootstrap servers, select a Traffic routing method:
        • Shared endpoint, to share an endpoint with the assigned deployment. You must allow connectivity from the deployment's ingress IP.
        • Dedicated endpoint, for network traffic through a dedicated endpoint in the assigned subnet in your VCN. You must allow connectivity from this connection's ingress IPs.
      2. Enter the Bootstrap server's Host and Port.
        Note

        If you enter a private IP, then OCI GoldenGate rewrites the private IP in the format, ip-10-0-0-0.ociggsvc.oracle.vcn.com.
    3. For Username, enter the Stream Pool username copied from the SASL Connection Settings in the prerequisite steps above.
    4. For Password, enter the Auth token copied in prerequisite steps above.
    5. Click Show advanced options.
      To use Snappy compression in Kafka replication, drag and drop or select Producer properties, and change replication settings as discussed in Using Compression OCI GoldenGate (Confluent) Kafka Replication.
  6. Click Create.
After the connection is created, it appears in the Connections list where you can select it to view its details. Ensure that you assign the connection to a deployment to use it as a source or target in a replication.