Connect to OCI Streaming

Learn to create a connection to OCI Streaming to use as a source or target with OCI GoldenGate

Before you begin

Ensure that you review how OCI GoldenGate connects to your source and targets.
Ensure that you configure the required policies to enable secure Vault and Secrets access, such as use secrets, use vaults, and read secret-bundles. For more information, see Minimum recommended policies.
Obtain the Stream Pool username:
1. From the Oracle Cloud console navigation menu, select Streaming, and then Stream Pools.
2. On the Stream Pools page, select your pool to view its details.
3. On the Stream Pool details page, under Resources, click Kafka Connection Settings.
4. Copy the username for SASL Connection Strings.
  
  Note
  
  If you're using Oracle Identity Cloud service, you must add OracleIdentityCloudService to your user name. For example, <tenancy-name>/OracleIdentityCloudService/<username>/<streampool-ocid>
Create an Auth token:
1. In the Oracle Cloud console global header, click Profile, and then select User settings.
2. On the User Details page, under Resources, click Auth Tokens, and then click Generate Token.
3. In the Generate Token dialog, enter a description, and then click Generate Token.
4. Copy the auth token from the dialog to a secure location from where you can retrieve it later, and then click Close.

The Stream Pool username and Auth token will be entered for the Stream connection's username and password.

Create the source connection

To create a source OCI Streaming connection:

From the OCI GoldenGate Overview page, click Connections.
You can also click Create Connection under the Get started section and skip to step 3.
On the Connections page, click Create Connection.
On the Create Connection page, complete the fields as follows:
1. For Name, enter a name for the connection.
2. (Optional) For Description, enter a description that helps you distinguish this connection from others.
3. For Compartment, select the compartment in which to create the connection.
4. From the Type dropdown, select OCI Streaming.
5. For Stream pool details, you can choose:
  - Select a stream pool
    1. Select a stream pool from the dropdown, or click Change compartment to select one in a different compartment.
  - Enter stream pool information to manually enter the stream pool information.
    1. Enter the Bootstrap server's Host and Port.
      Note
      
      If you enter a private IP, then OCI GoldenGate rewrites the private IP in the format, ip-10-0-0-0.ociggsvc.oracle.vcn.com.
6. For Username, enter the Stream Pool username copied from the SASL Connection Settings in the prerequisite steps above.
7. Select the Database user password secret. If located in a different compartment, use the dropdown to change compartments.
  Note
  - Secrets are credentials such as passwords, certificates, SSH keys, or authentication tokens that you use with OCI services. To create a secret, see Creating a secret. Ensure that you:
    
    Select Manual secret generation.
    
    Paste the credentials into Secret contents.
  - If you prefer not to use password secrets, ensure that you deselect Use secrets in vault in the Security section under Advanced Options, located at the bottom of this form.
8. Expand Show advanced options. You can configure the following options:
  - Security
    - Deselect Use vault secrets you prefer not to use password secrets for this connection. If not selected:
      
      Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
      
      Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
  - Network connectivity
    - Shared endpoint, to share an endpoint with the assigned deployment. You must allow connectivity from the deployment's ingress IP.
    - Dedicated endpoint, for network traffic through a dedicated endpoint in the assigned subnet in your VCN. You must allow connectivity from this connection's ingress IPs.
      Note
      
      If a dedicated connection remains unassigned for seven days, then the service converts it to a shared connection.
      
      Learn more about Oracle GoldenGate connectivity.
  - Settings
    - Deselect Use vault secrets you prefer not to use password secrets for this connection. If not selected:
      
      Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
      
      Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
    To capture from OCI Streaming, create a consumer.properties file with one of the following deserializers or converters, and then drag and drop the file to Consumer properties:
    - Kafka Consumer properties for JSON deserializer:
      key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
    - Kafka Consumer properties for JSON converter:
      key.converter=org.apache.kafka.connect.json.JsonConverter value.converter=org.apache.kafka.connect.json.JsonConverter
    - Kafka Consumer properties for Avro converter:
      key.converter=io.confluent.connect.avro.AvroConverter value.converter=io.confluent.connect.avro.AvroConverter
  - Tags: Add tags to organize your resources.
Click Create.

After the connection is created, it appears in the Connections list. Ensure that you assign the connection to a deployment to use it as a source or target in a replication.

Create the target connection

To create a target OCI Streaming connection:

From the OCI GoldenGate Overview page, click Connections.
You can also click Create Connection under the Get started section and skip to step 3.
On the Connections page, click Create Connection.
On the Create Connection page, complete the fields as follows:
1. For Name, enter a name for the connection.
2. (Optional) For Description, enter a description that helps you distinguish this connection from others.
3. For Compartment, select the compartment in which to create the connection.
4. From the Type dropdown, select OCI Streaming.
5. For Stream pool details, you can choose:
  - Select a stream pool
    1. Select a stream pool from the dropdown, or click Change compartment to select one in a different compartment.
  - Enter stream pool information to manually enter the stream pool information.
    1. Enter the Bootstrap server's Host and Port.
      Note
      
      If you enter a private IP, then OCI GoldenGate rewrites the private IP in the format, ip-10-0-0-0.ociggsvc.oracle.vcn.com.
6. For Username, enter the Stream Pool username copied from the SASL Connection Settings in the prerequisite steps above.
7. Select the Database user password secret. If located in a different compartment, use the dropdown to change compartments.
  Note
  - Secrets are credentials such as passwords, certificates, SSH keys, or authentication tokens that you use with OCI services. To create a secret, see Creating a secret. Ensure that you:
    
    Select Manual secret generation.
    
    Paste the credentials into Secret contents.
  - If you prefer not to use password secrets, ensure that you deselect Use secrets in vault in the Security section under Advanced Options, located at the bottom of this form.
8. Expand Show advanced options. You can configure the following options:
  - Security
    - Deselect Use vault secrets you prefer not to use password secrets for this connection. If not selected:
      
      Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
      
      Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
  - Network connectivity
    - Shared endpoint, to share an endpoint with the assigned deployment. You must allow connectivity from the deployment's ingress IP.
    - Dedicated endpoint, for network traffic through a dedicated endpoint in the assigned subnet in your VCN. You must allow connectivity from this connection's ingress IPs.
      Note
      
      If a dedicated connection remains unassigned for seven days, then the service converts it to a shared connection.
      
      Learn more about Oracle GoldenGate connectivity.
  - Settings
    - Deselect Use vault secrets you prefer not to use password secrets for this connection. If not selected:
      
      Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
      
      Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
    To use Snappy compression in Kafka replication, drag and drop or select Producer properties, and change replication settings as discussed in Using Compression OCI GoldenGate (Confluent) Kafka Replication.
  - Tags: Add tags to organize your resources.
Click Create.

After the connection is created, it appears in the Connections list. Ensure that you assign the connection to a deployment to use it as a source or target in a replication.

Oracle Cloud Infrastructure Documentation