Connect to Amazon S3

Learn to create a connection to Amazon S3 to use as a target for OCI GoldenGate.

Related Topics

Before you begin

Before you create the connection, ensure that you review how OCI GoldenGate connects to your source and targets.

Create the connection

To create an Amazon S3 connection:
  1. From the OCI GoldenGate Overview page, click Connections.
    You can also click Create Connection under the Get started section and skip to step 3.
  2. On the Connections page, click Create Connection.
  3. On the Create Connection page, complete the fields as follows:
    1. For Name, enter a name for the connection.
    2. (Optional) For Description, enter a description that helps you distinguish this connection from others.
    3. For Compartment, select the compartment in which to create the connection.
    4. For Type, select Amazon S3.
    5. For Access key id, enter the Amazon S3 access key ID.

      Note:

      The Access key id must contain only alphanumeric characters and underscores, and be 16 to 128 characters in length.
    6. Select the Secret access key secret. If located in a different compartment, use the dropdown to change compartments.

      Note:

      If you prefer not to use password secrets, ensure that you deselect Use secrets in vault in the Settings tab under Advanced Options, located at the bottom of this form.
    7. For Region, enter the name of the region where the bucket resides.
    8. For Endpoint, enter the Amazon VPC endpoint for S3.
    9. Expand Show advanced options. You can configure the following options:
      • Security
        • Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
        • Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
      • Network connectivity
        • Shared endpoint, to share an endpoint with the assigned deployment. You must allow connectivity from the deployment's ingress IP.
        • Dedicated endpoint, for network traffic through a dedicated endpoint in the assigned subnet in your VCN. You must allow connectivity from this connection's ingress IPs.

          Note:

          • If a dedicated connection remains unassigned for seven days, then the service converts it to a shared connection.
          • Learn more about Oracle GoldenGate connectivity.
      • Settings
        • Deselect Use vault secrets you prefer not to use password secrets for this connection.
      • Tags: Add tags to organize your resources.
  4. Click Create.
After the connection is created, it appears in the Connections list. Ensure that you assign the connection to a deployment to use it as a target in a replication.

Known issues

Java.net.UnknownHostException errors for Amazon S3 and Google Cloud Storage connections with Shared endpoints encounter

Workaround: Edit the connection and change the Traffic routing method to Dedicated, or select Dedicated when you create Amazon S3 and Google Cloud Storage connections.

Issue with Amazon S3 connections in OCI GoldenGate

If you encounter the following error when using Amazon S3 connections, then open a support ticket, share the details and error message. 

ERROR 2024-03-04 11:42:31.000505 [TaskEngine_2(FileFinalizeTask)] - Verify S3 bucket
      [ggstest] failed.com.amazonaws.SdkClientException: Unable to execute HTTP request: s3.us-east-2.amazonaws.com

You can then use the following steps as a temporary workaround.

Workaround:

  1. Connect to Cloud Shell.
  2. Create a new Amazon S3 connection using the following CLI sample:
    oci goldengate connection create-amazon-s3-connection --routing-method SHARED_SERVICE_ENDPOINT --display-name <connection_name> --compartment-id <compartment_ocid> --technology-type AMAZON_S3 --access-key-id <aws_access_key> --secret-access-key <aws_secret>
  3. Assign the connection to your deployment.
  4. Add and run a Replicat for Amazon S3.

Alternatively, if you prefer not to use public access, configure your Amazon S3 connection's buckets to use AWS VPC Endpoints. For OCI GoldenGate to access your S3 bucket using VPC Endpoints, you must also configure an IPsec VPN between your VCN and the AWS VPC.