Connect to Azure Data Lake Storage

Learn to create a connection to Azure Data Lake Storage to use as a target with OCI GoldenGate.

Before you begin

Before you create the connection, ensure that you review how OCI GoldenGate connects to your source and targets.

Create the connection

To create an Azure Data Lake Storage connection
  1. From the OCI GoldenGate Overview page, click Connections.
    You can also click Create Connection under the Get started section and skip to step 3.
  2. On the Connections page, click Create Connection.
  3. In the Create Connection panel, complete the General Information fields as follows:
    1. For Name, enter a name for the connection.
    2. (Optional) For Description, enter a description that helps you distinguish this connection from others.
    3. For Compartment, select the compartment in which to create the connection.
    4. From the Type dropdown, under Big data, select Azure Data Lake Storage.
    5. (Optional) Click Show advanced options to manage keys or add tags.
      1. Under Security, select one of the following:
        • Select Use Oracle-managed encryption key to leave all encryption key management to Oracle.
        • Select Use customer-managed encryption key to select a specific encryption key stored in your OCI Vault to encrypt your connection credentials.
      2. Under Tags, add tags to organize your resources.
  4. Click Next.
  5. Complete the Connection Details fields as follows:
    1. For Account name, enter the Azure Cloud Storage Account Name.
    2. (Optional) For Endpoint, enter the Azure Storage service endpoint.

      Note:

      If you enter a private IP, then OCI GoldenGate rewrites the private IP in the format, ip-10-0-0-0.ociggsvc.oracle.vcn.com.
    3. For Authentication type select from the following:
      • Shared key:
      • Shared access signature:
        • SAS token: Provide SAS token. Ensure that the SAS token is created on the account level. See Create an account SAS.
      • Azure Active Directory:

        Note:

        Before you configure the Azure Active Directory authentication type, ensure that you register an application in Azure AD App Registrations and assign the appropriate roles, for example "Storage BLOB Data Owner". See Use the portal to create an Azure AD application and service principal that can access resources.
        • Azure tenant id, located in Azure Active Directory/ App Registrations, select the application, and enter the Tenant ID.
        • Client id, located in Azure Active Directory/ App Registrations and select the application.
        • Client secret: Provide Azure Client Secret (Value) for the selected application id.
    4. For Network connectivity, select a Traffic routing method:
      • Shared endpoint, to share an endpoint with the assigned deployment. You must allow connectivity from the deployment's ingress IP.
      • Dedicated endpoint, for network traffic through a dedicated endpoint in the assigned subnet in your VCN. You must allow connectivity from this connection's ingress IPs.
  6. Click Create.
After the connection is created, it appears in the Connections list, where you can select it to view its details. Ensure that you assign the connection to a deployment to use it as a target in a replication.

Related Topics

Troubleshoot connection issues

Most Azure Data Lake Storage connection issues happen because of Azure Data Lake Storage private endpoint configurations.

The following are common connectivity related error messages that you could encounter in the Replicat report file:

  • =ERROR 2023-08-04 07:23:08.000008 [main] - Exception during initialisation of Azure blob service client for account[ociggtest].
com.azure.storage.blob.models.BlobStorageException: Status code 400, "{"error":{"code":"InvalidUri","message":"The request URI is invalid.
  • =ERROR 2023-08-01 20:23:24.000861 [main] - The Event Handler Framework failed to initialise.
  • =ERROR 2023-08-04 08:13:30.000477 [main] - Exception during initialization of Azure blob service client for account[ociggtest].
com.azure.storage.blob.models.BlobStorageException:Status code 403, "<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationFailure</Code><Message>This request is not authorized to perform this operation.

If you're using Azure Data Lake Storage private endpoints and having issues with connection and/or replication, ensure that you: