10.4.2 Apache Iceberg

• About Apache Iceberg Connections
  
• Before You Begin
  
• Create and Validate an Apache Iceberg Connection
  
• Use an Apache Iceberg Connection in Replicat
  

10.4.2.1 About Apache Iceberg Connections

An Apache Iceberg connection stores the information that Oracle GoldenGate DAA uses to connect to an Apache Iceberg catalog and its associated storage. After the connection is created and validated, you can reuse it when configuring Replicat processes that write transactional data to Apache Iceberg tables.

The connection supports multiple catalog implementations and storage providers. It stores the catalog configuration, storage configuration, authentication information, and optional runtime settings independently of the Replicat configuration so that multiple Replicat processes can reuse the same validated connection.

10.4.2.2 Before You Begin

Before creating an Apache Iceberg connection, ensure that you have the following:

• An Oracle GoldenGate DAA deployment.
• Network connectivity between the Oracle GoldenGate DAA deployment and the target Iceberg catalog.
• The Iceberg catalog type used by your deployment.
• The storage information required by the selected catalog.
• Authentication credentials required by the selected catalog or storage provider.
• The required client libraries and any additional runtime dependencies, if they are not already available in the deployment.
  • To download the required dependencies, use the Oracle GoldenGate DAA Dependency Downloader for the storage technology used by your Iceberg deployment. For example:

    Google Cloud Storage:

    {GGforDAA install}/DependencyDownloader/gcs.sh 2.x

    Dependency: https://search.maven.org/artifact/com.google.cloud/google-cloud-storage

    Amazon S3:

    {GGforDAA install}/DependencyDownloader/s3.sh 2.x

    Dependency: https://mvnrepository.com/artifact/software.amazon.awssdk/s3

    Azure Data Lake Storage:

    {GGforDAA install}/DependencyDownloader/azure_blob_storage.sh 12.25.3

    Dependency: https://search.maven.org/artifact/com.azure/azure-storage-blob

  • After downloading the required dependencies, configure the gg.classpath property in the Java Adapter properties file to include the downloaded libraries before starting the Replicat process.
• Configure common runtime settings, such as the runtime classpath, if additional client libraries are required.
• Proxy information if the Oracle GoldenGate deployment accesses the catalog or storage through an HTTP proxy.

10.4.2.3 Create and Validate an Apache Iceberg Connection

Use the Add Connection page to create an Apache Iceberg connection.

Procedure

1. In the navigation menu, select DB Connections.
2. Click Add Connection.
3. In Connection Alias, enter a unique name for the connection.
4. From Technology Type, select Apache Iceberg.
5. Configure the Catalog Settings:
   • Catalog Type – Specifies the Iceberg catalog implementation. Supported catalog types include Glue, Hadoop, Nessie, Polaris, and REST. This field is required.
6. Configure the storage settings that correspond to the selected catalog:
   • Amazon S3 Storage Settings
     • Storage Type – Select Amazon S3.
     • Access Key ID – Specifies the AWS access key ID used to access the S3 bucket.
     • Secret Access Key – Specifies the AWS secret access key associated with the access key ID. This field is treated as sensitive information.
     • Region – Specifies the AWS Region where the S3 bucket is located.
     • Endpoint – Specifies the S3 endpoint URL, if required.
     • Scheme Type – Specifies the S3 scheme used by the Iceberg catalog. Supported values are S3 and S3A.
   • Google Cloud Storage Settings
     • Storage Type – Select Google Cloud Storage.
     • Project ID – Specifies the Google Cloud project that contains the target bucket.
     • Bucket – Specifies the target Cloud Storage bucket.
     • Service Account Key – Upload the Google Cloud service account key used to authenticate with Cloud Storage. This property is deprecated and will be replaced by a secret identifier.
   • Azure Data Lake Storage Settings
     • Storage Type – Select Azure Data Lake Storage.
     • Storage Account – Specifies the Azure Storage account.
     • Container – Specifies the Azure Blob Storage container that stores the Iceberg tables.
     • Client ID – Specifies the Azure client application ID.
     • Client Secret – Specifies the Azure client secret. This field is treated as sensitive information.
     • Tenant ID – Specifies the Azure tenant ID.
     • Endpoint – Specifies the Azure Storage service endpoint.
   • Local Storage Settings
     • Storage Type – Select Local Storage.
     • Storage Location – Specifies the local file system location where the Iceberg tables are stored.

   Note:

   The available storage settings depend on the selected catalog type. Some storage providers are supported only for specific catalog implementations
7. Configure Common Settings, if additional runtime configuration is required:

   Classpath: (Required) Specify the full path to client libraries are required by the replicat type. You can use Dependency Downloader to download client libraries.

8. (Optional) Configure Proxy Settings if the Oracle GoldenGate DAA deployment accesses the Iceberg catalog or storage through an HTTP proxy:
   • Proxy URL – Specifies the URL or host name of the proxy server used to access Apache Iceberg.
   • Proxy Port – Specifies the port number of the proxy server.
   • Proxy Username – Specifies the user name for proxy authentication, if required.
   • Proxy Password – Specifies the password for proxy authentication. This field is treated as sensitive information.
   • Truststore – Specifies the Java truststore file that contains the certificate authority (CA) certificates used to validate the proxy server certificate, when required.
   • Truststore Password – Specifies the password associated with the truststore.
   • Keystore – Specifies the Java keystore file containing the client certificate and private key used for client authentication, when required.
   • Keystore Password – Specifies the password associated with the keystore.
   • SSL Key Password – Specifies the password protecting the private key stored in the keystore when it differs from the keystore password.

   Note:

   If the Oracle GoldenGate deployment has direct network connectivity to the Iceberg catalog and storage, leave the Proxy Settings blank.
9. Click Submit.
10. On the DB Connections page, click DB Login to validate the connection.

Note:

A successful DB Login validates only the saved connection information. It does not validate the Replicat configuration or Apache Iceberg handler configuration. After successful validation, the connection is available for selection during Replicat creation.

10.4.2.4 Use an Apache Iceberg Connection in Replicat

After the Apache Iceberg connection has been successfully validated:

1. Create or edit a Replicat.
2. Select Apache Iceberg as the target technology.
3. Select the validated Apache Iceberg connection.
4. Complete the remaining Replicat configuration.
5. Create and start the Replicat.