Table of Contents

• Title and Copyright Information
• Preface
  • Audience
  • Documentation Accessibility
  • Conventions
  • Related Information
• 1 Overview
  • 1.1 Understanding Oracle GoldenGate for Distributed Applications and Analytics
    • 1.1.1 Understanding Oracle GoldenGate for Distributed Applications and Analytics
      • 1.1.1.1 Oracle GoldenGate VAM Message Capture
        • 1.1.1.1.1 Message Capture Configuration Options
        • 1.1.1.1.2 Typical Configuration
      • 1.1.1.2 Oracle GoldenGate Java Delivery
      • 1.1.1.3 Delivery Configuration Options
      • 1.1.1.4 Adapter Integration Options
        • 1.1.1.4.1 Capturing Transactions to a Trail
        • 1.1.1.4.2 Applying Transactions from a Trail
      • 1.1.1.5  Using Oracle GoldenGate Java Adapter Properties
        • 1.1.1.5.1 Values in Property Files
        • 1.1.1.5.2 Location of Property Files
        • 1.1.1.5.3 Using Comments in the Property File
        • 1.1.1.5.4 Variables in Property Names
      • 1.1.1.6 Monitoring Performance
  • 1.2 What’s Supported in Oracle GoldenGate for Distributed Applications and Analytics
    • 1.2.1 Verifying Certification and System Requirements
    • 1.2.2 Understanding Handler Compatibility
    • 1.2.3 What are the Additional Support Considerations?
  • 1.3 Dependency Downloader
    • 1.3.1 Dependency Downloader Setup
    • 1.3.2 Running the Dependency Downloader Scripts
    • 1.3.3 Dependency Downloader Scripts
• 2 Get Started
  • 2.1 Getting Started with Oracle GoldenGate for Distributed Applications and Analytics
    • 2.1.1 Working With Deployments
    • 2.1.2 About Oracle GoldenGate Properties Files
      • 2.1.2.1 Parameter Files
    • 2.1.3 Using the Admin Client
    • 2.1.4 Controlling Oracle GoldenGate (Microservices Architecture) Processes
• 3 Prepare
  • 3.1 Preparing for Installation
    • 3.1.1 Downloading Oracle GoldenGate for Distributed Applications and Analytics
    • 3.1.2 Installation Overview
      • 3.1.2.1 Contents of the Installation ZIP File
      • 3.1.2.2 Using the Generic Build of Oracle GoldenGate
      • 3.1.2.3 Considerations for Using a Custom Build for a GoldenGate for Distributed Applications and Analytics Instance of Oracle GoldenGate
      • 3.1.2.4 Installing to a Non-Generic Instance of Oracle GoldenGate
    • 3.1.3 Directories and Variables in Microservices Architecture
    • 3.1.4 Setting up Environmental Variables
      • 3.1.4.1 Java on Linux/UNIX
      • 3.1.4.2 Java on Windows
• 4 Install
  • 4.1 Setting up Oracle GoldenGate for Distributed Applications and Analytics in a High Availability Environment
    • 4.1.1 Running GG for DAA from a Single Instance
    • 4.1.2 Running GG for DAA on a Cluster of Servers
    • 4.1.3 Shared Storage
  • 4.2 Installing Oracle GoldenGate for Distributed Applications and Analytics
    • 4.2.1 Installing Oracle GoldenGate MA for Distributed Applications and Analytics Using the UI
    • 4.2.2 Silent Installation
    • 4.2.3 Setting Up Secure or Non-Secure Deployments
      • 4.2.3.1 How to Add Secure or Non-Secure Deployments
      • 4.2.3.2 How to Remove a Deployment
        • 4.2.3.2.1 How to Remove a Deployment: GUI
        • 4.2.3.2.2 How to Remove a Deployment: Silent Mode
• 5 Upgrade
  • 5.1 Upgrading Oracle GoldenGate for Distributed Applications and Analytics
    • 5.1.1 Obtaining the Oracle GoldenGate Distribution
    • 5.1.2 Scope of Upgrade
      • 5.1.2.1 Replicat Upgrade Considerations
    • 5.1.3 Upgrading Oracle GoldenGate for Distributed Applications and Analytics – GUI Based
• 6 Configure
  • 6.1 Configuring Oracle GoldenGate for Distributed Applications and Analytics
    • 6.1.1 Running with Replicat
      • 6.1.1.1 Replicat Grouping
      • 6.1.1.2 About Replicat Checkpointing
      • 6.1.1.3 About Initial Load Support
      • 6.1.1.4 About the Unsupported Replicat Features
      • 6.1.1.5 How the Mapping Functionality Works
    • 6.1.2 About Schema Evolution and Metadata Change Events
    • 6.1.3 About Configuration Property CDATA[] Wrapping
    • 6.1.4 Using Regular Expression Search and Replace
      • 6.1.4.1 Using Schema Data Replace
      • 6.1.4.2 Using Content Data Replace
    • 6.1.5 Scaling Oracle GoldenGate for Distributed Applications and Analytics Delivery
    • 6.1.6 Configuring Cluster High Availability
    • 6.1.7 Using Identities in Oracle GoldenGate Credential Store
      • 6.1.7.1 Creating a Credential Store
      • 6.1.7.2 Adding Users to a Credential Store
      • 6.1.7.3 Configuring Properties to Access the Credential Store
  • 6.2 Logging
    • 6.2.1 About Replicat Process Logging
    • 6.2.2 About Java Layer Logging
  • 6.3 Configuring Logging
    • 6.3.1 Oracle GoldenGate Java Adapter Default Logging
      • 6.3.1.1 Default Logging Setup
      • 6.3.1.2 Log File Name
      • 6.3.1.3 Changing Logging Level
    • 6.3.2 Recommended Logging Settings
      • 6.3.2.1 Changing to the Recommended Logging Type
• 7 Quickstarts
  • 7.1 QuickStarts: Prerequisites
  • 7.2 Realtime Data Ingestion into Snowflake with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.2.1 Prerequisites for Internal Staging
    • 7.2.2 Install Dependency Files
    • 7.2.3 Create a Credential Store Entry
    • 7.2.4 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
  • 7.3 Realtime Parquet Ingestion into Google Cloud Storage with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.3.1 Prerequisites
    • 7.3.2 Install Dependency Files
    • 7.3.3 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
  • 7.4 Realtime Parquet Ingestion into AWS S3 buckets with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.4.1 Prerequisites
    • 7.4.2 Install Dependency Files
    • 7.4.3 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
  • 7.5 Realtime Parquet Ingestion into Azure Data Lake Storage with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.5.1 Prerequisites
    • 7.5.2 Install Dependency Files
    • 7.5.3 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
  • 7.6 Realtime Parquet Ingestion into OCI Object Storage with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.6.1 Prerequisites
    • 7.6.2 Install Dependency Files
    • 7.6.3 Configure Credentials for Oracle Cloud Infrastructure
    • 7.6.4 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
  • 7.7 Realtime Message Ingestion to OCI Streaming with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.7.1 Prerequisites
    • 7.7.2 Install Dependency Files
    • 7.7.3 Create Kafka Producer Properties
    • 7.7.4 Create a Replicat in GG for DAA
  • 7.8 Realtime Message Ingestion to Azure Event Hubs with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.8.1 Prerequisites
    • 7.8.2 Install Dependency Files
    • 7.8.3 Create a producer.properties for Azure Event Hubs
    • 7.8.4 Create a Replicat in Oracle GoldenGate for Distirbuted Applications and Analytics
  • 7.9 Realtime Data Ingestion into GCP BigQuery with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.9.1 Prerequisites for Google Cloud Platform BigQuery Stage and Merge
    • 7.9.2 Install Dependency Files
    • 7.9.3 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
  • 7.10 Realtime Message Ingestion to Google Pub/Sub with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.10.1 Prerequisites
    • 7.10.2 Install Dependency Files
    • 7.10.3 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
  • 7.11 Realtime Message Ingestion to Apache Kafka with Oracle GoldenGate for Distributed Applications and Analytics
    • 7.11.1 Prerequisites
    • 7.11.2 Install Dependency Files
    • 7.11.3 Create Kafka Producer Properties File
    • 7.11.4 Create a Replicat in Oracle GoldenGate for Distributed Applications and Analytics
• 8 Replicate Data
  • 8.1 Source
    • 8.1.1 Add Extract
    • 8.1.2 Amazon MSK
    • 8.1.3 Apache Cassandra
      • 8.1.3.1 Overview
      • 8.1.3.2 Setting Up Cassandra Extract Change Data Capture
        • 8.1.3.2.1 Setup SSH Connection to the Cassandra Nodes
        • 8.1.3.2.2 Data Types
        • 8.1.3.2.3 Cassandra Database Operations
        • 8.1.3.2.4 Set up Credential Store Entry to Detect Source Type
      • 8.1.3.3 Deduplication
      • 8.1.3.4 Topology Changes
      • 8.1.3.5 Data Availability in the CDC Logs
      • 8.1.3.6 Using Initial Load Extract
      • 8.1.3.7 Using Change Data Capture Extract
        • 8.1.3.7.1 Handling Schema Evolution
      • 8.1.3.8 Replicating to RDMBS Targets
      • 8.1.3.9 Partition Update or Insert of Static Columns
      • 8.1.3.10 Partition Delete
      • 8.1.3.11 Security and Authentication
        • 8.1.3.11.1 Configuring SSL
      • 8.1.3.12 Cleanup of CDC Commit Log Files
        • 8.1.3.12.1 Cassandra CDC Commit Log Purger
          • 8.1.3.12.1.1 How to Run the Purge Utility
          • 8.1.3.12.1.2 Sample config.properties for Local File System
          • 8.1.3.12.1.3 Argument cassCommitLogPurgerConfFile
          • 8.1.3.12.1.4 Argument purgeInterval
          • 8.1.3.12.1.5 Argument cassUnProcessedFilesPurgeInterval
      • 8.1.3.13 Multiple Extract Support
      • 8.1.3.14 CDC Configuration Reference
      • 8.1.3.15 Troubleshooting
      • 8.1.3.16 Cassandra Capture Client Dependencies
    • 8.1.4 Apache Kafka
      • 8.1.4.1 Overview
      • 8.1.4.2 Prerequisites
        • 8.1.4.2.1 Set up Credential Store Entry to Detect Source Type
      • 8.1.4.3 General Terms and Functionality of Kafka Capture
        • 8.1.4.3.1 Kafka Streams
        • 8.1.4.3.2 Kafka Message Order
        • 8.1.4.3.3 Kafka Message Timestamps
        • 8.1.4.3.4 Kafka Message Coordinates
        • 8.1.4.3.5 Start Extract Modes
          • 8.1.4.3.5.1 Start Earliest
          • 8.1.4.3.5.2 Start Timestamp
        • 8.1.4.3.6 General Configuration Overview
        • 8.1.4.3.7 OGGSOURCE parameter
        • 8.1.4.3.8 The Extract Parameter File
        • 8.1.4.3.9 Kafka Consumer Properties File
          • 8.1.4.3.9.1 Encrypt Kafka Producer Properties
      • 8.1.4.4 Generic Mutation Builder
      • 8.1.4.5 Kafka Connect Mutation Builder
        • 8.1.4.5.1 Functionality and Limitations of the Kafka Connect Mutation Builder
        • 8.1.4.5.2 Primary Key
        • 8.1.4.5.3 Kafka Message Key
        • 8.1.4.5.4 Kafka Connect Supported Types
        • 8.1.4.5.5 How to Enable the Kafka Connect Mutation Builder
      • 8.1.4.6 Example Configuration Files
        • 8.1.4.6.1 Example kc.prm file
        • 8.1.4.6.2 Example Kafka Consumer Properties File
    • 8.1.5 Azure Event Hubs
    • 8.1.6 Confluent Kafka
    • 8.1.7 DataStax
    • 8.1.8 Java Message Service (JMS)
      • 8.1.8.1 Prerequisites
        • 8.1.8.1.1 Set up Credential Store Entry to Detect Source Type
      • 8.1.8.2 Oracle GoldenGate Java Delivery
      • 8.1.8.3 Configuring Message Capture
        • 8.1.8.3.1 Oracle GoldenGate VAM Message Capture
          • 8.1.8.3.1.1 Message Capture Configuration Options
          • 8.1.8.3.1.2 Typical Configuration
        • 8.1.8.3.2 Configuring the VAM Extract
          • 8.1.8.3.2.1 Adding the Extract
          • 8.1.8.3.2.2 Configuring the Extract Parameters
          • 8.1.8.3.2.3 Configuring Message Capture
        • 8.1.8.3.3 Connecting and Retrieving the Messages
          • 8.1.8.3.3.1 Connecting to JMS
          • 8.1.8.3.3.2 Retrieving Messages
          • 8.1.8.3.3.3 Completing the Transaction
        • 8.1.8.3.4 Oracle GoldenGate VAM Message Capture
          • 8.1.8.3.4.1 Message Capture Configuration Options
          • 8.1.8.3.4.2 Typical Configuration
    • 8.1.9 MongoDB
      • 8.1.9.1 Overview
      • 8.1.9.2 Prerequisites to Setting up MongoDB
        • 8.1.9.2.1 Set up Credential Store Entry to Detect Source Type
      • 8.1.9.3 MongoDB Database Operations
      • 8.1.9.4 Using Extract Initial Load
      • 8.1.9.5 Using Change Data Capture Extract
      • 8.1.9.6 Positioning the Extract
      • 8.1.9.7 Security and Authentication
        • 8.1.9.7.1 SSL Configuration Setup
      • 8.1.9.8 MongoDB Bidirectional Replication
        • 8.1.9.8.1 Enabling Trandata
        • 8.1.9.8.2 Enabling MongoDB Bi-directional Replication
        • 8.1.9.8.3 Extracting from Target Replicat which is Bidirectionally Processed
        • 8.1.9.8.4 Troubleshooting
      • 8.1.9.9 Mongo DB Configuration Reference
      • 8.1.9.10 Columns in Trail File
      • 8.1.9.11 Update Operation Behavior
      • 8.1.9.12 Oplog Size Recommendations
      • 8.1.9.13 Troubleshooting
      • 8.1.9.14 MongoDB Capture Client Dependencies
        • 8.1.9.14.1 MongoDB Capture Client Dependencies: Reactive Streams Java Driver 4.4.1
    • 8.1.10 OCI Streaming
  • 8.2 Target
    • 8.2.1 Add Replicat
    • 8.2.2 Amazon Kinesis
      • 8.2.2.1 Overview
      • 8.2.2.2 Detailed Functionality
        • 8.2.2.2.1 Amazon Kinesis Java SDK
        • 8.2.2.2.2 Kinesis Streams Input Limits
      • 8.2.2.3 Setting Up and Running the Kinesis Streams Handler
        • 8.2.2.3.1 Set the Classpath in Kinesis Streams Handler
        • 8.2.2.3.2 Kinesis Streams Handler Configuration
        • 8.2.2.3.3 Using Templates to Resolve the Stream Name and Partition Name
        • 8.2.2.3.4 Resolving AWS Credentials
          • 8.2.2.3.4.1 AWS Kinesis Client Authentication
        • 8.2.2.3.5 Configuring the Proxy Server for Kinesis Streams Handler
        • 8.2.2.3.6 Configuring Security in Kinesis Streams Handler
      • 8.2.2.4 Kinesis Handler Performance Considerations
        • 8.2.2.4.1 Kinesis Streams Input Limitations
        • 8.2.2.4.2 Transaction Batching
        • 8.2.2.4.3 Deferring Flush at Transaction Commit
      • 8.2.2.5 Troubleshooting
        • 8.2.2.5.1 Java Classpath
        • 8.2.2.5.2 Kinesis Handler Connectivity Issues
        • 8.2.2.5.3 Logging
    • 8.2.3 Amazon MSK
    • 8.2.4 Amazon Redshift
      • 8.2.4.1 Detailed Functionality
      • 8.2.4.2 Operation Aggregation
        • 8.2.4.2.1 In-Memory Operation Aggregation
        • 8.2.4.2.2 Aggregation using SQL post loading data into the staging table
      • 8.2.4.3 Unsupported Operations and Limitations
      • 8.2.4.4 Uncompressed UPDATE records
      • 8.2.4.5 Error During the Data Load Proces
      • 8.2.4.6 Troubleshooting and Diagnostics
      • 8.2.4.7 Classpath
      • 8.2.4.8 Configuration
      • 8.2.4.9 INSERTALLRECORDS Support
      • 8.2.4.10 Redshift COPY SQL Authorization
      • 8.2.4.11 Co-ordinated Apply Support
    • 8.2.5 Amazon S3
      • 8.2.5.1 Overview
      • 8.2.5.2 Detailing Functionality
        • 8.2.5.2.1 Resolving AWS Credentials
          • 8.2.5.2.1.1 Amazon Web Services Simple Storage Service Client Authentication
        • 8.2.5.2.2 About the AWS S3 Buckets
        • 8.2.5.2.3 Troubleshooting
      • 8.2.5.3 Configuring the S3 Event Handler
    • 8.2.6 Apache Cassandra
      • 8.2.6.1 Overview
      • 8.2.6.2 Detailing the Functionality
        • 8.2.6.2.1 About the Cassandra Data Types
        • 8.2.6.2.2 About Catalog, Schema, Table, and Column Name Mapping
        • 8.2.6.2.3 About DDL Functionality
          • 8.2.6.2.3.1 About the Keyspaces
          • 8.2.6.2.3.2 About the Tables
          • 8.2.6.2.3.3 Adding Column Functionality
          • 8.2.6.2.3.4 Dropping Column Functionality
        • 8.2.6.2.4 How Operations are Processed
        • 8.2.6.2.5 About Compressed Updates vs. Full Image Updates
        • 8.2.6.2.6 About Primary Key Updates
      • 8.2.6.3 Setting Up and Running the Cassandra Handler
        • 8.2.6.3.1 Understanding the Cassandra Handler Configuration
        • 8.2.6.3.2 Review a Sample Configuration
        • 8.2.6.3.3 Configuring Security
      • 8.2.6.4 About Automated DDL Handling
        • 8.2.6.4.1 About the Table Check and Reconciliation Process
        • 8.2.6.4.2 Capturing New Change Data
      • 8.2.6.5 Performance Considerations
      • 8.2.6.6 Additional Considerations
      • 8.2.6.7 Troubleshooting
        • 8.2.6.7.1 Java Classpath
        • 8.2.6.7.2 Write Timeout Exception
        • 8.2.6.7.3 Datastax Driver Error
      • 8.2.6.8 Cassandra Handler Client Dependencies
        • 8.2.6.8.1 Cassandra Datastax Java Driver 4.12.0
        • 8.2.6.8.2 Cassandra Datastax Java Driver 4.9.0
    • 8.2.7 Apache HBase
      • 8.2.7.1 Overview
      • 8.2.7.2 Detailed Functionality
      • 8.2.7.3 Setting Up and Running the HBase Handler
        • 8.2.7.3.1 Classpath Configuration
        • 8.2.7.3.2 HBase Handler Configuration
        • 8.2.7.3.3 Sample Configuration
        • 8.2.7.3.4 Performance Considerations
      • 8.2.7.4 Security
      • 8.2.7.5 Metadata Change Events
      • 8.2.7.6 Additional Considerations
      • 8.2.7.7 Troubleshooting the HBase Handler
        • 8.2.7.7.1 Java Classpath
        • 8.2.7.7.2 HBase Connection Properties
        • 8.2.7.7.3 Logging of Handler Configuration
        • 8.2.7.7.4 HBase Handler Delete-Insert Problem
      • 8.2.7.8 HBase Handler Client Dependencies
        • 8.2.7.8.1 HBase 2.4.4
        • 8.2.7.8.2 HBase 2.3.3
        • 8.2.7.8.3 HBase 2.2.0
        • 8.2.7.8.4 HBase 2.1.5
        • 8.2.7.8.5 HBase 2.0.5
        • 8.2.7.8.6 HBase 1.4.10
        • 8.2.7.8.7 HBase 1.3.3
        • 8.2.7.8.8 HBase 1.2.5
        • 8.2.7.8.9 HBase 1.1.1
        • 8.2.7.8.10 HBase 1.0.1.1
    • 8.2.8 Apache HDFS
      • 8.2.8.1 Overview
      • 8.2.8.2 Writing into HDFS in SequenceFile Format
        • 8.2.8.2.1 Integrating with Hive
        • 8.2.8.2.2 Understanding the Data Format
      • 8.2.8.3 Setting Up and Running the HDFS Handler
        • 8.2.8.3.1 Classpath Configuration
        • 8.2.8.3.2 HDFS Handler Configuration
        • 8.2.8.3.3 Review a Sample Configuration
        • 8.2.8.3.4 Performance Considerations
        • 8.2.8.3.5 Security
      • 8.2.8.4 Writing in HDFS in Avro Object Container File Format
      • 8.2.8.5 Generating HDFS File Names Using Template Strings
      • 8.2.8.6 Metadata Change Events
      • 8.2.8.7 Partitioning
      • 8.2.8.8 HDFS Additional Considerations
      • 8.2.8.9 Best Practices
      • 8.2.8.10 Troubleshooting the HDFS Handler
        • 8.2.8.10.1 Java Classpath
        • 8.2.8.10.2 Java Boot Options
        • 8.2.8.10.3 HDFS Connection Properties
        • 8.2.8.10.4 Handler and Formatter Configuration
      • 8.2.8.11 HDFS Handler Client Dependencies
        • 8.2.8.11.1 Hadoop Client Dependencies
          • 8.2.8.11.1.1 HDFS 3.3.0
          • 8.2.8.11.1.2 HDFS 3.2.0
          • 8.2.8.11.1.3 HDFS 3.1.4
          • 8.2.8.11.1.4 HDFS 3.0.3
          • 8.2.8.11.1.5 HDFS 2.9.2
          • 8.2.8.11.1.6 HDFS 2.8.5
          • 8.2.8.11.1.7 HDFS 2.7.7
          • 8.2.8.11.1.8 HDFS 2.6.0
          • 8.2.8.11.1.9 HDFS 2.5.2
          • 8.2.8.11.1.10 HDFS 2.4.1
          • 8.2.8.11.1.11 HDFS 2.3.0
          • 8.2.8.11.1.12 HDFS 2.2.0
    • 8.2.9 Apache Kafka
      • 8.2.9.1 Apache Kafka
        • 8.2.9.1.1 Overview
        • 8.2.9.1.2 Detailed Functionality
        • 8.2.9.1.3 Setting Up and Running the Kafka Handler
          • 8.2.9.1.3.1 Classpath Configuration
          • 8.2.9.1.3.2 Kafka Handler Configuration
          • 8.2.9.1.3.3 Java Adapter Properties File
          • 8.2.9.1.3.4 Kafka Producer Configuration File
          • 8.2.9.1.3.5 Using Templates to Resolve the Topic Name and Message Key
          • 8.2.9.1.3.6 Kafka Configuring with Kerberos on a Hadoop Platform
          • 8.2.9.1.3.7 Kafka SSL Support
        • 8.2.9.1.4 Schema Propagation
        • 8.2.9.1.5 Performance Considerations
        • 8.2.9.1.6 About Security
        • 8.2.9.1.7 Metadata Change Events
        • 8.2.9.1.8 Snappy Considerations
        • 8.2.9.1.9 Kafka Interceptor Support
        • 8.2.9.1.10 Kafka Partition Selection
        • 8.2.9.1.11 Troubleshooting
          • 8.2.9.1.11.1 Verify the Kafka Setup
          • 8.2.9.1.11.2 Classpath Issues
          • 8.2.9.1.11.3 Invalid Kafka Version
          • 8.2.9.1.11.4 Kafka Producer Properties File Not Found
          • 8.2.9.1.11.5 Kafka Connection Problem
        • 8.2.9.1.12 Kafka Handler Client Dependencies
          • 8.2.9.1.12.1 Kafka 2.8.0
          • 8.2.9.1.12.2 Kafka 2.7.0
          • 8.2.9.1.12.3 Kafka 2.6.0
          • 8.2.9.1.12.4 Kafka 2.5.1
          • 8.2.9.1.12.5 Kafka 2.4.1
          • 8.2.9.1.12.6 Kafka 2.3.1
      • 8.2.9.2 Apache Kafka Connect Handler
        • 8.2.9.2.1 Overview
        • 8.2.9.2.2 Detailed Functionality
        • 8.2.9.2.3 Setting Up and Running the Kafka Connect Handler
          • 8.2.9.2.3.1 Kafka Connect Handler Configuration
          • 8.2.9.2.3.2 Using Templates to Resolve the Topic Name and Message Key
          • 8.2.9.2.3.3 Configuring Security in the Kafka Connect Handler
        • 8.2.9.2.4 Connecting to a Secure Schema Registry
        • 8.2.9.2.5 Kafka Connect Handler Performance Considerations
        • 8.2.9.2.6 Kafka Interceptor Support
        • 8.2.9.2.7 Kafka Partition Selection
        • 8.2.9.2.8 Troubleshooting the Kafka Connect Handler
          • 8.2.9.2.8.1 Java Classpath for Kafka Connect Handler
          • 8.2.9.2.8.2 Invalid Kafka Version
          • 8.2.9.2.8.3 Kafka Producer Properties File Not Found
          • 8.2.9.2.8.4 Kafka Connection Problem
        • 8.2.9.2.9 Kafka Connect Handler Client Dependencies
          • 8.2.9.2.9.1 Kafka 2.8.0
          • 8.2.9.2.9.2 Kafka 2.7.1
          • 8.2.9.2.9.3 Kafka 2.6.0
          • 8.2.9.2.9.4 Kafka 2.5.1
          • 8.2.9.2.9.5 Kafka 2.4.1
          • 8.2.9.2.9.6 Kafka 2.3.1
          • 8.2.9.2.9.7 Kafka 2.2.1
          • 8.2.9.2.9.8 Kafka 2.1.1
          • 8.2.9.2.9.9 Kafka 2.0.1
          • 8.2.9.2.9.10 Kafka 1.1.1
          • 8.2.9.2.9.11 Kafka 1.0.2
          • 8.2.9.2.9.12 Kafka  0.11.0.0
          • 8.2.9.2.9.13 Kafka 0.10.2.0
          • 8.2.9.2.9.14 Kafka 0.10.2.0
          • 8.2.9.2.9.15 Kafka 0.10.0.0
          • 8.2.9.2.9.16 Kafka 0.9.0.1
      • 8.2.9.3 Apache Kafka REST Proxy
        • 8.2.9.3.1 Overview
        • 8.2.9.3.2 Setting Up and Starting the Kafka REST Proxy Handler Services
          • 8.2.9.3.2.1 Using the Kafka REST Proxy Handler
          • 8.2.9.3.2.2 Downloading the Dependencies
          • 8.2.9.3.2.3 Classpath Configuration
          • 8.2.9.3.2.4 Kafka REST Proxy Handler Configuration
          • 8.2.9.3.2.5 Review a Sample Configuration
          • 8.2.9.3.2.6 Security
          • 8.2.9.3.2.7 Generating a Keystore or Truststore
          • 8.2.9.3.2.8 Using Templates to Resolve the Topic Name and Message Key
          • 8.2.9.3.2.9 Kafka REST Proxy Handler Formatter Properties
        • 8.2.9.3.3 Consuming the Records
        • 8.2.9.3.4 Performance Considerations
        • 8.2.9.3.5 Kafka REST Proxy Handler Metacolumns Template Property
    • 8.2.10 Apache Hive
    • 8.2.11 Azure Blob Storage
      • 8.2.11.1 Overview
      • 8.2.11.2 Prerequisites
      • 8.2.11.3 Storage Account, Container, and Objects
      • 8.2.11.4 Configuration
        • 8.2.11.4.1 Classpath Configuration
        • 8.2.11.4.2 Dependencies
        • 8.2.11.4.3 Authentication
          • 8.2.11.4.3.1 Azure Tenant ID, Client ID, and Client Secret
        • 8.2.11.4.4 Proxy Configuration
        • 8.2.11.4.5 Sample Configuration
        • 8.2.11.4.6 Azure Government Cloud Configuration
      • 8.2.11.5 Troubleshooting and Diagnostics
    • 8.2.12 Azure Data Lake Storage
      • 8.2.12.1 Azure Data Lake Gen1 (ADLS Gen1)
      • 8.2.12.2 Azure Data Lake Gen2 using Hadoop Client and ABFS
      • 8.2.12.3 Azure Data Lake Gen2 using BLOB endpoint
    • 8.2.13 Azure Event Hubs
    • 8.2.14 Azure Synapse Analytics Data Warehouse
      • 8.2.14.1 Detailed Functionality
        • 8.2.14.1.1 Database User Privileges
        • 8.2.14.1.2 Merge SQL Statement
        • 8.2.14.1.3 Prerequisites
      • 8.2.14.2 Operation Aggregation
        • 8.2.14.2.1 In-Memory Operation Aggregation
        • 8.2.14.2.2 Operation Aggregation Using SQL
      • 8.2.14.3 Compressed Update Handling
        • 8.2.14.3.1 MERGE statement with Uncompressed Updates
      • 8.2.14.4 Configuration
        • 8.2.14.4.1 Automatic Configuration
          • 8.2.14.4.1.1 File Writer Handler Configuration
          • 8.2.14.4.1.2 Parquet Event Handler Configuration
          • 8.2.14.4.1.3 Synapse Event Handler Configuration
        • 8.2.14.4.2 Synapse Database Credentials
        • 8.2.14.4.3 Classpath Configuration
          • 8.2.14.4.3.1 Dependencies
          • 8.2.14.4.3.2 Classpath
        • 8.2.14.4.4 INSERTALLRECORDS Support
        • 8.2.14.4.5 Large Object (LOB) Performance
        • 8.2.14.4.6 End-to-End Configuration
      • 8.2.14.5 Troubleshooting and Diagnostics
    • 8.2.15 Confluent Kafka
    • 8.2.16 Databricks
      • 8.2.16.1 Detailed Functionality
        • 8.2.16.1.1 Staging location
        • 8.2.16.1.2 Database User Privileges
        • 8.2.16.1.3 Prerequisites
      • 8.2.16.2 Configuration
        • 8.2.16.2.1 Automatic Configuration
          • 8.2.16.2.1.1 File Writer Handler Configuration
          • 8.2.16.2.1.2 ABS Event Handler Configuration
          • 8.2.16.2.1.3 s3 Event Handler Configuration
          • 8.2.16.2.1.4 GCS Event Handler Configuration
          • 8.2.16.2.1.5 Databricks Event Handler Configuration
        • 8.2.16.2.2 Authentication to Databricks
          • 8.2.16.2.2.1 Compute Settings for the Databricks JDBC Driver
        • 8.2.16.2.3 Unity Catalog
          • 8.2.16.2.3.1 Managed Tables
          • 8.2.16.2.3.2 External Tables
          • 8.2.16.2.3.3 Create a Storage Credential
        • 8.2.16.2.4 Create an External Location
          • 8.2.16.2.4.1 External Location to Access Azure Storage Account
          • 8.2.16.2.4.2 External Location to Access Google Storage Account
        • 8.2.16.2.5 Compute Clusters Without Unity Catalog
          • 8.2.16.2.5.1 Spark Configuration to Access Azure Storage Account
          • 8.2.16.2.5.2 Spark Configuration to Access Google Cloud Storage
          • 8.2.16.2.5.3 Spark Configuration to Access AWS S3
          • 8.2.16.2.5.4 Creating Databricks Secrets
        • 8.2.16.2.6 Classpath Configuration
          • 8.2.16.2.6.1 Dependencies
        • 8.2.16.2.7 Proxy Configuration
        • 8.2.16.2.8 Operation Aggregation
          • 8.2.16.2.8.1 In-memory Operation Aggregation
          • 8.2.16.2.8.2 Operation Aggregation using SQL
        • 8.2.16.2.9 Compressed Update Handling
          • 8.2.16.2.9.1 MERGE Statement with Uncompressed Updates
        • 8.2.16.2.10 End-to-End Configuration
          • 8.2.16.2.10.1 Databricks on Azure
          • 8.2.16.2.10.2 Databricks on AWS
          • 8.2.16.2.10.3 Databricks on GCP
        • 8.2.16.2.11 Table Mapping
          • 8.2.16.2.11.1 Mapping Table
      • 8.2.16.3 Troubleshooting and Diagnostics
    • 8.2.17 DataStax
    • 8.2.18 Elasticsearch
      • 8.2.18.1 Elasticsearch with Elasticsearch 7x and 6x
        • 8.2.18.1.1 Overview
        • 8.2.18.1.2 Detailing the Functionality
          • 8.2.18.1.2.1 About the Elasticsearch Version Property
          • 8.2.18.1.2.2 About the Index and Type
          • 8.2.18.1.2.3 About the Document
          • 8.2.18.1.2.4 About the Primary Key Update
          • 8.2.18.1.2.5 About the Data Types
          • 8.2.18.1.2.6 Operation Mode
          • 8.2.18.1.2.7 Operation Processing Support
          • 8.2.18.1.2.8 About the Connection
        • 8.2.18.1.3 Setting Up and Running the Elasticsearch Handler
          • 8.2.18.1.3.1 Configuring the Elasticsearch Handler
        • 8.2.18.1.4 Troubleshooting
          • 8.2.18.1.4.1 Incorrect Java Classpath
          • 8.2.18.1.4.2 Elasticsearch Version Mismatch
          • 8.2.18.1.4.3 Transport Client Properties File Not Found
          • 8.2.18.1.4.4 Cluster Connection Problem
          • 8.2.18.1.4.5 Unsupported Truncate Operation
          • 8.2.18.1.4.6 Bulk Execute Errors
        • 8.2.18.1.5 Performance Consideration
        • 8.2.18.1.6 About the Shield Plug-In Support
        • 8.2.18.1.7 About DDL Handling
        • 8.2.18.1.8 Known Issues in the Elasticsearch Handler
        • 8.2.18.1.9 Elasticsearch Handler Transport Client Dependencies
        • 8.2.18.1.10 Elasticsearch High Level REST Client Dependencies
      • 8.2.18.2 Elasticsearch 8x
        • 8.2.18.2.1 Overview
        • 8.2.18.2.2 Detailing the Functionality
        • 8.2.18.2.3 About the Index
        • 8.2.18.2.4 About the Document
        • 8.2.18.2.5 About the Data Types
        • 8.2.18.2.6 About the Connection
        • 8.2.18.2.7 About Supported Operation
        • 8.2.18.2.8 About DDL Handling
        • 8.2.18.2.9 About the Primary Key Update
        • 8.2.18.2.10 About UPSERT
        • 8.2.18.2.11 About Bulk Write
        • 8.2.18.2.12 About Routing
        • 8.2.18.2.13 About Request Headers
        • 8.2.18.2.14 About Java API Client
        • 8.2.18.2.15 Setting Up the Elasticsearch Handler
        • 8.2.18.2.16 Elasticsearch Handler Configuration
        • 8.2.18.2.17 Enabling Security for Elasticsearch
        • 8.2.18.2.18 Security Configuration for Elasticsearch Cluster
        • 8.2.18.2.19 Security Configuration for Elasticsearch Handler
        • 8.2.18.2.20 Troubleshooting
        • 8.2.18.2.21 Elasticsearch Handler Client Dependencies
          • 8.2.18.2.21.1 Elasticsearch 8.7.0
      • 8.2.18.3 Support for Vector Data
    • 8.2.19 Flat Files
      • 8.2.19.1 File Writer Handler
        • 8.2.19.1.1 Detailing the Functionality
          • 8.2.19.1.1.1 Using File Roll Events
          • 8.2.19.1.1.2 Automatic Directory Creation
          • 8.2.19.1.1.3 About the Active Write Suffix
          • 8.2.19.1.1.4 Maintenance of State
        • 8.2.19.1.2 Configuring the File Writer Handler
        • 8.2.19.1.3 Stopping the File Writer Handler
        • 8.2.19.1.4 Review a Sample Configuration
        • 8.2.19.1.5 File Writer Handler Partitioning
          • 8.2.19.1.5.1 File Writer Handler Partitioning Precondition
          • 8.2.19.1.5.2 Path Configuration
          • 8.2.19.1.5.3 Partitioning Configuration
          • 8.2.19.1.5.4 Partitioning Effect on Event Handler
      • 8.2.19.2 Optimized Row Columnar (ORC)
        • 8.2.19.2.1 Overview
        • 8.2.19.2.2 Detailing the Functionality
          • 8.2.19.2.2.1 About the Upstream Data Format
          • 8.2.19.2.2.2 About the Library Dependencies
          • 8.2.19.2.2.3 Requirements
        • 8.2.19.2.3 Configuring the ORC Event Handler
        • 8.2.19.2.4 Optimized Row Columnar Event Handler Client Dependencies
          • 8.2.19.2.4.1 ORC Client 1.6.9
          • 8.2.19.2.4.2 ORC Client 1.5.5
          • 8.2.19.2.4.3 ORC Client 1.4.0
      • 8.2.19.3 Parquet
        • 8.2.19.3.1 Parquet Handler
        • 8.2.19.3.2 Detailing the Functionality
          • 8.2.19.3.2.1 Configuring the Parquet Event Handler to Write to HDFS
          • 8.2.19.3.2.2 About the Upstream Data Format
        • 8.2.19.3.3 Configuring the Parquet Event Handler
        • 8.2.19.3.4 Parquet Event Handler Client Dependencies
          • 8.2.19.3.4.1 Parquet Client 1.12.0
          • 8.2.19.3.4.2 Parquet Client 1.11.1
          • 8.2.19.3.4.3 Parquet Client 1.10.1
          • 8.2.19.3.4.4 Parquet Client 1.9.0
    • 8.2.20 Google BigQuery
      • 8.2.20.1 Using Streaming API
        • 8.2.20.1.1 Detailing the Functionality
          • 8.2.20.1.1.1 Data Types
          • 8.2.20.1.1.2 Metadata Support
          • 8.2.20.1.1.3 Operation Modes
          • 8.2.20.1.1.4 Operation Processing Support
          • 8.2.20.1.1.5 Proxy Settings
          • 8.2.20.1.1.6 Mapping to Google Datasets
        • 8.2.20.1.2 Setting Up and Running the BigQuery Handler
          • 8.2.20.1.2.1 Schema Mapping for BigQuery
          • 8.2.20.1.2.2 Understanding the BigQuery Handler Configuration
          • 8.2.20.1.2.3 Review a Sample Configuration
          • 8.2.20.1.2.4 Configuring Handler Authentication
        • 8.2.20.1.3 Google BigQuery Dependencies
          • 8.2.20.1.3.1 BigQuery 2.7.1
      • 8.2.20.2 Google BigQuery Stage and Merge
        • 8.2.20.2.1 Overview
        • 8.2.20.2.2 Detailed Functionality
        • 8.2.20.2.3 Prerequisites
        • 8.2.20.2.4 Differences between BigQuery Handler and Stage and Merge BigQuery Event Handler
        • 8.2.20.2.5 Authentication or Authorization
          • 8.2.20.2.5.1 BigQuery Permissions
        • 8.2.20.2.6 Operation Aggregation
          • 8.2.20.2.6.1 In-Memory Operation Aggregation
          • 8.2.20.2.6.2 Operation Aggregation Using SQL
        • 8.2.20.2.7 Compressed Update Handling
          • 8.2.20.2.7.1 MERGE Statement with Uncompressed Updates
        • 8.2.20.2.8 Configuration
          • 8.2.20.2.8.1 Automatic Configuration
          • 8.2.20.2.8.2 Classpath Configuration
          • 8.2.20.2.8.3 Proxy Configuration
          • 8.2.20.2.8.4 INSERTALLRECORDS Support
          • 8.2.20.2.8.5 BigQuery Dataset and GCP ProjectId Mapping
          • 8.2.20.2.8.6 End-to-End Configuration
        • 8.2.20.2.9 Troubleshooting and Diagnostics
    • 8.2.21 Google Cloud Storage
      • 8.2.21.1 Overview
      • 8.2.21.2 Prerequisites
      • 8.2.21.3 Buckets and Objects
      • 8.2.21.4 Authentication and Authorization
        • 8.2.21.4.1 Bucket Permissions
        • 8.2.21.4.2 Object Permissions
      • 8.2.21.5 Configuration
        • 8.2.21.5.1 Classpath Configuration
          • 8.2.21.5.1.1 Dependencies
        • 8.2.21.5.2 Proxy Configuration
        • 8.2.21.5.3 Sample Configuration
    • 8.2.22 Google Pub/Sub
      • 8.2.22.1 Overview
      • 8.2.22.2 Detailed Functionality
      • 8.2.22.3 Setting up and Running the Google PubSub Handler
      • 8.2.22.4 Configuring Handler Authentication
      • 8.2.22.5 Google PubSub Handler Configuration
      • 8.2.22.6 Proxy Settings
      • 8.2.22.7 Sample Configuration
      • 8.2.22.8 Google PubSub Dependencies
    • 8.2.23 Java Message Service (JMS)
      • 8.2.23.1 Overview
      • 8.2.23.2 Setting Up and Running the JMS Handler
        • 8.2.23.2.1 Classpath Configuration
        • 8.2.23.2.2 Java Naming and Directory Interface Configuration
        • 8.2.23.2.3 Handler Configuration
        • 8.2.23.2.4 Sample Configuration Using Oracle WebLogic Server
      • 8.2.23.3 JMS Dependencies
        • 8.2.23.3.1 JMS 8.0
    • 8.2.24 Java Database Connectivity
      • 8.2.24.1 Overview
      • 8.2.24.2 Detailed Functionality
        • 8.2.24.2.1 Single Operation Mode
        • 8.2.24.2.2 Oracle Database Data Types
        • 8.2.24.2.3 MySQL Database Data Types
        • 8.2.24.2.4 Netezza Database Data Types
        • 8.2.24.2.5 Redshift Database Data Types
      • 8.2.24.3 Setting Up and Running the JDBC Handler
        • 8.2.24.3.1 Java Classpath
        • 8.2.24.3.2 Handler Configuration
        • 8.2.24.3.3 Statement Caching
        • 8.2.24.3.4 Setting Up Error Handling
      • 8.2.24.4 Sample Configurations
        • 8.2.24.4.1 Sample Oracle Database Target
        • 8.2.24.4.2 Sample Oracle Database Target with JDBC Metadata Provider
        • 8.2.24.4.3 Sample MySQL Database Target
        • 8.2.24.4.4 Sample MySQL Database Target with JDBC Metadata Provider
    • 8.2.25 MongoDB
      • 8.2.25.1 Overview
      • 8.2.25.2 MongoDB Wire Protocol
      • 8.2.25.3 Supported Target Types
      • 8.2.25.4 Detailed Functionality
        • 8.2.25.4.1 Document Key Column
        • 8.2.25.4.2 Primary Key Update Operation
        • 8.2.25.4.3 MongoDB Trail Data Types
      • 8.2.25.5 Setting Up and Running the MongoDB Handler
        • 8.2.25.5.1 Classpath Configuration
        • 8.2.25.5.2 MongoDB Handler Configuration
        • 8.2.25.5.3 Using Bulk Write
        • 8.2.25.5.4 Using Write Concern
        • 8.2.25.5.5 Using Three-Part Table Names
        • 8.2.25.5.6 Using Undo Handling
      • 8.2.25.6 Security and Authentication
        • 8.2.25.6.1 SSL Configuration Setup
      • 8.2.25.7 Reviewing Sample Configurations
      • 8.2.25.8 MongoDB to AJD/ATP Migration
        • 8.2.25.8.1 Overview
        • 8.2.25.8.2 Configuring MongoDB handler to Write to AJD/ATP
        • 8.2.25.8.3 Steps for Migration
        • 8.2.25.8.4 Best Practices
      • 8.2.25.9 MongoDB Handler Client Dependencies
        • 8.2.25.9.1 MongoDB Java Driver 4.6.0
        • 8.2.25.9.2 MongoDB Java Driver 3.12.8
    • 8.2.26 Netezza
    • 8.2.27 OCI Streaming
    • 8.2.28 Oracle NoSQL
      • 8.2.28.1 Overview
      • 8.2.28.2 On-Premise Connectivity
        • 8.2.28.2.1 Server Authentication
        • 8.2.28.2.2 Client Authentication
        • 8.2.28.2.3 Sample On-Premise Oracle NoSQL Configuration
      • 8.2.28.3 OCI Cloud Connectivity
        • 8.2.28.3.1 Server Authentication
        • 8.2.28.3.2 Client Authentication
        • 8.2.28.3.3 Sample Cloud Oracle NoSQL Configuration
        • 8.2.28.3.4 Sample OCI Configuration file
      • 8.2.28.4 Oracle NoSQL Types
      • 8.2.28.5 Oracle NoSQL Handler Configuration
      • 8.2.28.6 Performance Considerations
      • 8.2.28.7 Operation Processing Support
      • 8.2.28.8 Column Processing
      • 8.2.28.9 Table Check and Reconciliation Process
        • 8.2.28.9.1 Full Image Data Requirements
      • 8.2.28.10 Oracle NoSQL SDK Dependencies
        • 8.2.28.10.1 Oracle NoSQL SDK Dependencies 5.2.27
    • 8.2.29 OCI Autonomous Data Warehouse
      • 8.2.29.1 Detailed Functionality
      • 8.2.29.2 ADW Database Credential to Access OCI ObjectStore File
      • 8.2.29.3 ADW Database User Privileges
      • 8.2.29.4 Unsupported Operations/ Limitations
      • 8.2.29.5 Troubleshooting and Diagnostics
      • 8.2.29.6 Classpath
      • 8.2.29.7 Configuration
        • 8.2.29.7.1 Automatic Configuration
        • 8.2.29.7.2 File Writer Handler Configuration
        • 8.2.29.7.3 OCI Event Handler Configuration
        • 8.2.29.7.4 ADW Event Handler Configuration
        • 8.2.29.7.5 INSERTALLRECORDS Support
        • 8.2.29.7.6 End-to-End Configuration
        • 8.2.29.7.7 Compressed Update Handling
          • 8.2.29.7.7.1 MERGE Statement with Uncompressed Updates
    • 8.2.30 Oracle Cloud Infrastructure Object Storage
      • 8.2.30.1 Overview
      • 8.2.30.2 Detailing the Functionality
      • 8.2.30.3 Configuration
        • 8.2.30.3.1 Automatic Configuration
          • 8.2.30.3.1.1 File Writer Handler Configuration
          • 8.2.30.3.1.2 Formatter Configuration
      • 8.2.30.4 Configuring Credentials for Oracle Cloud Infrastructure
      • 8.2.30.5 Troubleshooting
      • 8.2.30.6 OCI Dependencies
        • 8.2.30.6.1 OCI 1.34.0
    • 8.2.31 Redis
      • 8.2.31.1 Data Structures Supported by the Redis Handler
        • 8.2.31.1.1 Hash Maps
        • 8.2.31.1.2 Streams
        • 8.2.31.1.3 JSONs
      • 8.2.31.2 Redis Handler Configuration Properties
      • 8.2.31.3 Security
      • 8.2.31.4 Authentication Using Credentials
      • 8.2.31.5 SSL Basic Auth
      • 8.2.31.6 SSL Mutual Auth
      • 8.2.31.7 Redis Handler Dependencies
      • 8.2.31.8 Redis Handler Client Dependencies
        • 8.2.31.8.1 jedis 4.2.3
    • 8.2.32 Snowflake
      • 8.2.32.1 Snowflake Stage and Merge Handler
        • 8.2.32.1.1 Overview
        • 8.2.32.1.2 Detailed Functionality
          • 8.2.32.1.2.1 Staging Location
          • 8.2.32.1.2.2 Database User Privileges
          • 8.2.32.1.2.3 Prerequisites
        • 8.2.32.1.3 Configuration
          • 8.2.32.1.3.1 Automatic Configuration
          • 8.2.32.1.3.2 Snowflake Storage Integration
          • 8.2.32.1.3.3 Classpath Configuration
          • 8.2.32.1.3.4 Proxy Configuration
          • 8.2.32.1.3.5 INSERTALLRECORDS Support
          • 8.2.32.1.3.6 Snowflake Key Pair Authentication
          • 8.2.32.1.3.7 Mapping Source JSON/XML to Snowflake VARIANT
          • 8.2.32.1.3.8 Operation Aggregation
          • 8.2.32.1.3.9 Compressed Update Handling
          • 8.2.32.1.3.10 End-to-End Configuration
          • 8.2.32.1.3.11 Table Mapping
        • 8.2.32.1.4 Troubleshooting and Diagnostics
      • 8.2.32.2 Snowflake Streaming Handler
        • 8.2.32.2.1 Overview
        • 8.2.32.2.2 Detailed Functionality
          • 8.2.32.2.2.1 Database User Privileges
          • 8.2.32.2.2.2 Prerequisites
          • 8.2.32.2.2.3 Staging using Snowpipe Streaming API
        • 8.2.32.2.3 Configuration
        • 8.2.32.2.4 Classpath Configuration
          • 8.2.32.2.4.1 Dependencies
          • 8.2.32.2.4.2 Maven Co-ordinates
        • 8.2.32.2.5 Proxy Configuration
        • 8.2.32.2.6 Snowflake Streaming Handler Key Pair Authentication
        • 8.2.32.2.7 Sample Configuration
        • 8.2.32.2.8 Troubleshooting and Diagnostics
    • 8.2.33 Additional Details
      • 8.2.33.1 Command Event Handler
        • 8.2.33.1.1 Overview - Command Event Handler
        • 8.2.33.1.2 Configuring the Command Event Handler
        • 8.2.33.1.3 Using Command Argument Template Strings
      • 8.2.33.2 HDFS Event Handler
        • 8.2.33.2.1 Detailing the Functionality
          • 8.2.33.2.1.1 Configuring the Handler
          • 8.2.33.2.1.2 Configuring the HDFS Event Handler
      • 8.2.33.3 Metacolumn Keywords
      • 8.2.33.4 Metadata Providers
        • 8.2.33.4.1 About the Metadata Providers
        • 8.2.33.4.2 Avro Metadata Provider
          • 8.2.33.4.2.1 Detailed Functionality
          • 8.2.33.4.2.2 Runtime Prerequisites
          • 8.2.33.4.2.3 Classpath Configuration
          • 8.2.33.4.2.4 Avro Metadata Provider Configuration
          • 8.2.33.4.2.5 Review a Sample Configuration
          • 8.2.33.4.2.6 Metadata Change Events
          • 8.2.33.4.2.7 Limitations
          • 8.2.33.4.2.8 Troubleshooting
        • 8.2.33.4.3 Java Database Connectivity Metadata Provider
          • 8.2.33.4.3.1 JDBC Detailed Functionality
          • 8.2.33.4.3.2 Java Classpath
          • 8.2.33.4.3.3 JDBC Metadata Provider Configuration
          • 8.2.33.4.3.4 Review a Sample Configuration
        • 8.2.33.4.4 Hive Metadata Provider
          • 8.2.33.4.4.1 Detailed Functionality
          • 8.2.33.4.4.2 Configuring Hive with a Remote Metastore Database
          • 8.2.33.4.4.3 Classpath Configuration
          • 8.2.33.4.4.4 Hive Metadata Provider Configuration Properties
          • 8.2.33.4.4.5 Review a Sample Configuration
          • 8.2.33.4.4.6 Security
          • 8.2.33.4.4.7 Metadata Change Event
          • 8.2.33.4.4.8 Limitations
          • 8.2.33.4.4.9 Additional Considerations
          • 8.2.33.4.4.10 Troubleshooting
        • 8.2.33.4.5 Google BigQuery Metadata Provider
          • 8.2.33.4.5.1 Authentication
          • 8.2.33.4.5.2 Supported BigQuery Datatypes
          • 8.2.33.4.5.3 Parameterized BigQuery Datatypes
          • 8.2.33.4.5.4 Unsupported BigQuery Datatypes
          • 8.2.33.4.5.5 Configuring BigQuery Metadata Provider
          • 8.2.33.4.5.6 Sample Configuration
          • 8.2.33.4.5.7 Proxy Settings
          • 8.2.33.4.5.8 Classpath Settings
          • 8.2.33.4.5.9 Limitations
      • 8.2.33.5 Pluggable Formatters
        • 8.2.33.5.1 Using Operation-Based versus Row-Based Formatting
          • 8.2.33.5.1.1 Operation Formatters
          • 8.2.33.5.1.2 Row Formatters
          • 8.2.33.5.1.3 Table Row or Column Value States
        • 8.2.33.5.2 Using the Avro Formatter
          • 8.2.33.5.2.1 Avro Row Formatter
          • 8.2.33.5.2.2 The Avro Operation Formatter
          • 8.2.33.5.2.3 Avro Object Container File Formatter
        • 8.2.33.5.3 Existing Avro Formatter
          • 8.2.33.5.3.1 Prerequisites
          • 8.2.33.5.3.2 Supported Output Formats
          • 8.2.33.5.3.3 Support for Mapping to Nested Avro Fields
          • 8.2.33.5.3.4 Support for Mapping to Nested Avro Fields
          • 8.2.33.5.3.5 Configuration
          • 8.2.33.5.3.6 Unsupported Features
        • 8.2.33.5.4 Using the Delimited Text Formatter
          • 8.2.33.5.4.1 Using the Delimited Text Row Formatter
          • 8.2.33.5.4.2 Delimited Text Operation Formatter
        • 8.2.33.5.5 Using the JSON Formatter
          • 8.2.33.5.5.1 Operation Metadata Formatting Details
          • 8.2.33.5.5.2 Operation Data Formatting Details
          • 8.2.33.5.5.3 Row Data Formatting Details
          • 8.2.33.5.5.4 Sample JSON Messages
          • 8.2.33.5.5.5 JSON Schemas
          • 8.2.33.5.5.6 JSON Formatter Configuration Properties
          • 8.2.33.5.5.7 Review a Sample Configuration
          • 8.2.33.5.5.8 Metadata Change Events
          • 8.2.33.5.5.9 JSON Primary Key Updates
          • 8.2.33.5.5.10 Integrating Oracle Stream Analytics
        • 8.2.33.5.6 Using the Length Delimited Value Formatter
          • 8.2.33.5.6.1 Formatting Message Details
          • 8.2.33.5.6.2 Sample Formatted Messages
          • 8.2.33.5.6.3 LDV Formatter Configuration Properties
          • 8.2.33.5.6.4 Additional Considerations
        • 8.2.33.5.7 Using the XML Formatter
          • 8.2.33.5.7.1 Message Formatting Details
          • 8.2.33.5.7.2 Sample XML Messages
          • 8.2.33.5.7.3 XML Schema
          • 8.2.33.5.7.4 XML Formatter Configuration Properties
          • 8.2.33.5.7.5 Review a Sample Configuration
          • 8.2.33.5.7.6 Metadata Change Events
          • 8.2.33.5.7.7 Primary Key Updates
      • 8.2.33.6 Stage and Merge Data Warehouse Replication
        • 8.2.33.6.1 Steps for Stage and Merge
          • 8.2.33.6.1.1 Stage
          • 8.2.33.6.1.2 Merge
          • 8.2.33.6.1.3 Configuration of Handlers
          • 8.2.33.6.1.4 File Writer Handler
          • 8.2.33.6.1.5 Operation Aggregation
          • 8.2.33.6.1.6 Object Store Event handler
          • 8.2.33.6.1.7 JDBC Metadata Provider
          • 8.2.33.6.1.8 Command Event handler Merge Script
          • 8.2.33.6.1.9 Stage and Merge Sample Configuration
          • 8.2.33.6.1.10 Variables in the Merge Script
          • 8.2.33.6.1.11 SQL Statements in the Merge Script
          • 8.2.33.6.1.12 Merge Script Functions
          • 8.2.33.6.1.13 Prerequisites
          • 8.2.33.6.1.14 Limitations
        • 8.2.33.6.2 Hive Stage and Merge
          • 8.2.33.6.2.1 Data Flow
          • 8.2.33.6.2.2 Configuration
          • 8.2.33.6.2.3 Merge Script Variables
          • 8.2.33.6.2.4 Prerequisites
      • 8.2.33.7 Template Keywords
      • 8.2.33.8 Velocity Dependencies
• 9 Administer
  • 9.1 Automatic Heartbeat for Oracle GoldenGate for Distributed Applications and Analytics
    • 9.1.1 Overview
    • 9.1.2 Automatic Heartbeat Tables
      • 9.1.2.1 ADD HEARTBEATTABLE
      • 9.1.2.2 ALTER HEARTBEAT TABLE
      • 9.1.2.3 INFO HEARTBEATTABLE
      • 9.1.2.4 LAG
      • 9.1.2.5 DELETE HEARTBEATTABLE
  • 9.2 Parsing the Message
    • 9.2.1 Parsing Overview
      • 9.2.1.1 Parser Types
      • 9.2.1.2 Source and Target Data Definitions
      • 9.2.1.3 Required Data
        • 9.2.1.3.1 Transaction Identifier
        • 9.2.1.3.2 Sequence Identifier
        • 9.2.1.3.3 Timestamp
        • 9.2.1.3.4 Table Name
        • 9.2.1.3.5 Operation Type
        • 9.2.1.3.6 Column Data
      • 9.2.1.4 Optional Data
        • 9.2.1.4.1 Transaction Indicator
        • 9.2.1.4.2 Transaction Name
        • 9.2.1.4.3 Transaction Owner
    • 9.2.2 Fixed Width Parsing
      • 9.2.2.1 Header
        • 9.2.2.1.1 Specifying Compound Table Names
        • 9.2.2.1.2 Specifying timestamp Formats
        • 9.2.2.1.3 Specifying the Function
      • 9.2.2.2 Header and Record Data Type Translation
      • 9.2.2.3 Key identification
      • 9.2.2.4 Using a Source Definition File
    • 9.2.3 Delimited Parsing
      • 9.2.3.1 Metadata Columns
      • 9.2.3.2 Parsing Properties
        • 9.2.3.2.1 Properties to Describe Delimiters
        • 9.2.3.2.2 Properties to Describe Values
        • 9.2.3.2.3 Properties to Describe Date and Time
      • 9.2.3.3 Parsing Steps
    • 9.2.4 XML Parsing
      • 9.2.4.1 Styles of XML
      • 9.2.4.2 XML Parsing Rules
      • 9.2.4.3 XPath Expressions
        • 9.2.4.3.1 Supported Constructs:
        • 9.2.4.3.2 Supported Expressions
        • 9.2.4.3.3 Obtaining Data Values
      • 9.2.4.4 Other Value Expressions
      • 9.2.4.5 Transaction Rules
      • 9.2.4.6 Operation Rules
      • 9.2.4.7 Column Rules
      • 9.2.4.8 Overall Rules Example
    • 9.2.5 Source Definitions Generation Utility
  • 9.3 Message Capture Properties
    • 9.3.1 Logging and Connection Properties
      • 9.3.1.1 Logging Properties
        • 9.3.1.1.1 gg.log
        • 9.3.1.1.2 gg.log.level
        • 9.3.1.1.3 gg.log.file
        • 9.3.1.1.4 gg.log.classpath
      • 9.3.1.2 JMS Connection Properties
        • 9.3.1.2.1 jvm.boot options
        • 9.3.1.2.2 jms.report.output
        • 9.3.1.2.3 jms.report.time
        • 9.3.1.2.4 jms.report.records
        • 9.3.1.2.5 jms.id
        • 9.3.1.2.6 jms.destination
        • 9.3.1.2.7 jms.connectionFactory
        • 9.3.1.2.8 jms.user, jms.password
      • 9.3.1.3 JNDI Properties
    • 9.3.2 Parser Properties
      • 9.3.2.1 Setting the Type of Parser
        • 9.3.2.1.1 parser.type
      • 9.3.2.2 Fixed Parser Properties
        • 9.3.2.2.1 fixed.schematype
        • 9.3.2.2.2 fixed.sourcedefs
        • 9.3.2.2.3 fixed.copybook
        • 9.3.2.2.4 fixed.header
        • 9.3.2.2.5 fixed.seqid
        • 9.3.2.2.6 fixed.timestamp
        • 9.3.2.2.7 fixed.timestamp.format
        • 9.3.2.2.8 fixed.txid
        • 9.3.2.2.9 fixed.txowner
        • 9.3.2.2.10 fixed.txname
        • 9.3.2.2.11 fixed.optype
        • 9.3.2.2.12 fixed.optype.insertval
        • 9.3.2.2.13 fixed.optype.updateval
        • 9.3.2.2.14 fixed.optype.deleteval
        • 9.3.2.2.15 fixed.table
        • 9.3.2.2.16 fixed.schema
        • 9.3.2.2.17 fixed.txind
        • 9.3.2.2.18 fixed.txind.beginval
        • 9.3.2.2.19 fixed.txind.middleval
        • 9.3.2.2.20 fixed.txind.endval
        • 9.3.2.2.21 fixed.txind.wholeval
      • 9.3.2.3 Delimited Parser Properties
        • 9.3.2.3.1 delim.sourcedefs
        • 9.3.2.3.2 delim.header
        • 9.3.2.3.3 delim.seqid
        • 9.3.2.3.4 delim.timestamp
        • 9.3.2.3.5 delim.timestamp.format
        • 9.3.2.3.6 delim.txid
        • 9.3.2.3.7 delim.txowner
        • 9.3.2.3.8 delim.txname
        • 9.3.2.3.9 delim.optype
        • 9.3.2.3.10 delim.optype.insertval
        • 9.3.2.3.11 delim.optype.updateval
        • 9.3.2.3.12 delim.optype.deleteval
        • 9.3.2.3.13 delim.schemaandtable
        • 9.3.2.3.14 delim.schema
        • 9.3.2.3.15 delim.table
        • 9.3.2.3.16 delim.txind
        • 9.3.2.3.17 delim.txind.beginval
        • 9.3.2.3.18 delim.txind.middleval
        • 9.3.2.3.19 delim.txind.endval
        • 9.3.2.3.20 delim.txind.wholeval
        • 9.3.2.3.21 delim.fielddelim
        • 9.3.2.3.22 delim.linedelim
        • 9.3.2.3.23 delim.quote
        • 9.3.2.3.24 delim.nullindicator
        • 9.3.2.3.25 delim.fielddelim.escaped
        • 9.3.2.3.26 delim.linedelim.escaped
        • 9.3.2.3.27 delim.quote.escaped
        • 9.3.2.3.28 delim.nullindicator.escaped
        • 9.3.2.3.29 delim.hasbefores
        • 9.3.2.3.30 delim.hasnames
        • 9.3.2.3.31 delim.afterfirst
        • 9.3.2.3.32 delim.isgrouped
        • 9.3.2.3.33 delim.dateformat | delim.dateformat.table | delim.dateform.table.column
      • 9.3.2.4 XML Parser Properties
        • 9.3.2.4.1 xml.sourcedefs
        • 9.3.2.4.2 xml.rules
        • 9.3.2.4.3 rulename.type
        • 9.3.2.4.4 rulename.match
        • 9.3.2.4.5 rulename.subrules
        • 9.3.2.4.6 txrule.timestamp
        • 9.3.2.4.7 txrule.timestamp.format
        • 9.3.2.4.8 txrule.seqid
        • 9.3.2.4.9 txrule.txid
        • 9.3.2.4.10 txrule.txowner
        • 9.3.2.4.11 txrule.txname
        • 9.3.2.4.12 oprule.timestamp
        • 9.3.2.4.13 oprule.timestamp.format
        • 9.3.2.4.14 oprule.seqid
        • 9.3.2.4.15 oprule.txid
        • 9.3.2.4.16 oprule.txowner
        • 9.3.2.4.17 oprule.txname
        • 9.3.2.4.18 oprule.schemandtable
        • 9.3.2.4.19 oprule.schema
        • 9.3.2.4.20 oprule.table
        • 9.3.2.4.21 oprule.optype
        • 9.3.2.4.22 oprule.optype.insertval
        • 9.3.2.4.23 oprule.optype.updateval
        • 9.3.2.4.24 oprule.optype.deleteval
        • 9.3.2.4.25 oprule.txind
        • 9.3.2.4.26 oprule.txind.beginval
        • 9.3.2.4.27 oprule.txind.middleval
        • 9.3.2.4.28 oprule.txind.endval
        • 9.3.2.4.29 oprule.txind.wholeval
        • 9.3.2.4.30 colrule.name
        • 9.3.2.4.31 colrule.index
        • 9.3.2.4.32 colrule.value
        • 9.3.2.4.33 colrule.isnull
        • 9.3.2.4.34 colrule.ismissing
        • 9.3.2.4.35 colrule.before.value
        • 9.3.2.4.36 colrule.before.isnull
        • 9.3.2.4.37 colrule.before.ismissing
        • 9.3.2.4.38 colrule.after.value
        • 9.3.2.4.39 colrule.after.isnull
        • 9.3.2.4.40 colrule.after.ismissing
  • 9.4 Oracle GoldenGate Java Delivery
    • 9.4.1 Configuring Java Delivery
      • 9.4.1.1 Configuring the JRE in the Properties File
      • 9.4.1.2 Configuring Oracle GoldenGate for Java Delivery
        • 9.4.1.2.1 Configuring a Replicat for Java Delivery
      • 9.4.1.3 Configuring the Java Handlers
    • 9.4.2 Running Java Delivery
      • 9.4.2.1 Starting the Application
        • 9.4.2.1.1 Starting Using Replicat
      • 9.4.2.2 Restarting the Java Delivery
        • 9.4.2.2.1 Restarting Java Delivery in Replicat
    • 9.4.3 Configuring Event Handlers
      • 9.4.3.1 Specifying Event Handlers
      • 9.4.3.2 JMS Handler
      • 9.4.3.3 File Handler
      • 9.4.3.4 Custom Handlers
      • 9.4.3.5 Formatting the Output
      • 9.4.3.6 Reporting
    • 9.4.4 Java Delivery Properties
      • 9.4.4.1 Common Properties
        • 9.4.4.1.1 Logging Properties
          • 9.4.4.1.1.1 gg.log
          • 9.4.4.1.1.2 gg.log.level
          • 9.4.4.1.1.3 gg.log.file
          • 9.4.4.1.1.4 gg.log.classpath
        • 9.4.4.1.2 JVM Boot Options
          • 9.4.4.1.2.1 jvm.bootoptions
      • 9.4.4.2 Delivery Properties
        • 9.4.4.2.1 General Properties
          • 9.4.4.2.1.1 goldengate.userexit.writers
          • 9.4.4.2.1.2 goldengate.userexit.chkptprefix
          • 9.4.4.2.1.3 goldengate.userexit.nochkpt
          • 9.4.4.2.1.4 goldengate.userexit.usetargetcols
        • 9.4.4.2.2 Statistics and Reporting
          • 9.4.4.2.2.1 jvm.stats.display
          • 9.4.4.2.2.2 jvm.stats.full
          • 9.4.4.2.2.3 jvm.stats.time | jvm.stats.numrecs
      • 9.4.4.3 Java Application Properties
        • 9.4.4.3.1 Properties for All Handlers
          • 9.4.4.3.1.1 gg.handlerlist
          • 9.4.4.3.1.2 gg.handler.name.type
        • 9.4.4.3.2 Properties for Formatted Output
          • 9.4.4.3.2.1 gg.handler.name.format
          • 9.4.4.3.2.2 gg.handler.name.includeTables
          • 9.4.4.3.2.3 gg.handler.name.excludeTables
          • 9.4.4.3.2.4 gg.handler.name.mode, gg.handler.name.format.mode
        • 9.4.4.3.3 Properties for CSV and Fixed Format Output
          • 9.4.4.3.3.1 gg.handler.name.format.delim
          • 9.4.4.3.3.2 gg.handler.name.format.quote
          • 9.4.4.3.3.3 gg.handler.name.format.metacols
          • 9.4.4.3.3.4 gg.handler.name.format.missingColumnChar
          • 9.4.4.3.3.5 gg.handler.name.format.presentColumnChar
          • 9.4.4.3.3.6 gg.handler.name.format.nullColumnChar
          • 9.4.4.3.3.7 gg.handler.name.format.beginTxChar
          • 9.4.4.3.3.8 gg.handler.name.format.middleTxChar
          • 9.4.4.3.3.9 gg.handler.name.format.endTxChar
          • 9.4.4.3.3.10 gg.handler.name.format.wholeTxChar
          • 9.4.4.3.3.11 gg.handler.name.format.insertChar
          • 9.4.4.3.3.12 gg.handler.name.format.updateChar
          • 9.4.4.3.3.13 gg.handler.name.format.deleteChar
          • 9.4.4.3.3.14 gg.handler.name.format.truncateChar
          • 9.4.4.3.3.15 gg.handler.name.format.endOfLine
          • 9.4.4.3.3.16 gg.handler.name.format.justify
          • 9.4.4.3.3.17 gg.handler.name.format.includeBefores
        • 9.4.4.3.4 File Writer Properties
          • 9.4.4.3.4.1 gg.handler.name.file
          • 9.4.4.3.4.2 gg.handler.name.append
          • 9.4.4.3.4.3 gg.handler.name.rolloverSize
        • 9.4.4.3.5 JMS Handler Properties
          • 9.4.4.3.5.1 Standard JMS Settings
          • 9.4.4.3.5.2 Group Transaction Properties
        • 9.4.4.3.6 JNDI Properties
        • 9.4.4.3.7 General Properties
          • 9.4.4.3.7.1 gg.classpath
          • 9.4.4.3.7.2 gg.report.time
          • 9.4.4.3.7.3 gg.binaryencoding
        • 9.4.4.3.8 Java Delivery Transaction Grouping
    • 9.4.5 Developing Custom Filters, Formatters, and Handlers
      • 9.4.5.1 Filtering Events
      • 9.4.5.2 Custom Formatting
        • 9.4.5.2.1 Coding a Custom Formatter in Java
        • 9.4.5.2.2 Using a Velocity Template
      • 9.4.5.3 Coding a Custom Handler in Java
      • 9.4.5.4 Additional Resources
• 10 Troubleshoot
  • 10.1 Troubleshooting the Java Adapters
    • 10.1.1 Checking for Errors
    • 10.1.2 Reporting Issues