Table of Contents
- Title and Copyright Information
- Preface
-
1
Introducing Oracle GoldenGate for Big Data
- 1.1 Understanding What’s Supported
- 1.2 Setting Up Oracle GoldenGate for Big Data
-
1.3
Configuring Oracle GoldenGate for Big Data
- 1.3.1 Running with Replicat
- 1.3.2 Overview of Logging
- 1.3.3 About Schema Evolution and Metadata Change Events
- 1.3.4 About Configuration Property CDATA[] Wrapping
- 1.3.5 Using Regular Expression Search and Replace
- 1.3.6 Scaling Oracle GoldenGate for Big Data Delivery
- 1.3.7 Using Identities in Oracle GoldenGate Credential Store
- 2 Using the BigQuery Handler
-
3
Using the Cassandra Handler
- 3.1 Overview
- 3.2 Detailing the Functionality
- 3.3 Setting Up and Running the Cassandra Handler
- 3.4 About Automated DDL Handling
- 3.5 Performance Considerations
- 3.6 Additional Considerations
- 3.7 Troubleshooting
- 4 Using the Elasticsearch Handler
- 5 Using the File Writer Handler
- 6 Using the Flume Handler
- 7 Using the HBase Handler
-
8
Using the HDFS Handler
- 8.1 Overview
- 8.2 Writing into HDFS in SequenceFile Format
- 8.3 Setting Up and Running the HDFS Handler
- 8.4 Writing in HDFS in Avro Object Container File Format
- 8.5 Generating HDFS File Names Using Template Strings
- 8.6 Metadata Change Events
- 8.7 Partitioning
- 8.8 HDFS Additional Considerations
- 8.9 Best Practices
- 8.10 Troubleshooting the HDFS Handler
- 9 Using the Java Database Connectivity Handler
- 10 Using the Kafka Handler
- 11 Using the Kafka Connect Handler
-
12
Using the Kafka REST Proxy Handler
- 12.1 Overview
-
12.2
Setting Up and Starting the Kafka REST Proxy Handler Services
- 12.2.1 Using the Kafka REST Proxy Handler
- 12.2.2 Downloading the Dependencies
- 12.2.3 Classpath Configuration
- 12.2.4 Kafka REST Proxy Handler Configuration
- 12.2.5 Review a Sample Configuration
- 12.2.6 Security
- 12.2.7 Generating a Keystore or Truststore
- 12.2.8 Using Templates to Resolve the Topic Name and Message Key
- 12.2.9 Kafka REST Proxy Handler Formatter Properties
- 12.3 Consuming the Records
- 12.4 Performance Considerations
- 12.5 Kafka REST Proxy Handler Metacolumns Template Property
-
13
Using the Kinesis Streams Handler
- 13.1 Overview
- 13.2 Detailed Functionality
-
13.3
Setting Up and Running the Kinesis Streams Handler
- 13.3.1 Set the Classpath in Kinesis Streams Handler
- 13.3.2 Kinesis Streams Handler Configuration
- 13.3.3 Using Templates to Resolve the Stream Name and Partition Name
- 13.3.4 Configuring the Client ID and Secret in Kinesis Handler
- 13.3.5 Configuring the Proxy Server for Kinesis Streams Handler
- 13.3.6 Configuring Security in Kinesis Streams Handler
- 13.4 Kinesis Handler Performance Considerations
- 13.5 Troubleshooting
- 14 Using the MongoDB Handler
-
15
Using the Metadata Providers
- 15.1 About the Metadata Providers
- 15.2 Avro Metadata Provider
- 15.3 Java Database Connectivity Metadata Provider
-
15.4
Hive Metadata Provider
- 15.4.1 Detailed Functionality
- 15.4.2 Configuring Hive with a Remote Metastore Database
- 15.4.3 Classpath Configuration
- 15.4.4 Hive Metadata Provider Configuration Properties
- 15.4.5 Review a Sample Configuration
- 15.4.6 Security
- 15.4.7 Metadata Change Event
- 15.4.8 Limitations
- 15.4.9 Additional Considerations
- 15.4.10 Troubleshooting
- 16 Using the Oracle NoSQL Handler
-
17
Using the Pluggable Formatters
-
17.1
Using the Avro Formatter
- 17.1.1 Avro Row Formatter
-
17.1.2
The Avro Operation Formatter
- 17.1.2.1 Operation Metadata Formatting Details
- 17.1.2.2 Operation Data Formatting Details
- 17.1.2.3 Sample Avro Operation Messages
- 17.1.2.4 Avro Schema
- 17.1.2.5 Avro Operation Formatter Configuration Properties
- 17.1.2.6 Review a Sample Configuration
- 17.1.2.7 Metadata Change Events
- 17.1.2.8 Special Considerations
- 17.1.3 Avro Object Container File Formatter
- 17.1.4 Setting Metacolumn Output
- 17.2 Using the Delimited Text Formatter
-
17.3
Using the JSON Formatter
- 17.3.1 Operation Metadata Formatting Details
- 17.3.2 Operation Data Formatting Details
- 17.3.3 Row Data Formatting Details
- 17.3.4 Sample JSON Messages
- 17.3.5 JSON Schemas
- 17.3.6 JSON Formatter Configuration Properties
- 17.3.7 Review a Sample Configuration
- 17.3.8 Metadata Change Events
- 17.3.9 Setting Metacolumn Output
- 17.3.10 JSON Primary Key Updates
- 17.3.11 Integrating Oracle Stream Analytics
- 17.4 Using the Length Delimited Value Formatter
- 17.5 Using Operation-Based versus Row-Based Formatting
- 17.6 Using the XML Formatter
-
17.1
Using the Avro Formatter
-
18
Using Oracle GoldenGate Capture for Cassandra
- 18.1 Overview
- 18.2 Setting Up Cassandra Change Data Capture
- 18.3 Deduplication
- 18.4 Topology Changes
- 18.5 Data Availability in the CDC Logs
- 18.6 Using Extract Initial Load
- 18.7 Using Change Data Capture Extract
- 18.8 Replicating to RDMBS Targets
- 18.9 Partition Update or Insert of Static Columns
- 18.10 Partition Delete
- 18.11 Security and Authentication
- 18.12 Multiple Extract Support
- 18.13 CDC Configuration Reference
- 18.14 Troubleshooting
- 19 Connecting to Microsoft Azure Data Lake
- A Cassandra Handler Client Dependencies
- B Cassandra Capture Client Dependencies
- C Elasticsearch Handler Client Dependencies
- D Flume Handler Client Dependencies
- E HBase Handler Client Dependencies
- F HDFS Handler Client Dependencies
- G Kafka Handler Client Dependencies
-
H
Kafka Connect Handler Client Dependencies
- H.1 Kafka 1.1.0
- H.2 Kafka 1.0.0
- H.3 Kafka 0.11.0.0
- H.4 Kafka 0.10.2.0
- H.5 Kafka 0.10.2.0
- H.6 Kafka 0.10.0.0
- H.7 Kafka 0.9.0.1
- H.8 Confluent 4.1.2
- H.9 Confluent 4.0.0
- H.10 Confluent 3.2.1
- H.11 Confluent 3.2.0
- H.12 Confluent 3.2.1
- H.13 Confluent 3.1.1
- H.14 Confluent 3.0.1
- H.15 Confluent 2.0.1
- H.16 Confluent 2.0.1
- I MongoDB Handler Client Dependencies
- J Optimized Row Columnar Event Handler Client Dependencies
- K Parquet Event Handler Client Dependencies