This chapter includes the following sections.
Topics:
Parent topic: Understanding Oracle GoldenGate for Big Data
This section provides an overview of the Oracle GoldenGate Adapters.
The Oracle GoldenGate Java Adapter integrates with Oracle GoldenGate instances.
The Oracle GoldenGate product enables you to:
Captures transactional changes from a source database
Sends and queues these changes as a set of database-independent files called the Oracle GoldenGate trail
Optionally alters the source data using mapping parameters and functions
Applies the transactions in the trail to a target system database
Oracle GoldenGate performs this capture and apply in near real-time across heterogeneous databases, platforms, and operating systems.
There are two major products which are based on the Oracle GoldenGate Adapter architecture:
The Oracle GoldenGate Java Adapter is the overall framework. This product allows you to implement custom code to handle Oracle GoldenGate trail records according to their specific requirements. It comes built-in with Oracle GoldenGate File Writer module that can be used for flat file integration purposes.
Oracle GoldenGate for Big Data. The Oracle GoldenGate for Big Data product contains built-in support to write operation data from Oracle GoldenGate trail records into various Big Data targets (such as, HDFS, HBase, Kafka, Flume, JDBC, Cassandra, and MongoDB). You do not need to write custom code to integrate with Big Data applications. The functionality is separated into handlers that integrate with third party applications and formatters, which transform the data into various formats, such as Avro, JSON, delimited text, and XML. In certain instances, the integration to a third-party tool is proprietary, like the HBase API. In these instances, the formatter exists without an associated handler.
The Oracle GoldenGate Java Adapter and the Oracle GoldenGate for Big Data products have some crossover in functionality so the handler exists without an associated formatter. The following list details the major areas of functionality and in which product or products the functionality is included:
Read JMS messages and deliver them as an Oracle GoldenGate trail. This feature is included in both Oracle GoldenGate Adapters and Oracle GoldenGate for Big Data products.
Read an Oracle GoldenGate trail and deliver transactions to a JMS provider or other messaging system or custom application. This feature is included in both Oracle GoldenGate Adapters and Oracle GoldenGate for Big Data products.
Read an Oracle GoldenGate trail and write transactions to a file that can be used by other applications. This feature is only included in the Oracle GoldenGate Adapters product.
Read an Oracle GoldenGate trail and write transactions to a Big Data targets. The Big Data integration features are only included in the Oracle GoldenGate for Big Data product.
Oracle GoldenGate message capture can be used to read messages from a queue and communicate with an Oracle GoldenGate Extract process to generate a trail containing the processed data.
The message capture processing is implemented as a Vendor Access Module (VAM) plug-in to a generic Extract process. A set of properties, rules and external files provide messaging connectivity information and define how messages are parsed and mapped to records in the target Oracle GoldenGate trail.
Currently this adapter supports capturing JMS text messages.
Oracle GoldenGate Java Adapter delivery can be used to apply transactional changes to targets other than a relational database: for example, ETL tools (DataStage, Ab Initio, Informatica), JMS messaging, Big Data Applications, or custom APIs. There are a variety of options for integration with Oracle GoldenGate:
Flat file integration: predominantly for ETL, proprietary or legacy applications, Oracle GoldenGate File Writer can write micro batches to disk to be consumed by tools that expect batch file input. The data is formatted to the specifications of the target application such as delimiter separated values, length delimited values, or binary. Near real-time feeds to these systems are accomplished by decreasing the time window for batch file rollover to minutes or even seconds.
Messaging: transactions or operations can be published as messages (for example, in XML) to JMS. The JMS provider is configurable to work with multiple JMS implementation; examples include ActiveMQ, JBoss Messaging, TIBCO, Oracle WebLogic JMS, WebSphere MQ, and others.
Java API: custom handlers can be written in Java to process the transaction, operation and metadata changes captured by Oracle GoldenGate on the source system. These custom Java handlers can apply these changes to a third-party Java API exposed by the target system.
Big Data integration: writing transaction data from the source trail files into various Big Data targets can be achieved by means of setting configuration properties. The Oracle GoldenGate for Big Data product contains built in Big Data handlers to write to HDFS, HBase, Kafka, and Flume targets.
All four options have been implemented as extensions to the core Oracle GoldenGate product.
For Java integration using either JMS or the Java API, use Oracle GoldenGate for Java.
For Big Data integration, you can configure Oracle GoldenGate Replicat to integrate with the Oracle GoldenGate Big Data module. Writing to Big Data targets in various formats can be configured using a set of properties with no programming required.
The Oracle GoldenGate Java Adapters, Big Data Handlers, and formatters are configured through predefined properties. These properties are stored in a separate properties file called the Adapter Properties file. Oracle GoldenGate functionality requires that the Replicat process configuration files must be in the dirprm directory and that configuration files must adhere to the following naming conventions:
Replicat process name.prm
It is considered to be a best practice that the Adapter Properties files are also located in the dirprm directory and that the Adapter Properties files adhere to one of the following naming conventions:
Replicat process name.props
or
Replicat process name.properties
All properties in Oracle GoldenGate Adapter property files are of the form:
property.name=value
Sample Oracle GoldenGate Adapter properties files are installed to the AdapterExamples subdirectory of the installation directory. These files should be copied, renamed, and the contents modified as needed and then moved to the dirprm subdirectory.
You must specify each of these property files through parameters or environmental variables as explained below. These settings allow you to change the name or location, but it is recommended that you follow the best practice for naming and location.
The following sample files are included:
ffwriter.properties
This stores the properties for the file writer. It is set with the CUSEREXIT parameter.
jmsvam.properties
This stores properties for the JMS message capture VAM. This is set with the Extract VAM parameter.
javaue.properties
This stores properties for the Java application used for message delivery. It is set through the environmental variable:
The name and location of the Adapter properties file is resolved by configuration in the Replicat process properties file.
The following explains how to resolve the name and location of the Adapter Properties file in the Replicat process.
TARGETDB LIBFILE libggjava.so SET property=dirprm/javaue.properties
Comments can be entered in the properties file with the # prefix at the beginning of the line. For example:
# This is a property comment some.property=value
Properties themselves can also be commented. This allows testing configurations without losing previous property settings.
Some properties have a variable in the property name. This allows identification of properties that are to be applied only in certain instances.
For example, you can declare more than one file writer using goldengate.flatfilewriter.writers property and then use the name of the file writer to set the properties differently:
For information on installing and configuring the core Oracle GoldenGate software for use with the Oracle GoldenGate adapter products, see the Oracle GoldenGate documentation:
Installation and Setup guides: There is one such guide for each database that is supported by Oracle GoldenGate. It contains system requirements, pre-installation and post-installation procedures, installation instructions, and other system-specific information for installing the Oracle GoldenGate replication solution.
Administering Oracle GoldenGate for Windows and UNIX: Explains how to plan for, configure, and implement the Oracle GoldenGate replication solution on the Windows and UNIX platforms.
Reference for Oracle GoldenGate for Windows and UNIX: Contains detailed information about Oracle GoldenGate parameters, commands, and functions for the Windows and UNIX platforms.