This chapter includes the following sections:
Oracle GoldenGate message capture connects to JMS messaging to parse messages and send them through a VAM interface to an Oracle GoldenGate Extract that builds an Oracle GoldenGate trail of message data. This allows JMS messages to be delivered to an Oracle GoldenGate system running for a target database.
Using Oracle GoldenGate JMS message capture requires two components:
The dynamically linked shared VAM library that is attached to the Oracle GoldenGate Extract process.
A separate utility, Gendef, that uses the message capture properties file and parser-specific data definitions to create an Oracle GoldenGate source definitions file.
The options for configuring the three parts of message capture are:
Message connectivity: Values in the property file set connection properties such as the Java class path for the JMS client, the JMS source destination name, JNDI connection properties, and security information.
Parsing: Values in the property file set parsing rules for fixed width, comma delimited, or XML messages. This includes settings such as the delimiter to be used, values for the beginning and end of transactions and the date format.
VAM interface: Parameters that identify the VAM,
so library and a property file are set for the Oracle GoldenGate core Extract process.
The following diagram shows a typical configuration for capturing JMS messages.
In this configuration, JMS messages are picked up by the Oracle GoldenGate Adapter JMS Handler and transferred using the adapter's message capture VAM to an Extract process. The Extract writes the data to a trail which is sent over the network by a Data Pump Extract to an Oracle GoldenGate target instance. The target Replicat then uses the trail to update the target database.
Figure 3-1 Configuration for JMS Message Capture
Through the Oracle GoldenGate Java API, transactional data captured by Oracle GoldenGate can be delivered to targets other than a relational database, such as a JMS (Java Message Service), files written to disk, or an integration with a custom application Java API.
Oracle GoldenGate for Java provides the ability to execute Java code from an Oracle GoldenGate Extract process or Replicat process. Replicat is the recommended way to integrate Java delivery because of the transaction grouping functionality and improved checkpointing provided by Replicat. Using Oracle GoldenGate for Java requires two components:
A dynamically linked or shared library implemented in C or C++, integrating as a User Exit (UE). The user exit shared libraries are different for Extract and Replicat. For Extract the library is
libggjava_ue.so on Linux and UNIX; it is
gjava_ue.dll for Windows. For Replicat, the library is
libggjava.so for Linux and UNIX; it is
ggjava.dll for Windows.
A set of Java libraries (JARS) that comprise the Oracle GoldenGate Java API. This Java framework communicates with the user exit through the Java Native Interface (JNI). The interface of the Java layer is identical for running with either the Extract process or the Replicat process.
Figure 3-2 Configuration for Delivering JMS Messages
The dynamically linked library is configurable using a simple properties file. The Java framework is loaded by the user exit and is also initialized by a properties file. Application behavior can be customized by:
Set host names, port numbers, output file names, JMS connection settings;
Add/remove targets (such as JMS or files) by listing any number of active handlers to which the transactions should be sent;
Turn on/off debug-level logging, etc.
Identify which message format should be used.
(Optional) Writing custom Java code to provide custom handling of transactions and operations, do filtering, or implementing custom message formats.
There are existing implementations (handlers) for sending messages via JMS and for writing out files to disk. There are several predefined message formats for sending the messages (e.g. XML or field-delimited); or custom formats can be implemented using templates. Each handler has documentation that describes its configuration properties; for example, a file name can be specified for a file writer, and a JMS queue name can be specified for the JMS handler. Some properties apply to more than one handler; for example, the same message format can be used for JMS and files.
This section explains how to run Java Adapter with the Oracle GoldenGate Extract process.
EXTRACT hdfs discardfile ./dirrpt/avro1.dsc, purge --SOURCEDEFS ./dirdef/dbo.def CUSEREXIT libjavaue.so CUSEREXIT PASSTHRU, INCLUDEUPDATEBEFORES, PARAMS "dirprm/hdfs.props" GETUPDATEBEFORES TABLE dbo.*;
The following is explanation of the Replicat configuration entries:
EXTRACT hdfs - The Extract process name.
discardfile ./dirrpt/avro1.dsc, purge - Set the discard file
--SOURCEDEFS ./dirdef/dbo.def - Source definitions are not required for 12.2 trial files.
CUSEREXIT libjavaue.so CUSEREXIT PASSTHRU, INCLUDEUPDATEBEFORES, PARAMS "dirprm/hdfs.props" - Set you exit shared library, and point to the Java Adapter Properties file
GETUPDATEBEFORES - Get update before images.
TABLE dbo.*; - Select which tables to replicate or exclude to filter.
The Extract process provides no functionality for transaction grouping. However, transaction grouping is still possible when integrating Java Delivery with the Extract process. The Java Delivery layer enables transaction grouping with configuration in the Java Adapter properties file.
To enable grouping, the value of this property must be set to tx.
Controls the maximum number of operations that can be held by an operation group - irrespective of whether the operation group holds operations from a single transaction or multiple transactions.
The operation group will send a transaction commit and end the group as soon as this number of operations is reached. This property leads to splitting of transactions across multiple operation groups.
This is the minimum number of operations that must exist in a group before the group can end.
This property helps to avoid groups that are too small by grouping multiple small transactions into one operation group so that it can be more efficiently processed.
maxGroupSize should always be greater than or equal to
minGroupSize; that is,
maxGroupSize >= minGroupSize.
It is not recommended to use the Java layer transaction grouping when running Java Delivery with the Replicat process. If running with the Replicat process, you should use Replicat transaction grouping controlled by the
GROUPTRANSOPS Replicat property.
This section explains how to run the Java Adapter with the Oracle GoldenGate Replicat process.
The following is an example of a Replicat process properties file for Java Adapter.
REPLICAT hdfs TARGETDB LIBFILE libggjava.so SET property=dirprm/hdfs.properties --SOURCEDEFS ./dirdef/dbo.def DDL INCLUDE ALL GROUPTRANSOPS 1000 MAPEXCLUDE dbo.excludetable MAP dbo.*, TARGET dbo.*;
The following is explanation of the Replicat configuration entries:
REPLICAT hdfs - The name of the Replicat process.
TARGETDB LIBFILE libggjava.so SET property=dirprm/hdfs.properties - Names the target database as you exit
libggjava.so and sets the Java Adapters Property file to
--SOURCEDEFS ./dirdef/dbo.def - Sets a source database definitions file. Commented out because Oracle GoldenGate 18.104.22.168 trail files provide metadata in trail.
GROUPTRANSOPS 1000 - To group 1000 transactions from the source trail files into a single target transaction. This is the default and improves the performance of Big Data integrations.
MAPEXCLUDE dbo.excludetable - To identify tables to exclude.
MAP dbo.*, TARGET dbo.*; - Shows the mapping of input to output tables.
The command to add and start the Replicat process in
ggsci is the following:
ADD REPLICAT hdfs, EXTTRAIL ./dirdat/gg START hdfs
The Replicat process provides the Replicat configuration property
GROUPTRANSOPS to control transaction grouping. By default, the Replicat process implements transaction grouping of 1000 source transactions into a single target transaction. If you want to turn off transaction grouping then the
GROUPTRANSOPS Replicat property should be set to
NODBCHECKPOINT are not applicable for Java Delivery with Replicat. Beside Replicat checkpoint file (
.cpr), additional checkpoint file (
dirchk/<group>.cpj) will be created that contains information similar to
CHECKPOINTTABLE in Replicat for RDBMS.
The following Replicat features are not supported in this release:
Conflict resolution and detection (CDR)
The Oracle GoldenGate Replicat process supports mapping functionality to custom target schemas. This functionality is not available using the Oracle GoldenGate Extract process. You must use the Metadata Provider functionality to define a target schema or schemas and then use the standard Replicat mapping syntax in the Replicat configuration file to define the mapping. Refer to the Oracle GoldenGate Replicat documentation to understand the Replicat mapping syntax in the Replication configuration file.