7 Configuring Java Delivery

This chapter includes the following sections:

7.1 Configuring the JRE in the Properties File

The current release of Oracle GoldenGate Java Delivery requires Java 7. Refer to the section on configuring Java for how to correctly access Java and the required Java shared libraries. Modify the Adapter Properties file to point to the location of the Oracle GoldenGate for Java main JAR (ggjava.jar) and set any additional JVM runtime boot options as required (these are passed directly to the JVM at startup):

jvm.bootoptions=-Djava.class.path=.:ggjava/ggjava.jar-Xmx512m -Xmx64m

Note the following options in particular:

  • java.class.path must include pathing to the core application (ggjava/ggjava.jar). The current directory (.) should be included as well in the classpath. Logging initializes when the JVM is loaded therefore the java.class.path variable should including any pathing to logging properties files (such as log4j properties files). The dependency JARs required for logging functionality are included in ggjava.jar and do not need to be explicitly included. Pathing can reference files and directories relative to the Oracle GoldenGate install directory, to allow storing Java property files, Velocity templates and other classpath resources in the dirprm subdirectory. It is also possible to append to the classpath in the Java application properties file. Pathing to handler dependency JARs can be added here as well. However, it is considered to be a better practice to use the gg.classpath variable to include any handler dependencies.

  • The jvm.bootoptions property also allows you to control the initial heap size of the JVM (Xms) and the maximum heap size of the JVM (Xmx). Increasing the maximum heap size can improve performance by requiring less frequent garbage collections. Additionally, you may need to increase the maximum heap size if a Java out of memory exception occurs.

Once the properties file is correctly configured for your system, it usually remains unchanged. See Common Properties, for additional configuration options.

7.2 Configuring Oracle GoldenGate for Java Delivery

Java Delivery is compatible with either the Oracle GoldenGate Replicat process or the Oracle GoldenGate Extract (Data Pump) process. The use of the Oracle GoldenGate Replicat process is considered the best practice. Using either process the functionality is similar. Transaction data is read from the Oracle GoldenGate trail files and delivered to the Java Delivery user exit library. The user exit library transfers the data across the JNI interface to the Oracle GoldenGate Java Delivery module. The Java Delivery module is configurable to allow data to be streamed into various targets. The supported targets for the Oracle GoldenGate Java Adapter product include JMS, file writing, and custom integrations. The Oracle GoldenGate for Big Data product includes all of those integrations plus streaming capabilities to Big Data targets such as HDFS, HBase, Flume, and Kafka.

There are subtle differences between using the Replicat process or Extract process. You need to review the documentation closely when deploying new integrations or when converting a previous integration from the Extract process to the Replicat process.

7.2.1 Configuring a Replicat for Java Delivery

The Oracle GoldenGate Replicat process can be configured to send transaction data to the Oracle GoldenGate for Java module. Replicat consumes a local trail (for example dirdat/aa) and sends the data to the Java Delivery module. The Java module is responsible for processing all the data and applying it to the desired target.

Following is an example of adding a Replicat process:

ADD REPLICAT javarep, EXTTRAIL ./dirdat/aa

The process names and trail names used in the preceding example can be replaced with any valid name. Process names must be 8 characters or less, trail names must to be two characters. In the Replicat parameter file (javarep.prm), specify the location of the user exit library.

The Replicat process has transaction grouping built into the application. Transaction grouping can significantly improve performance when streaming data to a target database. Transaction grouping can also significantly improve performance when streaming data to Big Data applications. The Replicat parameter to control transaction grouping is the GROUPTRANSOPS variable in the Replicat configuration file. The default value of this variable is 1000 which means the Replicat process will attempt to group 1000 operations into single target transaction. Performance testing has generally shown that the higher the GROUPTRANSOPS the better the performance when streaming data to Big Data applications. Setting the GROUPTRANSOPS variable to 1 means that the original transaction boundaries from the source trail file (source database) will be maintained.

The Oracle GoldenGate Java Delivery layer also provides functionality for transaction grouping. It is not considered to be a good practice to use the Java Delivery layer grouping functionality if a Replicat process is hosting the Java Delivery. The Java Delivery grouping functionality should only be used when Oracle GoldenGate Java Delivery is hosted in an Extract process.

Table 7-1 User Exit Replicat Parameters

Parameter Explanation 
REPLICAT javarep

All Replicat parameter files start with the Replicat name

 
SOURCEDEFS ./dirdef/tcust.def

(Optional) If the input trail files do not contain the metadata records, the Replicat process requires metadata describing the trail data. This can come from a database or a source definitions file. This metadata defines the column names and data types in the trail being read (./dirdat/aa).

 
TARGETDB LIBFILE libggjava.so SET properties= dirprm/javarep.properties

The TARGETDB LIBFILE libggjava.so parameter serves as a trigger to initialize the Java module. The SET clause to specify the Java properties file is optional. If specified, it should contain an absolute or relative path (relative to the Replicat executable) to the properties file for the Java module. The default value is replicat_name.properties in the dirprm directory.

 
MAP schema.*, TARGET *.*;

The tables to pass to the Java module; tables not included will be skipped. If mapping from source to target tables is required, one can use the MAP source_specification TARGET target_specification as describe in "Mapping and Manipulating Data" in Oracle GoldenGate Administering Oracle GoldenGate for Windows and UNIX.

DDL INCLUDE ALLGROUPTRANSOPS 1000

Group source transactions into a single larger target transaction for improved performance. GROUPTRANSOPS of 1000 is the default setting that maps up to 1000 source transactions into a single target transaction.

7.2.2 Configuring an Extract process (Data Pump) for Java Delivery

Java delivery can also be configured as an Extract (data pump) process. The data pump reads from a local trail (for example dirdat/aa) and sends the data to the Java Delivery user exit. The user exit is responsible for processing all the data.

Following is an example of adding a data pump Extract:

ADD EXTRACT javaue, EXTTRAILSOURCE ./dirdat/aa

The process names and trail names used in the preceding example can be replaced with any valid name. Process names must be 8 characters or less, trail names must to be two characters. In the user exit Extract parameter file (javaue.prm) specify the location of the user exit library.

The Oracle GoldenGate Extract process provides no transaction grouping functionality. If transaction grouping functionality is required for Java Delivery hosted in an Extract process then transaction grouping in the Java Delivery layer must be used.

Table 7-2 User Exit Extract Parameters

Parameter Explanation 
EXTRACT javaue

All Extract parameter files start with the Extract name

 
SOURCEDEFS ./dirdef/tcust.def

(Optional) If the input trail files do not contain the metadata records, the Extract process requires metadata describing the trail data. This can come from a database or a source definitions file. This metadata defines the column names and data types in the trail being read (./dirdat/aa).

 
SETENV (GGS_USEREXIT_CONF = "dirprm/javaue.properties")

(Optional) An absolute or relative path (relative to the Extract executable) to the properties file for the C user exit library. The default value is extract_name.properties in the dirprm directory.

 
SETENV (GGS_JAVAUSEREXIT_CONF = "dirprm/javaue.properties")

(Optional) The Java properties.This example places the properties file in the dirprm directory.

 
CUSEREXIT ggjava_ue.dll CUSEREXIT
PASSTHRU
INCLUDEUPDATEBEFORES

The CUSEREXIT parameter includes the following:

  • The location of the user exit library. For UNIX, the library would be suffixed .so

  • CUSEREXIT - the callback function name that must be uppercase

  • PASSTHRU - avoids the need for a dummy target trail.

  • INCLUDEUPDATEBEFORES - needed for transaction integrity.

 
TABLE schema.*;

The tables to pass to the user exit; tables not included will be skipped. No filtering may be done in the user exit Extract; otherwise transaction markers will be missed. You can filter in the primary Extract, or use another, upstream data pump, or filter data directly in the Java application.

Note:

Using PASSTHRU disables the statistical reporting that allows report counts to be included in the processing report. To collect report count statistics and send them to the Extract from the user exit, use the property gg.report.time.

  • SETENV (GGS_USEREXIT_CONF = "dirprm/javaue.properties")

    This changes the default configuration file used for the user exit shared library. The value given is either an absolute path, or a path relative to Extract. The preceding example uses a relative path to put this property file in the dirprm directory.

    The default file used is extract_name.properties, located in the dirprm directory. So, if the Extract name is pumpA, then the .prm file is dirprm/pumpA.prm and the properties file is dirprm/pumpA.properties.

  • SETENV (GGS_JAVAUSEREXIT_CONF = "dirprm/javaue.properties")

    This changes the default properties file used for the Oracle GoldenGate for Java framework. The value found is a path to a file found in the classpath or in a normal file system path.

    Both GGS_USEREXIT_CONF and GGS_JAVAUSEREXIT_CONF default to the same file; dirprm/extract_name.properties.

7.3 Configuring the Java Handlers

The Handlers are integrations with target applications which plug into the Oracle GoldenGate Java Delivery module. It is the Java Handlers which provide the functionality to push data to integration targets such as JMS or Big Data applications. The Java Adapter properties file is used to configure Java Delivery and Java handlers. To test the configuration, users may use the built-in file handler. Here are some example properties, followed by explanations of the properties (comment lines start with #):

# the list of active handlers
gg.handlerlist=myhandler
# set properties on 'myhandler'
gg.handler.myhandler.type=file
gg.handler.myhandler.format=tx2xml.vm
gg.handler.myhandler.file=output.xml

This property file declares the following:

  • Active event handlers. In the example a single event handler is active, called myhandler. Multiple handlers may be specified, separated by commas. For example: gg.handlerlist=myhandler, yourhandler

  • Configuration of the handlers. In the example myhandler is declared to be a file type of handler: gg.handler.myhandler.type=file

    Note:

    See the documentation for each type of handler (for example, the JMS handler or the file writer handler) for the list of valid properties that can be set.

  • The format of the output is defined by the Velocity template tx2xml.vm. You may specify your own custom template to define the message format; just specify the path to your template relative to the Java classpath.

This property file is actually a complete example that will write captured transactions to the output file output.xml. Other handler types can be specified using the keywords: jms_text (or jms), jms_map, singlefile (a file that does not roll), and others. Custom handlers can be implemented, in which case the type would be the fully qualified name of the Java class for the handler. Oracle GoldenGate Big Data package also contains built in Big Data target types. For more information, see Integrating Oracle GoldenGate for Big Data.

Note:

See the documentation for each type of handler (for example, the JMS handler or the file writer handler) for the list of valid properties that can be set.