1. Administering Oracle GoldenGate for Big Data
  2. Oracle GoldenGate Java Delivery
  3. Java Delivery Properties

11 Java Delivery Properties

11.1 Common Properties

The following properties are common to Java Delivery using either Replicat or Extract.

11.1.1 Logging Properties

Logging is controlled by the following properties.

11.1.1.1 gg.log

Specifies the type of logging that is to be used. The default implementation for the Oracle GoldenGate Adapters is the jdk option. This is the built-in Java logging called java.util.logging (JUL). The other logging options are log4j or logback.

For example, to set the type of logging to log4j: 

gg.log=log4j

The recommended setting is log4j. The log file is created in the dirrpt subdirectory of the installation. The default log file name includes the group name of the associated Extract and the file extension is .log.

<process name>_<log level>_log4j.log

Therefore if the Oracle GoldenGate Replicat process is called javaue, and the gg.log.level is set to debug, the resulting log file name will be:

javaue_debug_log4j.log

11.1.1.2 gg.log.level

Specifies the overall log level for all modules. The syntax is:

gg.log.level={ERROR|WARN|INFO|DEBUG|TRACE}

The log levels are defined as follows:

  • ERROR – Only write messages if errors occur

  • WARN – Write error and warning messages

  • INFO – Write error, warning and informational messages

  • DEBUG – Write all messages, including debug ones.

  • TRACE - Highest level of logging, includes all messages.

The default logging level is INFO. The messages in this case will be produced on startup, shutdown and periodically during operation. If the level is switched to DEBUG, large volumes of messages may occur which could impact performance. For example, the following sets the global logging level to INFO: 

# global logging level
gg.log.level=INFO
11.1.1.3 gg.log.file

Specifies the path to the log file. The syntax is:

gg.log.file=path_to_file

Where the path_to_file is the fully defined location of the log file. This allows a change to the name of the log, but you must include the Replicat name if you have more than one Replicat to avoid one overwriting the log of the other.

11.1.1.4 gg.log.classpath

Specifies the classpath to the JARs used to implement logging. This configuration property is not typically used as the ggjava.jar library includes the required logging dependency libraries. 

gg.log.classpath=path_to_jars

11.1.2 JVM Boot Options

The following options configure the Java Runtime Environment. Java classpath and memory options are configurable.

11.1.2.1 jvm.bootoptions

Specifies the initial Java classpath and other boot options that will be applied when the JVM starts. The java.class.path needs colon (:) separators for UNIX/Linux and semicolons (;) for Windows. This is where to specify various options for the JVM, such as initial and maximum heap size and classpath; for example:

  • -Xms: initial java heap size

  • -Xmx: maximum java heap size

  • -Djava.class.path: classpath specifying location of at least the main application JAR, ggjava.jar. Other JARs, such as JMS provider JARs, may also be specified here as well; alternatively, these may be specified in the Java application properties file. If using a separate log4j properties file then the location of the properties file must be included in the bootoptions java.class.path included in the bootoptions variable.

  • -verbose:jni: run in verbose mode (for JNI)

For example (all on a single line):

jvm.bootoptions= -Djava.class.path=ggjava/ggjava.jar 
-Dlog4j.configuration=my-log4j.properties -Xmx512m

The log4j.configuration property identifies a log4j properties file that is resolved by searching the classpath. You may use your own log4j configuration, or one of the preconfigured log4j settings: log4j.properties (default level of logging), debug-log4j.properties (debug logging) or trace-log4j.properties (very verbose logging). To use log4j logging with the Replicat process gg.log=log4j must be set.

Use of the one of the preconfigured log4j settings does not require any change to the classpath since those files are already included in the classpath. The -Djava.class.path variable must include the path to the directory containing a custom log4j configuration file without the * wild card appended.

11.2 Delivery Properties

The following properties are available to Java Delivery:

11.2.1 General Properties

The following properties apply to all writer configurations:

11.2.1.1 goldengate.userexit.writers

Specifies the name of the writer. This is always jvm and should not be modified.

For example:

goldengate.userexit.writers=jvm

All other properties in the file should be prefixed by the writer name, jvm.

11.2.1.2 goldengate.userexit.chkptprefix

Specifies a string value for the prefix added to the Java checkpoint file name. For example:

goldengate.userexit.chkptprefix=javaue_

11.2.1.3 goldengate.userexit.nochkpt

Disables or enables the checkpoint file. The default is false, the checkpoint file is enabled. Set this property to true if transactions are supported and enabled on the target.

For example, Java Adapter Properties if JMS is the target and JMS local transactions are enabled (the default), set goldengate.userexit.nochkpt=true to disable the user exit checkpoint file. If JMS transactions are disabled by setting localTx=false on the handler, the checkpoint file should be enabled by setting goldengate.userexit.nochkpt=false.

goldengate.userexit.nochkpt=true|false

11.2.1.4 goldengate.userexit.usetargetcols

Specifies whether or not mapping to target columns is allowed. The default is false, no target mapping.

goldengate.userexit.usetargetcols=true|false

11.2.2 Statistics and Reporting

Disables or enables the checkpoint file handling. This causes the standard Oracle GoldenGate reporting to be incomplete. Oracle GoldenGate for Java adds its own reporting to handle this issue.

Statistics can be reported every t seconds or every n records - or if both are specified, whichever criteria is met first.

There are two sets of statistics recorded: those maintained by the Replicat module and those obtained from the Java module. The reports received from the Java side are formatted and returned by the individual handlers.

The statistics include the total number of operations, transactions and corresponding rates.

11.2.2.1 jvm.stats.time | jvm.stats.numrecs

Specifies a time interval, in seconds or a number of records, after which statistics will be reported. The default is to report statistics every hour or every 10000 records (which ever occurs first).

For example, to report ever 10 minutes or every 1000 records, specify:

jvm.stats.time=600
jvm.stats.numrecs=1000

The Java application statistics are handler-dependent:

  • For the all handlers, there is at least the total elapsed time, processing time, number of operations, transactions;

  • For the JMS handler, there is additionally the total number of bytes received and sent.

  • The report can be customized using a template.

11.3 Java Application Properties

The following defines the properties which may be set in the Java application property file.

11.3.1 Properties for All Handlers

The following properties apply to all handlers:

11.3.1.1 gg.handlerlist

The handler list is a list of active handlers separated by commas. These values are used in the rest of the property file to configure the individual handlers. For example:

gg.handlerlist=name1, name2
gg.handler.name1.propertyA=value1
gg.handler.name1.propertyB=value2
gg.handler.name1.propertyC=value3
gg.handler.name2.propertyA=value1
gg.handler.name2.propertyB=value2
gg.handler.name2.propertyC=value3

Using the handlerlist property, you may include completely configured handlers in the property file and just disable them by removing them from the handlerlist.

11.3.1.2 gg.handler.name.type

This type of handler. This is either a predefined value for built-in handlers, or a fully qualified Java class name. The syntax is:

gg.handler.name.type={jms|jms_map|aq|singlefile|rollingfile|custom_java_class}

Where:

All but the last are pre-defined handlers:

  • jms – Sends transactions, operations, and metadata as formatted messages to a JMS provider

  • aq – Sends transactions, operations, and metadata as formatted messages to Oracle Advanced Queuing (AQ)

  • jms_map – Sends JMS map messages

  • singlefile – Writes to a single file on disk, but does not roll the file

  • rollingfile – Writes transactions, operations, and metadata to a file on disk, rolling the file over after a certain size, amount of time, or both. For example: 

    gg.handler.name1.rolloverSize=5000000
gg.handler.name1.rolloverTime=1m

  • custom_java_class – Any class that extends the Oracle GoldenGate for Java AbstractHandler class and can handle transaction, operation, or metadata events

    The Oracle GoldenGate foe Big Data package also contains more predefined handlers to write to various Big Data targets.

11.3.2 Properties for Formatted Output

The following properties apply to all handlers capable of producing formatted output; this includes:

  • The jms_text handler (but not the jms_map handler)

  • The aq handler

  • The singlefile and rolling handlers, for writing formatted output to files

  • The predefined Big Data handlers

11.3.2.1 gg.handler.name.format

Specifies the format used to transform operations and transactions into messages sent to JMS, to the Big Data target or to a file. The format is specified uniquely for each handler. The value may be:

  • Velocity template

  • Java class name (fully qualified - the class specified must be a type of formatter)

  • csv for delimited values (such as comma separated values; the delimiter can be customized)

  • fixed for fixed-length fields

  • Built-in formatter, such as:

    • xml – demo XML format

    • xml2 – internal XML format

For example, to specify a custom Java class:

gg.handlerlist=abc
gg.handler.abc.format=com.mycompany.MyFormat

Or, for a Velocity template:

gg.handlerlist=xyz
gg.handler.xyz.format=path/to/sample.vm

If using templates, the file is found relative to some directory or JAR that is in the classpath. By default, the Oracle GoldenGate installation directory is in the classpath, so the preceding template could be placed in the dirprm directory of the Oracle GoldenGate installation location.

The default format is to use the built-in XML formatter.

11.3.2.2 gg.handler.name.includeTables

Specifies a list of tables this handler will include.

If the schema (or owner) of the table is specified, then only that schema matches the table name; otherwise, the table name matches any schema. A comma separated list of tables can be specified. For example, to have the handler only process tables foo.customer and bar.orders: 

gg.handler.myhandler.includeTables=foo.customer, bar.orders

If the catalog and schema (or owner) of the table are specified, then only that catalog and schema matches the table name; otherwise, the table name matches any catalog and schema. A comma separated list of tables can be specified. For example, to have the handler only process tables dbo.foo.customer and dbo.bar.orders: 

gg.handler.myhandler.includeTables=dbo.foo.customer, dbo.bar.orders

Note:

In order to selectively process operations on a table by table basis, the handler must be processing in operation mode. If the handler is processing in transaction mode, then when a single transaction contains several operations spanning several tables, if any table matches the include list of tables, the transaction will be included.

11.3.2.3 gg.handler.name.excludeTables

Specifies a list of tables this handler will exclude.

If the schema (or owner) of the table is specified, then only that schema matches the table name; otherwise, the table name matches any schema. A list of tables may be specified, comma-separated. For example, to have the handler process all operations on all tables except table date_modified in all schemas: 

gg.handler.myhandler.excludeTables=date_modified

If the catalog and schema (or owner) of the table are specified, then only that catalog and schema matches the table name; otherwise, the table name matches any catalog and schema. A list of tables may be specified, comma-separated. For example, to have the handler process all operations on all tables except table date_modified in catalog dbo and schema bar: 

gg.handler.myhandler.excludeTables=dbo.bar.date_modified
11.3.2.4 gg.handler.name.mode, gg.handler.name.format.mode

Specifies whether to output one operation per message (op) or one transaction per message (tx). The default is op. Use gg.handler.name.format.mode when you have a custom formatter.

11.3.3 Properties for CSV and Fixed Format Output

If the handler is set to use either comma separated values (CSV) CSV or fixed format output, the following properties may also be set.

11.3.3.1 gg.handler.name.format.delim

Specifies the delimiter to use between fields. Set this to no value to have no delimiter used. For example:

gg.handler.handler1.format.delim=,
11.3.3.2 gg.handler.name.format.quote

Specifies the quote character to be used if column values are quoted. For example:

gg.handler.handler1.format.quote='
11.3.3.3 gg.handler.name.format.metacols

Specifies the metadata column values to appear at the beginning of the record, before any column data. Specify any of the following, in the order they should appear:

  • position – unique position indicator of records in a trail

  • opcode – I, U, or D for insert, update, or delete records (see: insertChar, updateChar, deleteChar)

  • txind – transaction indicator – such as 0=begin, 1=middle, 2=end, 3=whole tx (see beginTxChar, middleTxChar, endTxChar, wholeTxChar)

  • opcount – position of a record in a transaction, starting from 0

  • catalog – catalog of the schema for the record

  • schema – schema/owner of the table for the record

  • tableonly – just table (no schema/owner)

  • table – full name of table, catalog.schema.table

  • timestamp – commit timestamp of record

For example:

gg.handler.handler1.format.metacols=opcode, table, txind, position
11.3.3.4 gg.handler.name.format.missingColumnChar

Specifies a special column prefix for a column value that was not captured from the source database transaction log. The column value is not in trail and it is unknown if it has a value or is NULL

The character used to represent the missing state of the column value can be customized. For example:

gg.handler.handler1.format.missingColumnChar=M

By default, the missing column value is set to an empty string and does not show.

11.3.3.5 gg.handler.name.format.presentColumnChar

Specifies a special column prefix for a column value that exists in the trail and is not NULL.

The character used to represent the state of the column can be customized. For example:

gg.handler.handler1.format.presentColumnChar=P

By default, the present column value is set to an empty string and does not show.

11.3.3.6 gg.handler.name.format.nullColumnChar

Specifies a special column prefix for a column value that exists in the trail and is set to NULL.

The character used to represent the state of the column can be customized. For example:

gg.handler.handler1.format.nullColumnChar=N

By default, the null column value is set to an empty string and does not show.

11.3.3.7 gg.handler.name.format.beginTxChar

Specifies the header metadata character (see metacols) used to identify a record as the begin of a transaction. For example: 

gg.handler.handler1.format.beginTxChar=B
11.3.3.8 gg.handler.name.format.middleTxChar

Specifies the header metadata characters (see metacols) used to identify a record as the middle of a transaction. For example: 

gg.handler.handler1.format.middleTxChar=M
11.3.3.9 gg.handler.name.format.endTxChar

Specifies the header metadata characters (see metacols) used to identify a record as the end of a transaction. For example: 

gg.handler.handler1.format.endTxChar=E
11.3.3.10 gg.handler.name.format.wholeTxChar

Specifies the header metadata characters (see metacols) used to identify a record as a complete transaction; referred to as a whole transaction. For example: 

gg.handler.handler1.format.wholeTxChar=W
11.3.3.11 gg.handler.name.format.insertChar

Specifies the character to identify an insert operation. The default I.

For example, to use INS instead of I for insert operations: 

gg.handler.handler1.format.insertChar=INS
11.3.3.12 gg.handler.name.format.updateChar

Specifies the character to identify an update operation. The default is U.

For example, to use UPD instead of U for update operations: 

gg.handler.handler1.format.updateChar=UPD
11.3.3.13 gg.handler.name.format.deleteChar

Specifies the character to identify a delete operation. The default is D.

For example, to use DEL instead of D for delete operations: 

gg.handler.handler1.format.deleteChar=DEL
11.3.3.14 gg.handler.name.format.truncateChar

Specifies the character to identify a truncate operation. The default is T.

For example, to use TRUNC instead of T for truncate operations: 

gg.handler.handler1.format.truncateChar=TRUNC
11.3.3.15 gg.handler.name.format.endOfLine

Specifies the end-of-line character as:

  • EOL - Native platform

  • CR - Neutral (UNIX-style \n)

  • CRLF - Windows (\r\n)

For example:

gg.handler.handler1.format.endOfLine=CR
11.3.3.16 gg.handler.name.format.justify

Specifies the left or right justification of fixed fields. For example:

gg.handler.handler1.format.justify=left
11.3.3.17 gg.handler.name.format.includeBefores

Controls whether before images should be included in the output. There must be before images in the trail. For example:

gg.handler.handler1.format.includeBefores=false

11.3.4 File Writer Properties

The following properties only apply to handlers that write their output to files: the file handler and the singlefile handler.

11.3.4.1 gg.handler.name.file

Specifies the name of the output file for the given handler. If the handler is a rolling file, this name is used to derive the rolled file names. The default file name is output.xml.

11.3.4.2 gg.handler.name.append

Controls whether the file should be appended to (true) or overwritten upon restart (false).

11.3.4.3 gg.handler.name.rolloverSize

If using the file handler, this specifies the size of the file before a rollover should be attempted. The file size will be at least this size, but will most likely be larger. Operations and transactions are not broken across files. The size is specified in bytes, but a suffix may be given to identify MB or KB. For example:

gg.handler.myfile.rolloverSize=5MB

The default rollover size is 10MB.

11.3.5 JMS Handler Properties

The following properties apply to the JMS handlers. Some of these values may be defined in the Java application properties file using the name of the handler. Other properties may be placed into a separate JMS properties file, which is useful if using more than one JMS handler at a time. For example:

gg.handler.myjms.type=jms_text
gg.handler.myjms.format=xml
gg.handler.myjms.properties=weblogic.properties

Just as with Velocity templates and formatting property files, this additional JMS properties file is found in the classpath. The preceding properties file weblogic.properties would be found in {gg_install_dir}/dirprm/weblogic.properties, since the dirprm directory is included by default in the classpath.

Settings that can be made in the Java application properties file will override the corresponding value set in the supplemental JMS properties file (weblogic.properties in the preceding example). In the following example, the destination property is specified in the Java application properties file. This allows the same default connection information for the two handlers myjms1 and myjms2, but customizes the target destination queue. 

gg.handlerlist=myjms1,myjms2
gg.handler.myjms1.type=jms_text
gg.handler.myjms1.destination=queue.sampleA
gg.handler.myjms1.format=sample.vm
gg.handler.myjms1.properties=tibco-default.properties
gg.handler.myjms2.type=jms_map
gg.handler.myjms2.destination=queue.sampleB
gg.handler.myjms2.properties=tibco-default.properties

To set a property, specify the handler name as a prefix; for example:

gg.handlerlist=sample
gg.handler.sample.type=jms_text
gg.handler.sample.format=my_template.vm
gg.handler.sample.destination=gg.myqueue
gg.handler.sample.queueortopic=queue
gg.handler.sample.connectionUrl=tcp://host:61616?jms.useAsyncSend=true
gg.handler.sample.useJndi=false
gg.handler.sample.connectionFactory=ConnectionFactory
gg.handler.sample.connectionFactoryClass=\
    org.apache.activemq.ActiveMQConnectionFactory
gg.handler.sample.timeToLive=50000
11.3.5.1 Standard JMS Settings

The following outlines the JMS properties which may be set, and the accepted values. These apply for both JMS handler types: jms_text (TextMessage) and jms_map (MapMessage).

11.3.5.1.1 gg.handler.name.destination

The queue or topic to which the message is sent. This must be correctly configured on the JMS server. Typical values may be: queue/A, queue.Test, example.MyTopic, etc. 

gg.handler.name.destination=queue_or_topic
11.3.5.1.2 gg.handler.name.user

(Optional) User name required to send messages to the JMS server.

gg.handler.name.user=user_name
11.3.5.1.3 gg.handler.name.password

(Optional) Password required to send messages to the JMS server 

gg.handler.name.password=password
11.3.5.1.4 gg.handler.name.queueOrTopic

Whether the handler is sending to a queue (a single receiver) or a topic (publish / subscribe). This must be correctly configured in the JMS provider. This property is an alias of gg.handler.name.destination. The syntax is: 

gg.handler.name.queueOrTopic=queue|topic

Where:

  • queue – a message is removed from the queue once it has been read. This is the default.

  • topic – messages are published and may be delivered to multiple subscribers.

11.3.5.1.5 gg.handler.name.persistent

If the delivery mode is set to persistent or not. If the messages are to be persistent, the JMS provider must be configured to log the message to stable storage as part of the client's send operation. The syntax is:

gg.handler.name.persistent={true|false}
11.3.5.1.6 gg.handler.name.priority

JMS defines a 10 level priority value, with 0 as the lowest and 9 as the highest. Priority is set to 4 by default. The syntax is:

gg.handler.name.priority=integer

For example:

gg.handler.name.priority=5
11.3.5.1.7 gg.handler.name.timeToLive

The length of time in milliseconds from its dispatch time that a produced message should be retained by the message system. A value of zero specifies the time is unlimited. The default is zero. The syntax is:

gg.handler.name.timeToLive=milliseconds

For example:

gg.handler.name.timeToLive= 36000
11.3.5.1.8 gg.handler.name.connectionFactory

Name of the connection factory to lookup using JNDI. ConnectionFactoryJNDIName is an alias. The syntax is: 

gg.handler.name.connectionFactory=JNDI_name
11.3.5.1.9 gg.handler.name.useJndi

If gg.handler.name.usejndi is false, then JNDI is not used to configure the JMS client. Instead, factories and connections are explicitly constructed. The syntax is: 

gg.handler.name.useJndi=true|false
11.3.5.1.10 gg.handler.name.connectionUrl

Connection URL is used only when not using JNDI to explicitly create the connection. The syntax is:

gg.handler.name.connectionUrl=url
11.3.5.1.11 gg.handler.name.connectionFactoryClass

The Connection Factory Class is used to access a factory only when not using JNDI. The value of this property is the Java class name to instantiate; constructing a factory object explicitly. 

gg.handler.name.connectionFactoryClass=java_class_name
11.3.5.1.12 gg.handler.name.localTX

Specifies whether or not local transactions are used. The default is true, local transactions are used. The syntax is: 

gg.handler.name.localTX=true|false
11.3.5.1.13 gg.handlerlist.nop

Disables the sending of JMS messages to allow testing of message generation. This is a global property used only for testing. The events are still generated and handled and the message is constructed. The default is false; do not disable message send. The syntax is: 

gg.handlerlist.nop=true|false

Users can take advantage of this option to measure the performance of trail records processing without involving the handler module. This approach can narrow down the possible culprits of a suspected performance issue while applying trail records to the target system.

11.3.5.1.14 gg.handler.name.physicalDestination

Name of the queue or topic object, obtained through the ConnectionFactory API instead of the JNDI provider. 

gg.handler.name.physicalDestination=queue_name

11.3.5.2 Group Transaction Properties

These properties set limits for grouping transactions.

11.3.6 JNDI Properties

These JNDI properties are required for connection to an Initial Context to look up the connection factory and initial destination. 

java.naming.provider.url=url
java.naming.factory.initial=java-class-name

If JNDI security is enabled, the following properties may be set: 

java.naming.security.principal=user-name
java.naming.security.credentials=password-or-other-authenticator

For example:

java.naming.provider.url= t3://localhost:7001
java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory
java.naming.security.principal=jndiuser
java.naming.security.credentials=jndipw

11.3.7 General Properties

The following are general properties that are used for the Java framework:

11.3.7.1 gg.classpath

Specifies a comma delimited list of additional paths to directories or JARs to add to the classpath. Optionally, the list can be delimited by semicolons for Windows systems or by colons for UNIX. For example:

gg.classpath=C:\Program Files\MyProgram\bin;C:\Program Files\ProgramB\app\bin;

This Adapter properties file configuration property should be used to configure pathing to custom Java JARs or to the external dependencies of Big Data applications.

11.3.7.2 gg.report.time

Specifies how often statistics are calculated and sent to Extract for the processing report. If Extract is configured to print a report, these statistics are included. The syntax is:

gg.report.time=report_interval{s|m|h}

Where:

  • report_interval is an integer

  • The valid time units are:

    • s - seconds

    • m - minutes

    • h - hours

If no value is entered, the default is to calculate and send every 24 hours.

11.3.7.3 gg.binaryencoding

Specifies the binary encoding type. The desired output encoding for binary data can be configured using this property. For example:

gg.binaryencoding=base64|hex

The default value is base64. The valid values to represent binary data are:

  • base64 - a base64 string 

  • hex - a hexadecimal string

11.3.8 Java Delivery Transaction Grouping

Transaction grouping can significantly improve the performance of Java integrations especially Big Data integrations. Java Delivery provides functionality to perform transaction grouping. When Java Delivery is hosted by a Replicat process then the GROUPTRANSOPS Replicat configuration should be used to perform transaction grouping.