Oracle® GoldenGate Big Data and Application Adapters

1.1 New Features for 12.1.2.1.2 — Big Data

This section describes the new features for release 12.1.2.1.2.

• Big data handlers

  The Big Data handlers now supports the Three-Part Table Name and Unified Trail Format features.

• Token Retrieval Options

  Use the following combinations of new token properties to retrieve these different tokens:

  This combination:	Will retrieve these tokens:
  gg.includeggtokens=false gg.handler.name.includeTokens=true	User Tokens
  gg.includeggtokens=true gg.handler.name.includeTokens=true	User and GG Tokens
  gg.includeggtokens=true gg.handler.name.includeTokens=false	No Tokens

  
  

• Retrieval of GGS Tokens

  Use the new gg.includeggtokens=true property to retrieve GGS tokens from the data source. When tokens are available, use op.getToken(<ggstokenname>) to retrieve the value.

• Retrieval of All Available Tokens

  When gg.handler{handlername}.includeTokens=true, the handler will include all available user tokens (key value pairs) to the existing record. This will include the GGS tokens available for the corresponding record.

1.2 New Features for 12.1.2.1.1 — Application Adapters

This section describes the new features for release 12.1.2.1.1.

• Java User Exit Reporting

  Running the Java user exit in PASSTHRU mode disables statistical reporting that allows report counts to be included in the Extract processing report. As of release 12.1.2.1.1, you can trigger the user exit to collect report count statistics and send them to Extract by setting a reporting interval for the newly documented gg.report.time property. Then, if Extract is configured to print a report or a request is sent with SEND REPORT, the user exit statistics will be included.

• Simplified Logging

  Logging now defaults to the built-in Java logging, java.util.logging (JUL). Optionally, java.util.logging can be changed to either log4j or logback.

  Defaults have been put in place for each of the three possible implementations to simplify the set-up needed for logging. Once the type of logging is identified, the user is not required to enter any properties.

  By default, log files are written to the dirrpt subdirectory of the Oracle GoldenGate Adapters installation directory. The format of the log file name varies, but all three implementations use the Extract group as part of the default name.

• Grouped Transactions

  Release 12.1.2.1.1 includes the option to limit transaction size based on the number of operations. See Section 2.2.1.1, "Transaction Grouping Properties" for details on the properties.

  When using the group transactions properties, gg.handler.name.mode property and gg.handler.name.format, must be set to one transaction per message (transaction mode).

• Unified Trail Format Support

  Release 12.1.2.1.1 supports the reading of trails that are in the unified trail format. This allows 12.1.x Oracle GoldenGate trails to be processed by the Oracle GoldenGate Adapters flat file writer or user exit. Support for writing unified trail format has been deferred to the next release.

• Three-Part Table Name Support

  Release 12.1.2.1.1 supports the reading of trails that are written with three-part table names (database.schema.table). Both the Java File Writer and Flat File Writer now support three-part table names.

2.1 Changed Behavior for 12.1.2.1.4 — Big Data

By default, earlier releases of GoldenGate Big Data Adapter ignored a transaction if an operation failed processing in transaction mode. The entire transaction would be skipped, the adapter process would not abend, and the adapter would continue processing the following transactions.

Starting with release 12.1.2.1.4, the adapter process will ABEND in the event of an exception in transaction mode to prevent data loss. Fault transactions are fail-fast and will no longer be skipped. This change affects all handlers in transaction mode (operation mode handlers already exhibit this behavior).

Oracle recommends setting the goldengate.userexit.nochkpt property to false in order to skip operations or transactions that have already been processed and sent to the target. This will avoid having duplicate records.

2.2 New or Changed Properties for 12.1.2.1.1 — Application Adapters

The following properties have been added, changed, or documented for the 12.1.2.1.1 release.

3.1 Deprecated Properties for 12.1.2.1.1 — Application Adapters

This section lists the properties and property options that are deprecated for release 12.1.2.1.1.

4.1 Installation Instructions

The installation file downloaded from Oracle is not compressed using gzip; instead it is a standard zip file.

4.2 Upgrades

This section lists changes in the upgrade procedure for Oracle GoldenGate Adapters.

5.1 Release 12.1.2.1.11 — September 2, 2016 - Application Adapters

Bug 24416072 - OGG Java Adapter Hadoop not recognize iSeries multi-member table specification

Complete support for native names handling for DB2 for i was added.

Bug 24369897 - Pound Symbol Shows incorrectly when terminal character set is ISO-8859-1 1998

An output encoding property that allows you to control which encoding to use when outputting character data is introduced. This output encoding is specified with the enumeration value of the desired encoding.

5.2 Release 12.1.2.1.10 — March 28, 2016 - Application Adapters

The following tickets have been implemented for the 12.1.2.1.10 Application Adapters.

Bug DB # 2295116 - MAPQA01: Product making inactive @child ord level resulting in a Java error

Access to all user tokens and all Oracle GoldenGate tokens that are consumed by the Oracle GoldenGate Application Adapters implementation was added. This allows Oracle GoldenGate Application Adapters to access these tokens for all platforms.

5.3 Release 12.1.2.1.9 — February 23, 2016 - Application Adapters

The following ticket has been implemented for the 12.1.2.1.9 Application Adapters.

Bug DB # 22518330 - HBASE adaptor key column delimiter/separator is not configurable

The HBase Adapter was modified to add functionality such that the HBase row key delimiter is now configurable. The default value is still a pipe character (|); you can configure the rowkeyDelimiter by using the following syntax:

gg.handler.name.rowkeyDelimiter=your_delimiter

5.4 Release 12.1.2.1.8 — January 19, 2016 - Application Adapters

The following ticket has been implemented for the 12.1.2.1.8 Application Adapters.

Bug DB # 22529284 - AIX: symbol _Z19REP_get_ggs_tkn_bufvlibggjava_ue.so was referenced

AIX contains a limitation such that the adapter module will not be able to process tokens that might be inserted as part of the source trail record.

5.5 Release 12.1.2.1.7 - November 24, 2015 - Application Adapters

The following tickets have been implemented for the 12.1.2.1.7 Application Adapters.

Bug DB #22120256 - This fix preserves the value of transaction indicator flag as seen in the source trail record. The user exit implementation will no longer try to alter the value of transaction indicator flag based to match the downstream or outgoing transaction profile. There will be no logic in the user exit implementation which relies on the value of the transaction indicator flag.

Bug DB #22146222 - Fixed an issue where large source data is truncated to 4000 bytes at the target.

5.6 Release 12.1.2.1.6 — October 27, 2015 — Application Adapters

The following tickets have been implemented for the 12.1.2.1.6 Application Adapters.

Bug DB #21984176 - Fixed an issue where multiple "Begin-Transaction" operations were received for the same source.

5.7 Release 12.1.2.1.5 — October 7, 2015 — Application Adapters

The following tickets have been implemented for the 12.1.2.1.5 Application Adapters.

Bug DB #21836162 - Fixed an issue where the decimal representation of the record position deviates from what is being documented. A record position in decimal representation is (sequence number x 10,000,000,000 + RBA).

5.8 Release 12.1.2.1.4 — September 9, 2015 — Big Data

The following tickets have been implemented for the 12.1.2.1.4 Big Data.

Bug DB #21548335 - Transaction exceptions will no longer be ignored. Instead the adapter process will immediately abend to avoid loss of data. See Section 2.1, "Changed Behavior for 12.1.2.1.4 — Big Data" for more information.

5.9 Release 12.1.2.1.3 — July 20, 2015

The following tickets have been implemented for the 12.1.2.1.3 Adapters.

Bug DB #21099203 - The v12.1.2 Java Adapter correctly includes tokens in the output, in transaction mode.

5.10 Release 12.1.2.1.2 — June 10, 2015 — Big Data

The following tickets have been implemented for the 12.1.2.1.2 Big Data.

Bug DB #20432399 - GoldenGate Big Data no longer abends with exception while processing Unified Trail Format trail files.

BugDB #21208201 - GoldenGate Adapter correctly reads 12c source definition files.

5.11 Release 12.1.2.1.1 — May 1, 2015 — Application Adapters

The following tickets have been implemented for the 12.1.2.1.1 Adapters.

BugDB #20041455 - Fixed a crash due to incorrect character set handling when userexit.utf8mode=true.

BugDB #19982656 - Implemented support to handle XMLTYPE columns.

BugDB #19611993 - Fixed a crash in Solaris x86_64 due to incompatible fast compilation done in the Adapter target.

BugDB #13888220 - Implemented support to handle GUID or unique identifier columns.

BugDB #20523498 - The behavior of handler metacols transaction indicator (op.getTxState), which was changed in 11.2.1.0.3, caused errors in GETENV. To fix this we are deprecating that behavior and reverting to the original behavior.

BugDB #19898067 - The Java rolling file handler has been fixed and works correctly on size, or time, or both.

BugDB #20731562 - Fixed an IllegalArguementException on the EventObject source during a rollback event.

BugDB #20657093 - Java user exit now correctly handles different charset data.

BugDB #20639793 - Oracle GoldenGate now issues an exception when the checkpoint by base process is ahead of the one in Java exit.

BugDB #20639752 - Oracle GoldenGate now issues an exception when using grouping with multiple handlers.

BugDB #20525694 - The Java File Writer and Flat File Writer now support the reading of trails that are written with three-part table names (database.schema.table).

BugDB #19607132 - The Java handler now supports physicalDestination property in order to obtain the queue or topic object from the ConnectionFactory API instead of the JNDI provider.

BugDB #18397579 - The Oracle GoldenGate JMS Adapter now correctly encodes characters for all Cyrillic characters.

BugDB #19368122 - Oracle GoldenGate core fixed CLOB chunk retrieval logic, which affected large character object retrieved by the Adapter.

BugDB #17798874 - Oracle GoldenGate core fixed LOB chunk retrieval logic, which affected large object retrieval by the Adapter.

5.12 Release 12.1.2.1.0 — April 9, 2015 — Application Adapters

Initial release.

6.1 Issues for Release 12.1.2.1.2 — Big Data

The following are known issues for release 12.1.2.1.2 of the Oracle GoldenGate Application Adapters.

• JMS Capture may abend while parsing JMS messages that include three-part table names.

6.2 Issues for Release 12.1.2.1.1 — Application Adapters

The following are known issues for release 12.1.2.1.1 of the Oracle GoldenGate Application Adapters.

• When grouping transactions, the Java user exit may need to wait for additional operations to meet the minimum requirement to send the transaction. As the user exit waits, the Extract checkpoint moves on. If the program abends at this point, it is necessary to manually position the restart in order not to lose data. In this event, the report file and log file will print a message similar to "Alter Extract to SeqNo seq_no and RBA rba before restarting the Java Adapter."

• The underlying Extract (OGGCORE 12.1.2.1.1) now has logic to check out-of-order operations based on the sequence id of the captured records. When the process detects the out-of-order condition, it throws the error "ERROR OGG-04036 Positioning sequence ID is out of order." This can happen when the captured records in the JMS queue are coming from two (or more) different data sources, which means each data source will have its own sequence id generation potentially causing interleaved sequence ids. See BugDB #20390986 for more information.

• Tokens within Extract do not work in a Windows environment. As a workaround, you can use Replicat instead of Extract.

7.1 Documentation Changes for 12.1.2.1.1 — Application Adapters

Documentation for the Oracle GoldenGate Adapters has the following changes for release 12.1.2.1.1.

• New library pages - the documentation libraries are now organized into a task-based structure designed to make it easier to locate information about the task you are interested in.

• The Flat File and Java Adapter guides have been merged into one Administering Oracle GoldenGate Application Adapters guide. This guide is divided into three parts to cover flat file, Java message capture, and Java message delivery.

• Installation and upgrade have been separated out into separate guides.

• Book and chapter titles have been changed to be more task oriented. Each chapter now begins with a brief summary and a list of the topics included in the chapter. Each section begins with a brief summary.

• To improve the documentation for properties:

  • The variables that are part of property names are now included in the documentation. There is also an explanation of the use of variables in property names in section 1.3 of the Administering guide.

  • The property descriptions have been changed to include a syntax statement, or an example, or both.

  • The property syntax has been changed to better conform to Oracle standards by removing the curly brackets from variable names and instead denoting variables with italics.

  • An effort has been made to specify the default value for each property variable.

  • Multiple properties that were previously defined together using abbreviated names have been separated and/or changed to use complete names to allow for easier lookups.