11 Developing Custom Filters, Formatters, and Handlers

By default, all transactions, operations and metadata events are passed to the DataSourceListener event handlers. An event filter can be implemented to filter the events sent to the handlers. The filter could select certain operations on certain tables containing certain column values, for example

Filters are additive: if more than one filter is set for a handler, then all filters must return true in order for the event to be passed to the handler.

You can configure filters using the Java application properties file:

# handler "foo" only receives certain events
gg.handler.one.type=jms
gg.handler.one.format=mytemplate.vm
gg.handler.one.filter=com.mycompany.MyFilter

To activate the filter, you write the filter and set it on the handler; no additional logic needs to be added to specific handlers.

You can customize the output format of a built-in handler by:

• Writing a custom formatter in Java or

• Using a velocity template

A custom handler can be implemented by extending AbstractHandler as in the following example:

import oracle.goldengate.datasource.*;
import static oracle.goldengate.datasource.GGDataSource.Status;
public class SampleHandler extends AbstractHandler {
        @Override
        public void init(DsConfiguration conf, DsMetaData metaData) {
            super.init(conf, metaData);
            // ... do additional config...
        }
        @Override
        public Status operationAdded(DsEvent e, DsTransaction tx, DsOperation op) { ... }
        @Override
        public Status transactionCommit(DsEvent e, DsTransaction tx) { ... }
        @Override
        public Status metaDataChanged(DsEvent e, DsMetaData meta) { .... }
        @Override
        public void destroy() { /* ... do cleanup ... */ }
        @Override
        public String reportStatus() { return "status report..."; }
        @Override
        public Status ddlOperation(OpType opType, ObjectType objectType, String objectName, String ddlText) }

The method in AbstractHandler is not abstract rather it has a body. In the body it performs cached metadata invalidation by marking the metadata object as dirty. It also provides TRACE-level logging of DDL events when the ddlOperation method is specified. You can override this method in your custom handler implementations. You should always call the super method before any custom handling to ensure the functionality in AbstractHandler is executed

When a transaction is processed from the Extract, the order of calls into the handler is as follows:

1. Initialization:

   • First, the handler is constructed.

   • Next, all the "setters" are called on the instance with values from the property file.

   • Finally, the handler is initialized; the init(...) method is called before any transactions are received. It is important that the init(...) method call super.init(...) to properly initialize the base class.

2. Metadata is then received. If the Java module is processing an operation on a table not yet seen during this run, a metadata event is fired, and the metadataChanged(...) method is called. Typically, there is no need to implement this method. The DsMetaData is automatically updated with new data source metadata as it is received.

3. A transaction is started. A transaction event is fired, causing the transactionBegin(...) method on the handler to be invoked (this is not shown). This is typically not used, since the transaction has zero operations at this point.

4. Operations are added to the transaction, one after another. This causes the operationAdded(...) method to be called on the handler for each operation added. The containing transaction is also passed into the method, along with the data source metadata that contains all processed table metadata. The transaction has not yet been committed, and could be aborted before the commit is received.

   Each operation contains the column values from the transaction (possibly just the changed values when Extract is processing with compressed updates.) The column values may contain both before and after values.

   For the ddlOperation method, the options are:

   • opType - Is an enumeration that identifies the DDL operation type that is occurring (CREATE, ALTER, and so on).

   • objectType - Is an enumeration that identifies the type of the target of the DDL (TABLE, VIEW, and so on).

   • objectName - Is the fully qualified source object name; typically a fully qualified table name.

   • ddlText - Is the raw DDL text executed on the source relational database.

5. The transaction is committed. This causes the transactionCommit(...) method to be called.

6. Periodically, reportStatus may be called; it is also called at process shutdown. Typically, this displays the statistics from processing (the number of operations andtransactions processed and other details).

An example of a simple printer handler, which just prints out very basic event information for transactions, operations and metadata follows. The handler also has a property myoutput for setting the output file name; this can be set in the Java application properties file as follows:

gg.handlerlist=sample
# set properties on 'sample'
gg.handler.sample.type=sample.SampleHandler
gg.handler.sample.myoutput=out.txt

To use the custom handler,

1. Compile the class

2. Include the class in the application classpath,

3. Add the handler to the list of active handlers in the Java application properties file.

To compile the handler, include the Oracle GoldenGate for Java JARs in the classpath and place the compiled .class files in gg_install_dir/javaue/resources/classes:

javac -d gg_install_dir/dirprm
-classpath ggjava/ggjava.jar SampleHandler.java

The resulting class files would be located in resources/classes, in correct package structure, such as:

gg_install_dir/dirprm/sample/SampleHandler.class

Alternatively, custom classes can be put into a JAR and included in the classpath. Either include the custom JAR files in the JVM classpath using the Java properties (using java.class.path in the jvm.bootoptions property), or by setting the Java application properties file to include your custom JAR:

# set properties on 'one'
gg.handler.one.type=sample.SampleHandler
gg.handler.one.myoutput=out.txt
gg.classpath=/path/to/my.jar,/path/to/directory/of/jars/*

The classpath property can be set on any handler to include additional individual JARs, a directory (which would contain resources or extracted class files) or a whole directory of JARs. To include a whole directory of JARs, use the Java 6 style syntax:

c:/path/to/directory/* (or on UNIX: /path/to/directory/* )

Only the wildcard * can be specified; a file pattern cannot be used. This automatically matches all files in the directory ending with the .jar suffix. To include multiple JARs or multiple directories, you can use the system-specific path separator (on UNIX, the colon and on Windows the semicolon) or you can use platform-independent commas, as shown in the preceding example.

If the handler requires many properties to be set, just include the property in the parameter file, and your handler's corresponding "setter" will be called. For example:

gg.handler.one.type=com.mycompany.MyHandler
gg.handler.one.myOutput=out.txt
gg.handler.one.myCustomProperty=12345

The preceding example would invoke the following methods in the custom handler:

public void setMyOutput(String s) {

        // use the string...

} public void setMyCustomProperty(int j) {

        // use the int...

}

Any standard Java type may be used, such as int, long, String, boolean. For custom types, you may create a custom property editor to convert the String to your custom type.

There is Javadoc available for the Java API. The Javadoc has been intentionally reduced to a set of core packages, classes and interfaces in order to only distribute the relevant interfaces and classes useful for customizing and extension.

In each package, some classes have been intentionally omitted for clarity. The important classes are:

• oracle.goldengate.datasource.DsTransaction: represents a database transaction. A transaction contains zero or more operations.

• oracle.goldengate.datasource.DsOperation: represents a database operation (insert, update, delete). An operation contains zero or more column values representing the data-change event. Columns indexes are offset by zero in the Java API.

• oracle.goldengate.datasource.DsColumn: represents a column value. A column value is a composite of a before and an after value. A column value may be 'present' (having a value or be null) or 'missing' (is not included in the source trail).

  • oracle.goldengate.datasource.DsColumnComposite is the composite

  • oracle.goldengate.datasource.DsColumnBeforeValue is the column value before the operation (this is optional, and may not be included in the operation)

  • oracle.goldengate.datasource.DsColumnAfterValue is the value after the operation

• oracle.goldengate.datasource.meta.DsMetaData: represents all database metadata seen; initially, the object is empty. DsMetaData contains a hash map of zero or more instances of TableMetaData, using the TableName as a key.

• oracle.goldengate.datasource.meta.TableMetaData: represents all metadata for a single table; contains zero or more ColumnMetaData.

• oracle.goldengate.datasource.meta.ColumnMetaData: contains column names and data types, as defined in the database and/or in the Oracle GoldenGate source definitions file.

See the Javadoc for additional details.