Apache Cassandra

The Cassandra Handler provides the interface to Apache Cassandra databases.

This chapter describes how to use the Cassandra Handler.

Overview

Apache Cassandra is a NoSQL Database Management System designed to store large amounts of data. A Cassandra cluster configuration provides horizontal scaling and replication of data across multiple machines. It can provide high availability and eliminate a single point of failure by replicating data to multiple nodes within a Cassandra cluster. Apache Cassandra is open source and designed to run on low-cost commodity hardware.

Cassandra relaxes the axioms of a traditional relational database management systems (RDBMS) regarding atomicity, consistency, isolation, and durability. When considering implementing Cassandra, it is important to understand its differences from a traditional RDBMS and how those differences affect your specific use case.

Cassandra provides eventual consistency. Under the eventual consistency model, accessing the state of data for a specific row eventually returns the latest state of the data for that row as defined by the most recent change. However, there may be a latency period between the creation and modification of the state of a row and what is returned when the state of that row is queried. The benefit of eventual consistency is that the latency period is predicted based on your Cassandra configuration and the level of work load that your Cassandra cluster is currently under, see http://cassandra.apache.org/.

The Cassandra Handler provides some control over consistency with the configuration of the gg.handler.name.consistencyLevel property in the Java Adapter properties file.

Detailing the Functionality

About the Cassandra Data Types

Cassandra provides a number of column data types and most of these data types are supported by the Cassandra Handler.

Supported Cassandra Data Types

ASCII
BIGINT
BLOB
BOOLEAN
DATE
DECIMAL
DOUBLE
DURATION
FLOAT
INET
INT
SMALLINT
TEXT
TIME
TIMESTAMP
TIMEUUID
TINYINT
UUID
VARCHAR
VARINT

Unsupported Cassandra Data Types

COUNTER
MAP
SET
LIST
UDT (user defined type)
TUPLE
CUSTOM_TYPE

Supported Database Operations

INSERT
UPDATE (captured as INSERT)
DELETE

The Cassandra commit log files do not record any before images for the UPDATE or DELETE operations. So the captured operations never have a before image section. Oracle GoldenGate features that rely on before image records, such as Conflict Detection and Resolution, are not available.

Unsupported Database Operations

TRUNCATE
DDL (CREATE, ALTER, DROP)

The data type of the column value in the source trail file must be converted to the corresponding Java type representing the Cassandra column type in the Cassandra Handler. This data conversion introduces the risk of a runtime conversion error. A poorly mapped field (such as varchar as the source containing alpha numeric data to a Cassandra int) may cause a runtime error and cause the Cassandra Handler to abend. You can view the Cassandra Java type mappings at: DataStax Documentation

It is possible that the data may require specialized processing to get converted to the corresponding Java type for intake into Cassandra. If this is the case, you have two options:

About Catalog, Schema, Table, and Column Name Mapping

Traditional RDBMSs separate structured data into tables. Related tables are included in higher-level collections called databases. Cassandra contains both of these concepts. Tables in an RDBMS are also tables in Cassandra, while database schemas in an RDBMS are keyspaces in Cassandra.

It is important to understand how data maps from the metadata definition in the source trail file are mapped to the corresponding keyspace and table in Cassandra. Source tables are generally either two-part names defined as schema.table,or three-part names defined as catalog.schema.table.

The following table explains how catalog, schema, and table names map into Cassandra. Unless you use special syntax, Cassandra converts all keyspace, table names, and column names to lower case.

Table Name in Source Trail File Cassandra Keyspace Name Cassandra Table Name
QASOURCE.TCUSTMER qasource tcustmer
dbo.mytable dbo mytable
GG.QASOURCE.TCUSTORD gg_qasource tcustord

About DDL Functionality

About the Keyspaces

The Cassandra Handler does not automatically create keyspaces in Cassandra. Keyspaces in Cassandra define a replication factor, the replication strategy, and topology. The Cassandra Handler does not have enough information to create the keyspaces, so you must manually create them.

You can create keyspaces in Cassandra by using the CREATE KEYSPACE command from the Cassandra shell.

About the Tables

The Cassandra Handler can automatically create tables in Cassandra if you configure it to do so. The source table definition may be a poor source of information to create tables in Cassandra. Primary keys in Cassandra are divided into:

In the default mapping for automated table creation, the first primary key is the partition key, and any additional primary keys are mapped as clustering keys.

Automated table creation by the Cassandra Handler may be fine for proof of concept, but it may result in data definitions that do not scale well. When the Cassandra Handler creates tables with poorly constructed primary keys, the performance of ingest and retrieval may decrease as the volume of data stored in Cassandra increases. Oracle recommends that you analyze the metadata of your replicated tables, then manually create corresponding tables in Cassandra that are properly partitioned and clustered for higher scalability.

Primary key definitions for tables in Cassandra are immutable after they are created. Changing a Cassandra table primary key definition requires the following manual steps:

  1. Create a staging table.

  2. Populate the data in the staging table from original table.

  3. Drop the original table.

  4. Re-create the original table with the modified primary key definitions.

  5. Populate the data in the original table from the staging table.

  6. Drop the staging table.

Adding Column Functionality

You can configure the Cassandra Handler to add columns that exist in the source trail file table definition but are missing in the Cassandra table definition. The Cassandra Handler can accommodate metadata change events of this kind. A reconciliation process reconciles the source table definition to the Cassandra table definition. When the Cassandra Handler is configured to add columns, any columns found in the source table definition that do not exist in the Cassandra table definition are added. The reconciliation process for a table occurs after application startup the first time an operation for the table is encountered. The reconciliation process reoccurs after a metadata change event on a source table, when the first operation for the source table is encountered after the change event.

Dropping Column Functionality

You can configure the Cassandra Handler to drop columns that do not exist in the source trail file definition but exist in the Cassandra table definition. The Cassandra Handler can accommodate metadata change events of this kind. A reconciliation process reconciles the source table definition to the Cassandra table definition. When the Cassandra Handler is configured to drop, columns any columns found in the Cassandra table definition that are not in the source table definition are dropped.

Caution: Dropping a column permanently removes data from a Cassandra table. Carefully consider your use case before you configure this mode.

Note: Primary key columns cannot be dropped. Attempting to do so results in an abend.

Note: Column name changes are not well-handled because there is no DDL is processed. When a column name changes in the source database, the Cassandra Handler interprets it as dropping an existing column and adding a new column.

How Operations are Processed

The Cassandra Handler pushes operations to Cassandra using either the asynchronous or synchronous API. In asynchronous mode, operations are flushed at transaction commit (grouped transaction commit using GROUPTRANSOPS) to ensure write durability. The Cassandra Handler does not interface with Cassandra in a transactional way.

Supported Database Operations

INSERT
UPDATE (captured as INSERT)
DELETE

The Cassandra commit log files do not record any before images for the UPDATE or DELETE operations. So the captured operations never have a before image section. Oracle GoldenGate features that rely on before image records, such as Conflict Detection and Resolution, are not available.

Unsupported Database Operations

TRUNCATE
DDL (CREATE, ALTER, DROP)

Insert, update, and delete operations are processed differently in Cassandra than a traditional RDBMS. The following explains how insert, update, and delete operations are interpreted by Cassandra:

The state of the data in Cassandra is idempotent. You can replay the source trail files or replay sections of the trail files. The state of the Cassandra database must be the same regardless of the number of times that the trail data is written into Cassandra.

About Compressed Updates vs. Full Image Updates

Oracle GoldenGate allows you to control the data that is propagated to the source trail file in the event of an update. The data for an update in the source trail file is either a compressed or a full image of the update, and the column information is provided as follows:

Compressed

For the primary keys and the columns for which the value changed. Data for columns that have not changed is not provided in the trail file.

Full Image

For all columns, including primary keys, columns for which the value has changed, and columns for which the value has not changed.

The amount of information about an update is important to the Cassandra Handler. If the source trail file contains full images of the change data, then the Cassandra Handler can use prepared statements to perform row updates in Cassandra. Full images also allow the Cassandra Handler to perform primary key updates for a row in Cassandra. In Cassandra, primary keys are immutable, so an update that changes a primary key must be treated as a delete and an insert. Conversely, when compressed updates are used, prepared statements cannot be used for Cassandra row updates. Simple statements identifying the changing values and primary keys must be dynamically created and then executed. With compressed updates, primary key updates are not possible and as a result, the Cassandra Handler will abend.

You must set the control properties gg.handler.name.compressedUpdates and gg.handler.name.compressedUpdatesfor so that the handler expects either compressed or full image updates.

The default value, true, sets the Cassandra Handler to expect compressed updates. Prepared statements are not be used for updates, and primary key updates cause the handler to abend.

When the value is false, prepared statements are used for updates and primary key updates can be processed. A source trail file that does not contain full image data can lead to corrupted data columns, which are considered null. As a result, the null value is pushed to Cassandra. If you are not sure about whether the source trail files contains compressed or full image data, set gg.handler.name.compressedUpdates to true.

CLOB and BLOB data types do not propagate LOB data in updates unless the LOB column value changed. Therefore, if the source tables contain LOB data, set gg.handler.name.compressedUpdates to true.

About Primary Key Updates

Primary key values for a row in Cassandra are immutable. An update operation that changes any primary key value for a Cassandra row must be treated as a delete and insert. The Cassandra Handler can process update operations that result in the change of a primary key in Cassandra only as a delete and insert. To successfully process this operation, the source trail file must contain the complete before and after change data images for all columns. The gg.handler.name.compressed configuration property of the Cassandra Handler must be set to false for primary key updates to be successfully processed.

Setting Up and Running the Cassandra Handler

Instructions for configuring the Cassandra Handler components and running the handler are described in the following sections.

Before you run the Cassandra Handler, you must install the Datastax Driver for Cassandra and set the gg.classpath configuration property.

Get the Driver Libraries

The Cassandra Handler has been updated to use the newer 4.x versions of the Datastax Java Driver or 2.x versions of the Datastax Enterprise Java Driver. The Datastax Java Driver for Cassandra does not ship with Oracle GoldenGate for Distributed Applications and Analytics (GG for DAA). For more information, see Datastax Java Driver for Apache Cassandra.

You can use the Dependency Downloader scripts to download the Datastax Java Driver and its associated dependencies.

Set the Classpath

You must configure the gg.classpath configuration property in the Java Adapter properties file to specify the JARs for the Datastax Java Driver for Cassandra. Ensure that this JAR is first in the list.

gg.classpath=/path/to/4.x/cassandra-java-driver/*

Understanding the Cassandra Handler Configuration

The following are the configurable values for the Cassandra Handler. These properties are located in the Java Adapter properties file (not in the Replicat properties file).

To enable the selection of the Cassandra Handler, you must first configure the handler type by specifying gg.handler.name.type=cassandra and the other Cassandra properties as follows:

Table 9 Cassandra Handler Configuration Properties

Properties Required/ Optional Legal Values Default Explanation
gg.handlerlist Required Any string None Provides a name for the Cassandra Handler. The Cassandra Handler name then becomes part of the property names listed in this table.
gg.handler.name.type=cassandra Required cassandra None Selects the Cassandra Handler for streaming change data capture into name.
gg.handler.name.mode Optional op | tx op The default is recommended. In op mode, operations are processed as received. In tx mode, operations are cached and processed at transaction commit. The txmode is slower and creates a larger memory footprint.
gg.handler.**name.contactPoints= Optional A comma separated list of host names that the Cassandra Handler will connect to. localhost A comma-separated list of the Cassandra host machines for the driver to establish an initial connection to the Cassandra cluster. This configuration property does not need to include all the machines enlisted in the Cassandra cluster. By connecting to a single machine, the driver can learn about other machines in the Cassandra cluster and establish connections to those machines as required.
gg.handler.name.username Optional A legal username string. None A user name for the connection to name. Required if Cassandra is configured to require credentials.
gg.handler.name.password Optional A legal password string. None A password for the connection to name. Required if Cassandra is configured to require credentials.
gg.handler.name.compressedUpdates Optional true | false true

Sets the Cassandra Handler whether to expect full image updates from the source trail file. A value of true means that updates in the source trail file only contain column data for the primary keys and for columns that changed. The Cassandra Handler executes updates as simple statements updating only the columns that changed.

A value of false means that updates in the source trail file contain column data for primary keys and all columns regardless of whether the column value has changed. The Cassandra Handler is able to use prepared statements for updates, which can provide better performance for streaming data to name.

gg.handler.name.ddlHandling Optional CREATE | ADD | DROP in any combination with values delimited by a comma None

Configures the Cassandra Handler for the DDL functionality to provide. Options include CREATE, ADD, and DROP. These options can be set in any combination delimited by commas.

When CREATE is enabled, the Cassandra Handler creates tables in Cassandra if a corresponding table does not exist.

When ADD is enabled, the Cassandra Handler adds columns that exist in the source table definition that do not exist in the corresponding Cassandra table definition.

When DROP is enabled, the handler drops columns that exist in the Cassandra table definition that do not exist in the corresponding source table definition.

gg.handler.name.cassandraMode Optional async | sync async

Sets the interaction between the Cassandra Handler and name. Set to async for asynchronous interaction. Operations are sent to Cassandra asynchronously and then flushed at transaction commit to ensure durability. Asynchronous provides better performance.

Set to sync for synchronous interaction. Operations are sent to Cassandra synchronously.

gg.handler.name.consistencyLevel Optional ALL | ANY | EACH_QUORUM | LOCAL_ONE | LOCAL_QUORUM | ONE | QUORUM | THREE | TWO The Cassandra default. Sets the consistency level for operations with name. It configures the criteria that must be met for storage on the Cassandra cluster when an operation is executed. Lower levels of consistency may provide better performance, while higher levels of consistency are safer.
gg.handler.name.port Optional Integer 9042 Set to configure the port number that the Cassandra Handler attempts to connect to Cassandra server instances. You can override the default in the Cassandra YAML files.
gg.handler.name.batchType Optional String unlogged Sets the type for Cassandra batch processing.
  • unlogged - Does not use Cassandra's distributed batch log.
  • logged - Cassandra first writes to its distributed batch log to ensure atomicity of the batch.
  • counter - Use if counter types are updated in the batch.
gg.handler.name.abendOnUnmappedColumns Optional Boolean true Only applicable when gg.handler.name.ddlHanding is not configured with ADD. When set to true, the replicat process will abend if a column exists in the source table, but does not exist in the target Cassandra table. When set to false, the replicat process will not abend if a column exists in the source table, but does not exist in the target Cassandra table. Instead, that column will not be replicated.
gg.handler.name.DatastaxJSSEConfigPath Optional String None Set the path and file name of a properties file containing the Cassandra driver configuration. Use when the Cassandra driver configuration needs to be configured for non-default values and potentially SSL connectivity. For more information, see Cassandra Driver Configuration Documentation. You need to follow the syntax of the configuration file for the driver version you are using. The suffix of the Cassandra driver configuration file must be .conf.
gg.handler.name.dataCenter Optional The datacenter name datacenter1 Set the datacenter name. If the datacenter name does not match the configured name on the server, then it will not connect to the database.

Review a Sample Configuration

The following is a sample configuration for the Cassandra Handler from the Java Adapter properties file:

gg.handlerlist=cassandra

#The handler properties
gg.handler.cassandra.type=cassandra
gg.handler.cassandra.mode=op
gg.handler.cassandra.contactPoints=localhost
gg.handler.cassandra.ddlHandling=CREATE,ADD,DROP
gg.handler.cassandra.compressedUpdates=true
gg.handler.cassandra.cassandraMode=async
gg.handler.cassandra.consistencyLevel=ONE

Configuring Security

The Cassandra Handler connection to the Cassandra Cluster can be secured using user name and password credentials. These are set using the following configuration properties:

gg.handler.name.username
gg.handler.name.password

To configure SSL, the recommendation is to configure the SSL properties via the Datastax Java Driver configuration file and point to the configuration file via the gg.handler.name.DatastaxJSSEConfigPath property. See https://docs.datastax.com/en/developer/java-driver/4.14/manual/core/ssl/ for the SSL settings instructions.

Sample configuration file is as follows. Uncomment the relevant parameters and change to your required values.

datastax-java-driver {
  advanced.ssl-engine-factory {
    class = DefaultSslEngineFactory

    # This property is optional. If it is not present, the driver won't explicitly enable cipher
    # suites on the engine, which according to the JDK documentations results in "a minimum quality
    # of service".
    // cipher-suites = [ "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_CBC_SHA" ]

    # Whether or not to require validation that the hostname of the server certificate's common
    # name matches the hostname of the server being connected to. If not set, defaults to true.
    // hostname-validation = true

    # The locations and passwords used to access truststore and keystore contents.
    # These properties are optional. If either truststore-path or keystore-path are specified,
    # the driver builds an SSLContext from these files. If neither option is specified, the
    # default SSLContext is used, which is based on system property configuration.
    // truststore-path = /path/to/client.truststore
    // truststore-password = password123
    // keystore-path = /path/to/client.keystore
    // keystore-password = password123
  }
}

About Automated DDL Handling

The Cassandra Handler performs the table check and reconciliation process the first time an operation for a source table is encountered. Additionally, a DDL event or a metadata change event causes the table definition in the Cassandra Handler to be marked as not suitable.

Therefore, the next time an operation for the table is encountered, the handler repeats the table check, and reconciliation process as described in this topic.

About the Table Check and Reconciliation Process

The Cassandra Handler first interrogates the target Cassandra database to determine whether the target Cassandra keyspace exists. If the target Cassandra keyspace does not exist, then the Cassandra Handler abends. Keyspaces must be created by the user. The log file must contain the error of the exact keyspace name that the Cassandra Handler is expecting.

Next, the Cassandra Handler interrogates the target Cassandra database for the table definition. If the table does not exist, the Cassandra Handler either creates a table if gg.handler.name.ddlHandling includes the CREATE option or abends the process. A message is logged that shows you the table that does not exist in Cassandra.

If the table exists in Cassandra, then the Cassandra Handler reconciles the table definition from the source trail file and the table definition in Cassandra. This reconciliation process searches for columns that exist in the source table definition and not in the corresponding Cassandra table definition. If it locates columns fitting this criteria and the gg.handler.name.ddlHandling property includes ADD, then the Cassandra Handler adds the columns to the target table in Cassandra. Otherwise, it ignores these columns.

Next, the Cassandra Handler searches for columns that exist in the target Cassandra table but do not exist in the source table definition. If it locates columns that fit this criteria and the gg.handler.name.ddlHandling property includes DROP, then the Cassandra Handler removes these columns from the target table in Cassandra. Otherwise those columns are ignored.

Finally, the prepared statements are built.

Capturing New Change Data

You can capture all of the new change data into your Cassandra database, including the DDL changes in the trail, for the target apply. Following is the acceptance criteria:

AC1: Support Cassandra as a bulk extract
AC2: Support Cassandra as a CDC source
AC4: All Cassandra supported data types are supported
AC5: Should be able to write into different tables based on any filter conditions, like Updates to Update tables or based on primary keys
AC7: Support Parallel processing with multiple threads
AC8: Support Filtering based on keywords
AC9: Support for Metadata provider
AC10: Support for DDL handling on sources and target
AC11: Support for target creation and updating of metadata.
AC12: Support for error handling and extensive logging
AC13: Support for Conflict Detection and Resolution
AC14: Performance should be on par or better than HBase

Performance Considerations

Configuring the Cassandra Handler for async mode provides better performance than sync mode. Set Replicat property GROUPTRANSOPS must be set to the default value of 1000.

Setting the consistency level directly affects performance. The higher the consistency level, the more work must occur on the Cassandra cluster before the transmission of a given operation can be considered complete. Select the minimum consistency level that still satisfies the requirements of your use case.

The Cassandra Handler can work in either operation (op) or transaction (tx) mode. For the best performance operation mode is recommended:

gg.handler.name.mode=op

Additional Considerations

Troubleshooting

This section contains information to help you troubleshoot various issues.

Java Classpath

When the classpath that is intended to include the required client libraries, a ClassNotFound exception appears in the log file. To troubleshoot, set the Java Adapter logging to DEBUG, and then run the process again. At the debug level, the log contains data about the JARs that were added to the classpath from the gg.classpath configuration variable. The gg.classpath variable selects the asterisk (*) wildcard character to select all JARs in a configured directory. For example, /usr/cassandra/cassandra-java-driver4.9.0/*:/usr/cassandra/cassandra-java-driver-4.9.0/lib/*.

For more information about setting the classpath, see Setting Up and Running the Cassandra Handler and Cassandra Handler Client Dependencies.

Write Timeout Exception

When running the Cassandra handler, you may experience a com.datastax.driver.core.exceptions.WriteTimeoutException exception that causes the Replicat process to abend. It is likely to occur under some or all of the following conditions:

When this problem occurs, the Cassandra Handler is streaming data faster than the Cassandra cluster can process it. The write latency in the Cassandra cluster finally exceeds the write request timeout period, which in turn results in the exception.

The following are potential solutions:

Datastax Driver Error

The Cassandra Handler has been changed to use the 4.x version of the Datastax Java Driver. ClassNotFound exceptions can occur under either of the following conditions:

Cassandra Handler Client Dependencies

What are the dependencies for the Cassandra Handler to connect to Apache Cassandra databases?

The following Maven dependencies are required for the Cassandra Handler:

Artifact: java-driver-core

GroupId: com.datastax.oss

AtifactId: java-driver-core

Version: 4.x

Artifact: java-driver-query-builder

GroupId: com.datastax.oss

Artifact ID: java-driver-query-builder

Version: 4.x

Cassandra Datastax Java Driver 4.12.0

asm-9.1.jar
asm-analysis-9.1.jar
asm-commons-9.1.jar
asm-tree-9.1.jar
asm-util-9.1.jar
config-1.4.1.jar
esri-geometry-api-1.2.1.jar
HdrHistogram-2.1.12.jar
jackson-annotations-2.12.2.jar
jackson-core-2.12.2.jar
jackson-core-asl-1.9.12.jar
jackson-databind-2.12.2.jar
java-driver-core-4.12.0.jar
java-driver-query-builder-4.12.0.jar
java-driver-shaded-guava-25.1-jre-graal-sub-1.jar
jcip-annotations-1.0-1.jar
jffi-1.3.1.jar
jffi-1.3.1-native.jar
jnr-a64asm-1.0.0.jar
jnr-constants-0.10.1.jar
jnr-ffi-2.2.2.jar
jnr-posix-3.1.5.jar
jnr-x86asm-1.0.2.jar
json-20090211.jar
jsr305-3.0.2.jar
metrics-core-4.1.18.jar
native-protocol-1.5.0.jar
netty-buffer-4.1.60.Final.jar
netty-codec-4.1.60.Final.jar
netty-common-4.1.60.Final.jar
netty-handler-4.1.60.Final.jar
netty-resolver-4.1.60.Final.jar
netty-transport-4.1.60.Final.jar
reactive-streams-1.0.3.jar
slf4j-api-1.7.26.jar
spotbugs-annotations-3.1.12.jar

Cassandra Datastax Java Driver 4.9.0

asm-7.1.jar
asm-analysis-7.1.jar
asm-commons-7.1.jar
asm-tree-7.1.jar
asm-util-7.1.jar
commons-collections-3.2.2.jar
commons-configuration-1.10.jar
commons-lang-2.6.jar
commons-lang3-3.8.1.jar
config-1.3.4.jar
esri-geometry-api-1.2.1.jar
gremlin-core-3.4.8.jar
gremlin-shaded-3.4.8.jar
HdrHistogram-2.1.11.jar
jackson-annotations-2.11.0.jar
jackson-core-2.11.0.jar
jackson-core-asl-1.9.12.jar
jackson-databind-2.11.0.jar
java-driver-core-4.9.0.jar
java-driver-query-builder-4.9.0.jar
java-driver-shaded-guava-25.1-jre-graal-sub-1.jar
javapoet-1.8.0.jar
javatuples-1.2.jar
jcip-annotations-1.0-1.jar
jcl-over-slf4j-1.7.25.jar
jffi-1.2.19.jar
jffi-1.2.19-native.jar
jnr-a64asm-1.0.0.jar
jnr-constants-0.9.12.jar
jnr-ffi-2.1.10.jar
jnr-posix-3.0.50.jar
jnr-x86asm-1.0.2.jar
json-20090211.jar
jsr305-3.0.2.jar
metrics-core-4.0.5.jar
native-protocol-1.4.11.jar
netty-buffer-4.1.51.Final.jar
netty-codec-4.1.51.Final.jar
netty-common-4.1.51.Final.jar
netty-handler-4.1.51.Final.jar
netty-resolver-4.1.51.Final.jar
netty-transport-4.1.51.Final.jar
reactive-streams-1.0.2.jar
slf4j-api-1.7.26.jar
spotbugs-annotations-3.1.12.jar
tinkergraph-gremlin-3.4.8.jar