Installing Oracle GoldenGate in Installing and Configuring Oracle GoldenGate for Oracle Database
Understanding Oracle GoldenGate Adapters in Administering Oracle GoldenGate Application Adapters
Note:
To use HotCache, you must have licenses for Oracle GoldenGate and Coherence Grid Edition. HotCache can be used with Oracle GoldenGate for Big Data 12c (12.3.1.1.0 or higher) and Oracle GoldenGate 11gR1, 11gR2, and 12c releases.
This chapter includes the following sections:
persistence.xml
file.(pof-config.xml)
to allow for HotCache and TopLink Grid framework classes to be registered.HotCache can be added to any Coherence application. Standard JPA is used to capture the mappings from database data to Java objects. The configuration can be captured in XML exclusively or in XML with annotations.
The following scenario describes how HotCache could be used to work with the database and with applications that use Coherence caches. Figure 2-1 illustrates the scenario.
Start GoldenGate. See Starting the Application in Administering Oracle GoldenGate Application Adapters. GoldenGate monitors the transaction log for changes of interest. These changes will be placed into a "trail file".
Start the Coherence cache server and warm the cache, if required.
Start HotCache so that it can propagate changes in the trail file into the cache. If changes occur during cache warming, then they will be applied to the cache once HotCache is started so no changes are lost.
Start an application client. As part of its operation, assume that the application performs repeated queries on the cache.
A third-party application performs a direct database update.
GoldenGate detects the database change which is then propagated to the Coherence cache by HotCache.
The application client detects the change in cache.
Figure 2-1 How HotCache Propagates Database Changes to the Cache
This section includes the following topics:
HotCache processes database change events delivered by GoldenGate and maps those changes onto the affected objects in the Coherence cache. It is able to do this through the use of Java Persistence API (JPA) mapping metadata. JPA is the Java standard for object-relational mapping in Java and it defines a set of annotations (and corresponding XML) that describe how Java objects are mapped to relational tables. As Example 2-1 illustrates, instances of an Employee
class could be mapped to rows in an EMPLOYEE
table with the following annotations.
Example 2-1 Mapping Instances of Employee Class to Rows with Java Code
@Entity @Table(name="EMPLOYEE") Public class Employee { @Id @Column(name="ID") private int id; @Column(name="FIRSTNAME") private String firstName; … }
The @Entity
annotation marks the Employee
class as being persistent and the @Id
annotation identifies the id
field as containing its primary key. In the case of Coherence cached objects, the @Id
field must also contain the value of the key under which the object is cached. The @Table
and @Column
annotations associate the class with a named table and a field with a named column, respectively.
For simplification, JPA assumes a number of default mappings such as table name=class
name
and column
name=field
name
so many mappings need only be specified when the defaults are not correct. In Example 2-1, both the table and field names match the Java names so the @Table
and @Column
can be removed to make the code more compact, as illustrated in Example 2-2.
Example 2-2 Simplified Java Code for Mapping Instances of Employee Class to Rows
@Entity Public class Employee { @Id private int id; private String firstName; … }
Note that the Java code in the previous examples can also be expressed as XML. Example 2-3 illustrates the XML equivalent of the Java code in Example 2-1.
Example 2-3 Mapping Instances of Employee Class to Rows with XML
<entity class="Employee"> <table name="EMPLOYEE"/> <attributes> <id name="id"> <column name="ID"/> </id> <basic name="firstName"/> <column name="FIRSTNAME"/> </basic> ... </attributes> </entity>
Similarly, Example 2-4 illustrates the XML equivalent for the simplified Java code in Example 2-2.
Example 2-4 Simplified XML for Mapping Instances of Employee Class to Rows
<entity class="Employee"> <attributes> <id name="id"/> <basic name="firstName"/> ... </attributes> </entity>
JPA mapping metadata provides mappings from object to relational; however, it also provides the inverse relational to object mappings which HotCache can use. Given the Employee
example, consider an update to the FIRSTNAME
column of a row in the EMPLOYEE
table. Figure 2-2 illustrates the EMPLOYEE
table before the update, where the first name John is associated with employee ID 1, and the EMPLOYEE
table after the update where first name Bob is associated with employee ID 1.
Figure 2-2 EMPLOYEE Table Before and After an Update
With GoldenGate monitoring changes to the EMPLOYEE
table and HotCache configured on the appropriate trail file, the adapter processes an event indicating the FIRSTNAME
column of the EMPLOYEE
row with primary key 1
has been changed to Bob
. The adapter will use the JPA mapping metadata to first identify the class associated with the EMPLOYEE
table, Employee
, and then determine the column associated with an Employee
's ID field, ID
. With this information, the adapter can extract the ID column value from the change event and update the firstName
field (associated with the FIRSTNAME
column) of the Employee
cached under the ID
column value.
Database INSERT
, UPDATE
, and DELETE
operations are supported by the GoldenGate Java Adapter. INSERT
operations into a mapped table result in the addition of a new instance of the associated class populated with the data from the newly inserted row. Changes applied through an UPDATE
operation are propagated to the corresponding cached object. If the cache does not contain an object corresponding to the updated row, then the cache is unchanged by default. To change the default behavior, see HonorRedundantInsert Property. A DELETE
operation results in the removal of the corresponding object from the cache, if one exists.
HotCache does not support the JPA relationship mappings one-to-one, one-to-many, many-to-one, and many-to-many. However HotCache does support JPA embeddable classes and JPA element collections. Embeddable classes and element collections can be used with HotCache to model relationships between domain objects. Domain objects used with HotCache may also refer to each other by an identifier (analogous to foreign keys in a relational database).
As a performance optimization, when using JPA element collections with HotCache, it is suggested to configure GoldenGate with an ADD TRANDATA
command specifying the column in the element collection table that is the foreign key to the parent table. The optimization allows HotCache to efficiently find the cache entry to update when a row in the element collection table changes.
Setting up a database includes:
creating a database and tables
granting user permissions
enabling logging
provisioning the tables with data
Example 2-5 illustrates a list of sample commands for the Oracle Database that creates a user named csdemo
and grants user permissions to the database.
Note the ALTER
DATABASE
ADD
SUPPLEMENTAL
LOG
DATA
command. When supplemental logging is enabled, all columns are specified for extra logging. At the very least, minimal database-level supplemental logging must be enabled for any change data capture source database. If the values of primary key columns in a database table can change, it is important to include the following commands for Oracle Database: ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
and ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (UNIQUE) COLUMNS;
.
Example 2-5 Sample Commands to Create a User, Grant Permissions, and Enable Logging
CREATE USER csdemo IDENTIFIED BY csdemo; GRANT DBA TO csdemo; grant alter session to csdemo; grant create session to csdemo; grant flashback any table to csdemo; grant select any dictionary to csdemo; grant select any table to csdemo; grant select any transaction to csdemo; grant unlimited tablespace to csdemo; ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
The instructions also assume that you have installed Oracle GoldenGate and started the manager. This includes the following tasks:
downloading and installing Oracle GoldenGate
running ggsci
to create the GoldenGate subdirectories
creating a manager parameter (mgr.prm
) file, specifying the listener port
adding JVM libraries to the libraries path
starting the manager
A detailed description of these tasks is beyond the scope of this documentation. See Installing Oracle GoldenGate in Installing and Configuring Oracle GoldenGate for Oracle Database , Installing Oracle GoldenGate Application Adapters, and Understanding Oracle GoldenGate Adapters in Administering Oracle GoldenGate Application Adapters.
Note:
It is important to use the generic build of Oracle GoldenGate when installing Oracle GoldenGate Application Adapters. Do not use a non-generic instance of Oracle GoldenGate.
This section includes the following topics:
Indicate the table that you want to monitor for changes by using the ADD
TRANDATA
command. The ADD
TRANDATA
command can be used on the command line or as part of a ggsci
script. For example, to monitor changes to tables in the csdemo
schema, use the following command:
ADD TRANDATA csdemo.*
Example 2-6 illustrates a sample ggsci
script named cs-cap.ggsci
.
The script starts the manager and logs into the database. It stops and deletes any running extract named cs-cap
.
The ADD
TRANDATA
command instructs the extract that tables named csdemo*
should be monitored for changes.
The SHELL
command deletes all trail files in the dirdat
directory to ensure that if the extract is being recreated, there will be no old trail files. Note that the rm -f
command is platform-specific. An extract named cs-cap
is created using parameters from the dirprm/cs-cap.prm
file. A trail is added at dirdat/cs
from the extract cs-cap
file.
The start
command starts the cs-cap.ggsci
script.
The ADD
EXTRACT
command automatically uses the cs-cap.prm
file as the source of parameters, so a PARAMS
dirprm/cs-cap.prm
, statement is not necessary.
Example 2-6 Sample GoldenGate Java Adapter ggsci Script to Monitor Table Changes
start mgr DBLOGIN USERID csdemo, PASSWORD csdemo STOP EXTRACT cs-cap DELETE EXTRACT cs-cap ADD TRANDATA csdemo.* ADD EXTRACT cs-cap, tranlog, begin now SHELL rm -f dirdat/cs* ADD EXTTRAIL dirdat/cs, EXTRACT cs-cap start cs-cap
Configure GoldenGate to ignore changes made by the user that the Coherence CacheStores are logged in as. This avoids GoldenGate processing any changes made to the database by Coherence that are already in the cache.
The TranLogOptions
excludeUSER
command can be used on the command line or in a ggsci
script. For example, the following command instructs GoldenGate extract process to ignore changes to the database tables made by the Coherence CacheStore user logged in as csdemo
.
TranLogOptions excludeUser csdemo
Example 2-7 illustrates a sample extract .prm
file named cs-cap.prm
. The user that the Coherence CacheStore is logged in as is csdemo
. The recoveryOptions
OverwriteMode
line indicates that the extract overwrites the existing transaction data in the trail after the last write-checkpoint position, instead of appending the new data. The EXTRAIL
parameter identifies the trail as dirdat/cs
. The BR
BROFF
parameter controls the Bounded Recovery (BR) feature. The BROFF
value turns off Bounded Recovery for the run and for recovery. The GETUPDATEBEFORES
parameter indicates that the before images of updated columns are included in the records that are processed by Oracle GoldenGate. The TABLE
parameter identifies csdemo.*
as the tables that should be monitored for changes. The TranLogOptions
excludeUSER
parameter indicates that GoldenGate should ignore changes to the tables made by the Coherence CacheStore user logged in as csdemo
.
Note:
The OverwriteMode
option is not applicable in Oracle GoldenGate for Big Data.
Example 2-7 Sample Extract .prm File for the GoldenGate Java Adapter
EXTRACT cs-cap USERID csdemo, PASSWORD csdemo RecoveryOptions OverwriteMode EXTTRAIL dirdat/cs BR BROFF getUpdateBefores TABLE csdemo.*; TranLogOptions excludeUser csdemo --ignore changes made by csuser
persistence.xml
file. See How Does HotCache Work. The connection from HotCache to the Coherence cluster can be made by using Coherence*Extend (TCP), or the HotCache JVM can join the Coherence cluster as a member.The following sections describe the properties needed to configure HotCache and provide details about connecting with Coherence*Extend:
Create a text file with the filename extension .properties
. In the file, enter the configuration for HotCache. A minimal configuration should contain the list of event handlers and the fully-qualified Java class of the event handler.
Note:
The path to the .properties
file must be set in the HotCache replicat TARGETDB
parameter in a .prm
file, for example:
TARGETDB LIBFILE libggjava.so SET property=/home/oracle/gg/hotcache.properties
Example 2-8 illustrates a .properties
file that contains the minimal configuration for a HotCache project. The following properties are used in the file:
gg.handlerlist=hotcache
The gg.handlerlist
property specifies a comma-separated list of active handlers. This example defines the logical name hotcache
as database change event handler. The name of a handler can be defined by the user, but it must match the name used in the gg.handler.{
name}
.type property in the following bullet.
gg.handler.hotcache.type=[oracle.toplink.goldengate.CoherenceAdapter|oracle.toplink.goldengate.CoherenceAdapter1220]
The gg.handler.{name}.type
property defines the handler for HotCache. The {name}
field should be replaced with the name of an event handler listed in the gg.handlerlist
property. The only handlers that can be set for HotCache are oracle.toplink.goldengate.CoherenceAdapter
or oracle.toplink.goldengate.CoherenceAdapter1220
. Use oracle.toplink.goldengate.CoherenceAdapter1220
with GoldenGate Application Adapters release 12.2.0 or later. Use oracle.toplink.goldengate.CoherenceAdapter
with GoldenGate Application Adapters releases earlier than 12.2.0.
There are many other properties that can be used to control the behavior of the GoldenGate Java Adapter. See Using Oracle GoldenGate Application Adapters Properties in Administering Oracle GoldenGate Adapters.
Example 2-8 .properties File for a HotCache Project
# ==================================================================== # List of active event handlers # ==================================================================== gg.handlerlist=hotcache # ==================================================================== # HotCache event handler # ==================================================================== gg.handler.hotcache.type=oracle.toplink.goldengate.CoherenceAdapter1220 # ====================================== # JVM boot options # ====================================== jvm.bootoptions=-Xms32M -Xmx32M -Djava.class.path=dirprm:ggjava/ggjava.jar:/u01/app/oracle/product/12.1.0/dbhome_1/jdbc/lib/ojdbc6.jar:/u01/Oracle/Middleware/Oracle_Home/coherence/lib/coherence.jar:/u01/Oracle/Middleware/Oracle_Home/coherence/lib/coherence-hotcache.jar:/u01/Oracle/Middleware/Oracle_Home/oracle_common/modules/javax.persistence.jar:/u01/Oracle/Middleware/Oracle_Home/oracle_common/modules/oracle.toplink/eclipselink.jar:/u01/Oracle/Middleware/Oracle_Home/oracle_common/modules/oracle.toplink/toplink-grid.jar:/home/oracle/cgga/workspace/CacheStoreDemo/bin -Dcoherence.cacheconfig=/home/oracle/cgga/workspace/CacheStoreDemo/client-cache-config.xml -Dcoherence.distributed.localstorage=false -Dtoplink.goldengate.persistence-unit=employee
This section describes the properties that must appear in the JVM boot options section of the .properties
file. These options are defined by using the jvm.bootoptions
property. A sample jvm.bootoptions
listing is illustrated in JVM boot options
section of Example 2-8.
This section includes the following topics:
The following is a list of directories and JAR files that should be included in the Java classpath. The directories and JAR files are defined with the java.class.path
property.
dirprm
– the GoldenGate dirprm
directory
ggjava.jar
– contains the GoldenGate Java adapter libraries
ojdbc6.jar
– contains the JDBC driver classes for the source database
coherence.jar
– contains the Oracle Coherence libraries
coherence-hotcache.jar
– contains the Oracle Coherence GoldenGate HotCache libraries
javax.persistence.jar
– contains the Java persistence libraries
eclipselink.jar
– contains the EclipseLink libraries
toplink-grid.jar
– contains the Oracle TopLink libraries required by HotCache
domain classes – the JAR file or directory containing the user classes cached in Coherence that are mapped with JPA for use in HotCache. Also, the Coherence configuration files, persistence.xml
file, and any orm.xml
file.
The toplink.goldengate.persistence-unit
property is required as it identifies the persistence unit defined in persistence.xml
file that HotCache should load. The persistence unit contains information such as the list of participating domain classes, configuration options, and optionally, database connection information.
The toplink.goldengate.on-error
property is optional. It controls how the adapter responds to errors while processing a change event. This response applies to both expected optimistic lock exceptions and to unexpected exceptions. This property is optional, as its value defaults to "Refresh". Refresh causes the adapter to attempt to read the latest data for a given row from the database and update the corresponding object in the cache. Refresh requires a database connection to be specified in the persistence.xml
file. This connection will be established during initialization of HotCache. If a connection cannot be made, then an exception is thrown and HotCache will fail to start.
The other on-error strategies do not require a database connection. They are:
Ignore
—Log the exception only. The cache may be left with stale data. Depending on application requirements and cache eviction policies this may be acceptable.
Evict
—Log a warning and evict the object corresponding to the change database row from the cache
Fail
—Throw an exception and exit HotCache
Any Coherence property can be passed as a system property in the Java boot options. The coherence.distributed.localstorage
system property with a value of false
is the only Coherence property that is required to be passed in the Java boot options. Like all Coherence properties, precede the property name with the -D
prefix in the jvm.bootoptions
statement, for example:
-Dcoherence.distributed.localstorage=false
The following logging properties can be defined for HotCache.
The -Dlog4j.configuration=default-log4j.properties
property specifies the default Log4J configuration file. Example properties are located in $GOLDEN_GATE_HOME/ggjava/resources/classes/
directory. You can merge these with your existing Log4J configuration.
The Log4J properties file that is bundled with GoldenGate for Java is for demonstration purposes only. The file can be used as-is, or you can merge its contents with the existing Log4J properties.
If the file is used as-is, then it should be copied into the dirprm
directory, given a new name, and specified with the -Dlog4j.configuration
property. For example, the following line specifies the user-defined my-log4j.properties
file in the dirprm
directory (note the dirprm
directory is already on the classpath):
-Dlog4j.configuration=my-log4j.properties
Using the default properties file in its current location can cause problems during upgrades: your changes will lost when a new distribution is installed.
To allow HotCache to log warnings, add the following line to the property file:
log4j.logger.oracle.toplink.goldengate=WARN, stdout, rolling
To allow HotCache to log errors, add the following line to the property file you use:
-Dlog4j.logger.oracle.toplink.goldengate=DEBUG, stdout, rolling
Note:
A Coherence Log4J configuration can co-exist with the GoldenGate Log4J configuration. Both can be included in the same file that is configured on the jvm.bootoptions
path.
The connection between HotCache and the Coherence cluster can be made with Coherence*Extend. For more information on Coherence*Extend, see Developing Remote Clients for Oracle Coherence.
The Coherence configuration files must be in a directory referenced by the jvm.bootoptions=-Djava.class.path=
... entry in the .properties
file. For an example, see the jvm.bootoptions
entry in Example 2-8.
Example 2-9 illustrates the section of a client cache configuration file that uses Coherence*Extend to connect to the Coherence cluster. In the client cache configuration file, Coherence*Extend is configured in the <remote-cache-scheme>
section. By default, the connection port for Coherence*Extend is 9099
.
Example 2-9 Coherence*Extend Section of a Client Cache Configuration File
<cache-config> ... <caching-schemes> <remote-cache-scheme> <scheme-name>CustomRemoteCacheScheme</scheme-name> <service-name>CustomExtendTcpCacheService</service-name> <initiator-config> <tcp-initiator> <remote-addresses> <socket-address> <address>localhost</address> <port>9099</port> </socket-address> </remote-addresses> </tcp-initiator> <outgoing-message-handler> ... </outgoing-message-handler> </initiator-config> </remote-cache-scheme> ... </cache-config>
Example 2-10 illustrates the section of a server cache configuration file that listens for Coherence*Extend connections. In the server cache configuration file, Coherence*Extend is configured in the <proxy-scheme>
section. By default, the listener port for Coherence*Extend is 9099
.
Example 2-10 Coherence*Extend Section of a Server Cache Configuration File
<cache-config> ... <caching-schemes> ... <proxy-scheme> <scheme-name>CustomProxyScheme</scheme-name> <service-name>CustomProxyService</service-name> <thread-count>2</thread-count> <acceptor-config> <tcp-acceptor> <local-address> <address>localhost</address> <port>9099</port> </local-address> </tcp-acceptor> </acceptor-config> <load-balancer>proxy</load-balancer> <autostart>true</autostart> </proxy-scheme> </caching-schemes> </cache-config>
This section includes the following topic:
This section describes the parameters that can be defined in the replicat .prm
file for a GoldenGate Java adapter. The parameters that are illustrated in Example 2-11 constitute a minimal configuration for a HotCache project.
For a complete list of parameters, see Summary of Replicat Parameters in Reference for Oracle GoldenGate for Windows and UNIX.
TARGETDB LIBFILE libggjava.so SET property=/home/oracle/gg/hotcache.properties
GROUPTRANSOPS 1
The GROUPTRANSOPS
parameter controls transaction grouping by the GoldenGate replicat process. A value of 1
tells the GoldenGate replicat process to honor source database transaction boundaries in the trail file. A value greater than 1
tells the GoldenGate replicat process to group operations from multiple source database transactions into a single target transaction. See GROUPTRANSOPS in Reference for Oracle GoldenGate for Windows and UNIX
MAP scott.*, TARGET scott.*;
The MAP parameter tells the GoldenGate replicat process how to map source database tables to the replication target. The parameter syntax assumes the replication target is a relational database. For HotCache it is appropriate to specify an identical mapping. See TABLE and MAP Options in Reference for Oracle GoldenGate for Windows and UNIX
Example 2-11 illustrates a sample .prm
file for a GoldenGate Java adapter.
Example 2-11 Sample .prm Parameter File for a GoldenGate Java Adapter
REPLICAT hotcache TARGETDB LIBFILE libggjava.so SET property=/home/user/project/hotcache.properties GROUPTRANSOPS 1 MAP scott.*, TARGET scott.*;
coherence-hotcache.jar
– contains the Oracle Coherence GoldenGate HotCache libraries
javax.persistence.jar
– contains the Java persistence libraries
eclipselink.jar
– contains the EclipseLink libraries
toplink-grid.jar
– contains the Oracle TopLink libraries required by HotCache
domain classes – the JAR file or directory containing the user classes cached in Coherence that are mapped with JPA for use in HotCache.
(pof-config.xml)
to allow for HotCache and TopLink Grid framework classes to be registered.The pof-config.xml
file must include the coherence-hotcache-pof-config.xml
file and must register the TopLinkGridPortableObject
user type and TopLinkGridSerializer
as the serializer. The <type-id>
for each class must be unique and must match across all cluster instances. See Registering POF Objects in Developing Applications with Oracle Coherence.
The <allow-interfaces>
element must be set to true
to allow you to register a single class for all implementors of the TopLinkGridPortableObject
interface.
Example 2-12 illustrates a sample pof-config.xml
file for HotCache. The value integer_value
represents a unique integer value greater than 1000
.
Example 2-12 Sample POF Configuration File for HotCache
<?xml version='1.0'?>
<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config
coherence-pof-config.xsd">
<user-type-list>
<include>coherence-hotcache-pof-config.xml</include>
<!-- User types must be above 1000 -->
...
<user-type>
<type-id><integer_value></type-id>
<class-name>oracle.eclipselink.coherence.integrated.cache.TopLinkGridPortableObject</class-name>
<serializer>
<class-name>oracle.eclipselink.coherence.integrated.cache.TopLinkGridSerializer</class-name>
</serializer>
</user-type>
...
</user-type-list>
<allow-interfaces>true</allow-interfaces>
...
</pof-config>
@Property
annotation on the JPA entity class or by a <property>
element in the persistence.xml
file. The latter takes precedence in the event of conflicting configuration.This section includes the following topics:
EnableUpsert
The EnableUpsert
property controls whether HotCache inserts a cache entry when an update operation is received in the GoldenGate trail file but no corresponding cache entry is present in cache at the entity key. By default, HotCache ignores updates to absent entities. Set this property to true
if you want HotCache to insert missing entities into the cache when update operations are received in the trail file. The default value of this property is false
.
Setting this property to true
can facilitate warming caches in an event-driven manner if it is likely that entities will be accessed from the cache after their corresponding records are updated in the underlying database.
Note:
There are risks to consider when using this property:
The entity to insert is read from the database, as the trail file may not contain values for all fields of the entity to be inserted. This can reduce the throughput of the HotCache process by waiting on database reads.
Cache capacity can be exhausted if more rows in the DB are updated than the number of entities in the cache for which capacity was provisioned.
Entity Class Annotation
@Property(name = "EnableUpsert", value = "false",valueType = boolean.class)
Persistence XML Property
<property name="[fully qualified entity class name].EnableUpsert" value="[true|false]"/>
HonorRedundantInsert
The HonorRedundantInsert
property controls whether HotCache honors an insert operation in the GoldenGate trail file when a cache entry at that key is already present. By default, HotCache ignores a redundant insert operation. However, when a JPA entity is mapped to a complex materialized view in Oracle Database and a row is updated in a table underlying that materialized view (thus updating one or more rows of the materialized view), Oracle Database inserts a new row into the materialized view with the same PK as an existing row but with a new rowid and deletes the existing row. Therefore, HotCache sees a redundant insert operation that really represents an update to the cached JPA entity. Users in this situation should also consider suppressing replication of delete operations on that materialized view through the use of GoldenGate configuration; otherwise, the cached entity is deleted by HotCache. The default value of this property is false
.
Entity Class Annotation
@Property(name = "HonorRedundantInsert", value = "false",valueType = boolean.class)
Persistence XML Property
<property name="[fully qualified entity class name].HonorRedundantInsert" value="[true|false]"/>
SyntheticEvent
The SyntheticEvent
property controls whether cache writes by HotCache are synthetic or not. Synthetic writes to Coherence caches do not trigger events in Coherence; they do not engage Federated Caching; and they do not call Coherence CacheStore
implementations. Set this property to false
for a JPA entity class if you want cache writes by HotCache for that class to be non-synthetic so that events are triggered, Federated Caching is engaged, and CacheStore
implementations are called (if any of those are configured for the entity class cache). The default value of this property is true
for every entity class.
Note:
There is a risk of infinite replication loops if the SyntheticEvent
is set to true
for an entity class and a CacheStore
implementation is configured on that entity class cache and writing to the same database HotCache is replicating to Coherence. This risk can be mitigated by filtering transactions by database user. See Filter Changes Made by the Current User.
Entity Class Annotation
@Property(name = "SyntheticEvent", value = "[true|false]", valueType = boolean.class)
Persistence XML Property
<property name="[fully qualified entity class name].SyntheticEvent" value="[true|false]"/>
eclipselink.cache.shared.default
The eclipselink.cache.shared.default
property is used to enable the EclipseLink internal shared cache. It is important to disable the internal shared cache in the HotCache JVM by setting the property to false
in the persistence.xml
file.
Persistence XML Property
<property name=" eclipselink.cache.shared.default" value="false"/>
This section includes the following topics:
IL0001
, IL0002
, etc... in the GG_HOME/dirdat
directory. If no files are generated, then review the GG_HOME/dirrpt/initload.rpt
file.initload.rpt
) with the number of entries in the target Coherence caches according to Coherence MBeans or command-line interface commands.In best-practice deployments, Coherence cache services and proxy services are already highly available due to redundancy of service members (for example, multiple cache server processes and multiple proxy server processes) and due to built-in automatic failover capabilities within Coherence. For example, if a proxy server should fail, then Coherence*Extend clients that are using the proxy server automatically fail over to another proxy server. Likewise, if a cache server should fail then another cache server assumes responsibility for its data and client interactions with that data automatically redirect to the new cache server owning the data.
Making the HotCache client itself highly available relies on standard GoldenGate HA techniques since the HotCache JVM runs embedded in a GoldenGate process.
GoldenGate implements “single server” HA through AUTOSTART
and AUTORESTART
parameters enforced by the Manager process in a GoldenGate installation. The Manager process automatically starts registered GoldenGate processes configured with AUTOSTART
. It also detects the death of (and automatically restarts), registered GoldenGate processes configured with AUTORESTART
.
To protect against failure of the Manager process itself or the host on which it runs or the network connecting that host, GoldenGate relies on Oracle Clusterware to detect the death of the active GoldenGate installation and fail over to a passive GoldenGate installation.
GoldenGate high availability, including GoldenGate processes that embed HotCache JVMs, is discussed in the Oracle GoldenGate High Availability Using Oracle Clusterware white paper.
SDO_GEOMETRY
from the Oracle Spatial and Graph option for Oracle Database and XMLType
in all Oracle Database editions.It is important to understand that data is presented to EclipseLink differently when used in HotCache than when used in the typical JPA scenario. In the typical JPA scenario, EclipseLink interacts with the database through a JDBC connection and EclipseLink consumes data as presented by the JDBC API and driver-specific extensions (for example an SDO_GEOMETRY
column is represented as an instance of java.sql.Struct
). Whereas in HotCache, data is read from a GoldenGate trail file; there is no JDBC connection involved. Therefore EclipseLink consumes the GoldenGate representation of data as opposed to the JDBC representation of data. For example, GoldenGate represents an SDO_GEOMETRY
column as an XML document and not as an instance of java.sql.Struct
.
These differences in data representation may necessitate the use of HotCache-specific EclipseLink converters when using EclipseLink within HotCache that take the place of standard EclipseLink converters used in typical JPA scenarios. See @Converter in Java Persistence API (JPA) Extensions Reference for EclipseLink. The following sections describe HotCache support for specific Oracle Database data types supported by EclipseLink and how to configure EclipseLink within HotCache to use those data types.
EclipseLink supports the Oracle Spatial and Graph option of Oracle Database by mapping SDO_GEOMETRY
columns to instances of oracle.spatial.geometry.JGeometry
(the Java class shipped with the Oracle Spatial and Graph option). See Using Oracle Spatial and Graph in Solutions Guide for EclipseLink.
Therefore, HotCache supports mapping columns of type SDO_GEOMETRY
to instances of oracle.spatial.geometry.JGeometry
bound to fields of JPA entities. This support requires configuring a HotCache-specific EclipseLink Converter of class oracle.toplink.goldengate.spatial.GoldenGateJGeometryConverter
as shown in the following example.
import javax.persistence.Access; import javax.persistence.AccessType; import javax.persistence.Convert; import javax.persistence.Converter; import javax.persistence.Entity; import oracle.spatial.geometry.JGeometry; import oracle.toplink.goldengate.spatial.GoldenGateJGeometryConverter; @Entity @Converter(name=”JGeometry”, converterClass= GoldenGateJGeometryConverter.class) public class SpatialEntity { private JGeometry geometry; @Access(AccessType.PROPERTY) @Convert(“JGeometry”) public JGeometry getGeometry() { return geometry; }
This converter converts the GoldenGate XML representation of an SDO_GEOMETRY
column into an instance of oracle.spatial.geometry.JGeometry
bound to a field of a JPA entity. The GoldenGateJGeometryConverter
class is contained in coherence-hotcache.jar
which should already be on the classpath of the HotCache JVM and Coherence cache server JVMs used in HotCache deployments (along with the eclipselink.jar
file on which it depends). However the JGeometry
class is contained in sdoapi.jar
from an installation of Oracle Spatial and Graph option. The sdoapi.jar
file must be on the classpath of the HotCache JVM, and any other JVM where the JPA entity containing a JGeometry
field will be deserialized.
The oracle.spatial.geometry.JGeometry
class implements java.io.Serializable
, so JPA entities with JGeometry
fields cached in Coherence can be serialized with java.io.Serializable
without any additional configuration. To use Coherence’s Portable Object Format (POF) to serialize a JPA entity with a JGeometry
field, the JGeometrySerializer
must be added to the POF configuration file used in the Coherence deployment, as in the following example.
<user-type> <type-id>1001</type-id><!—use a type-id value above 1000 that doesn’t conflict with other POF type-ids--> <class-name>oracle.spatial.geometry.JGeometry</class-name> <serializer> <class-name>oracle.spatial.geometry.JGeometryPofSerializer</class-name> </serializer> </user-type>
The oracle.spatial.geometry.JGeometryPofSerializer
class is contained in coherence-hotcache.jar
, which must be on the classpath of any JVM that will serialize or deserialize a JPA entity with a JGeometry
field using POF.
EclipseLink supports the Oracle Database XMLType
data type by mapping XMLType
columns to instances of java.lang.String
or org.w3c.dom.Document
(depending on the type of the mapped field in the JPA entity). See DirectToXMLTypeMapping in EclipseLink API Reference and Mapping XMLTYPE in the On Persistence blog.
Therefore, HotCache supports mapping columns of type XMLType
to instances of java.lang.String
or org.w3c.dom.Document
bound to fields of JPA entities. This support requires configuring a standard EclipseLink DirectToXMLTypeMapping
.
GoldenGate must be configured to use integrated capture mode for support of XMLType
columns. See Summary of Supported Oracle Data Types and Objects Per Capture Mode and Deciding Which Capture Method to Use in Installing and Configuring Oracle GoldenGate for Oracle Database.
Transactions and their operations appear in the trail file in the order in which they were committed in the source database. By default, HotCache applies operations one at a time on a single thread to ensure the operations are applied to the cache in the exact same order in which they were applied to the source database. When using multi-threading, operations can be applied in a different order than that in which they were applied to the source database tables and can result in correctness risks.
When determining the potential risk, consider the following examples:
If one database transaction inserts a row in a table and the next database transaction deletes that row, then applying operations out of order can leave an object in the cache whose corresponding database row is deleted.
If one database transaction updates a column to an older value and the next database transaction updates that column to a newer value, then applying operations out of order can leave the older value in the cached object instead of the newer value. (You can use the JPA optimistic locking features, which are supported by HotCache, to mitigate this particular update risk).
If you determine that using multiple threads to apply trail file operations to Coherence caches poses no data correctness risks in the system using HotCache, then HotCache can be configured to use multi-threading as follows:
Edit the GoldenGate Java Adapter properties file and configure the HotCache event handler to use transaction mode:
gg.handlerlist=hotcache goldengate.handler.hotcache.type=oracle.toplink.goldengate.CoherenceAdapter1220 goldengate.handler.hotcache.mode=tx
By default, GoldenGate Java Adapter event handlers use operation mode. In operation mode (op
), event handlers process operations one at a time. In transaction mode (tx
), event handlers process all operations in a transaction at a time.
Edit the GoldenGate Java Adapter properties file and set the coherence.hotcache.concurrency
system property on the HotCache JVM with a value between one and eight times the number of cores on the JVM host, inclusive (as reported by java.lang.Runtime.getAvailableProcessors()
). For example:
jvm.bootoptions=-Dcoherence.hotcache.concurrency=16 …
The value of this property determines the number of threads HotCache uses to concurrently apply trail file operations to Coherence caches.
Edit the HotCache replicat .prm
file and set the GROUPTRANOPS
property. A value of 1
causes source database transaction boundaries to be honored. A value greater than 1
causes transaction grouping within the GoldenGate replicat. The default value is 1000
.
Summary of Hot Cache Thread Behavior
Assuming HotCache is run in a GoldenGate replicat process as recommended, the risks stemming from conflicting source database transactions only materialize if the GROUPTRANOPS
property is configured to a value other than one 1
. A value of 1
causes the source database transaction boundaries and sequencing to be honored by the HotCache replicat. Therefore, the operations in one transaction are applied in parallel followed by the operations in the next transaction and so on. The GROUPTRANOPS
property default is 1000
, which groups trail file operations from multiple successive source database transactions into one target transaction of at least 1,000 operations. The likelihood of data correctness risks materializing when the GROUPTRANOPS
parameter is set to greater than one is equivalent to the likelihood of conflicting operations within the grouped source database transactions, given the magnitude of the GROUPTRANOPS
property value and the write rate and volume of the source database. See GROUPTRANSOPS in Reference for Oracle GoldenGate for Windows and UNIX.
The following table summarizes HotCache thread behavior depending on the values of the GoldenGate Java Adapter mode
property and the coherence.hotcache.concurrency
property.
Table 2-1 Hot Cache Thread Behavior
Mode | Concurrency | Behavior |
---|---|---|
|
N/A |
In op mode, HotCache applies trail file operations one at a time on a single thread (the GoldenGate Java Adapter thread) as each operation is read from the trail file. This is the HotCache default behavior. The value of the concurrency property is not considered in operation mode. |
|
1 |
In tx mode with a concurrency property value of 1 , HotCache iterates and applies a transaction worth of trail file operations on a single thread (the GoldenGate Java Adapter thread). The group of operations comprising the transaction is determined by the value of the GoldenGate replicat GROUPTRANOPS property. The default value of the concurrency property is 1 . This configuration may exhibit greater throughput than the operation mode configuration, even though it is still single-threaded and therefore poses no data correctness risks. |
|
>1 |
In tx mode with a concurrency property value greater than 1 , HotCache applies a transaction worth of operations in parallel on multiple HotCache threads. The group of operations comprising the transaction is determined by the value of the GoldenGate replicat GROUPTRANOPS property. This configuration should exhibit greater throughput than single-threaded configurations and throughput generally increases with the number of threads configured to a maximum of eight times the number of cores on the HotCache host. |
The GoldenGate trail file is assumed to include data for a single tenant, and must be configured to include data for a single tenant. If you want to refresh caches for multiple tenants, then a different HotCache process must be used for each tenant. If you are using Oracle Database and have not already done so, configure GoldenGate multitenant support before completing these instructions. See Configuring Oracle GoldenGate in a Multitenant Container Database in Installing and Configuring Oracle GoldenGate for Oracle Database.
To use HotCache multitenant mode:
Edit the HotCache properties file and include a coherence.hotcache.tenantid
system property, within the JVM boot options section, that is set to the database tenant identifier. For example:
-Dcoherence.hotcache.tenantid=oracle
Edit the Coherence cache configuration file to include a scope name for the configuration. For example
<?xml version='1.0'?> <cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config" xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd"> <defaults> <scope-name>accounts</scope-name> </defaults> ...
Note:
Given the above tenant identifier and scope name, the cache service name changes to oracle/accounts:servicename
.
This section includes the following topics:
The CoherenceAdapterMXBean
MBean represents a Golden Gate HotCache adapter and provides operational and performance statistics. Zero or more instances of this managed bean are created: one managed bean instance for each adapter instance.
The object name of the MBean is:
Type=CoherenceAdapter,name=replicat name,member=member name
To view the CoherenceAdapterMXBean
MBean from an MBean browser, you must enable Coherence management. If you are new to Coherence JMX management, see Using JMX to Manage Oracle Coherence.
Attributes
Table 2-2 describes the attributes for CacheMBean
.
Table 2-2 CoherenceAdapterMXBean
Attribute | Type | Access | Description |
---|---|---|---|
|
|
read-only |
The names of the caches that were refreshed by the CoherenceAdapter |
|
|
read-only |
Summary statistics about the execution time for each operation in nanoseconds since the statistics were last reset |
|
|
read-only |
Summary statistics about the execution time for each transaction in nanoseconds since the statistics were last reset |
|
|
read-only |
Summary statistics about the number of invocations for each operation since the statistics were last reset |
|
|
read-only |
Summary statistics about the execution time for each operation in nanoseconds since this method was last called |
|
|
read-only |
Summary statistics about operation replication lag in milliseconds since this method was last called |
|
|
read-only |
The aggregate number of operations processed since the statistics were last reset |
|
|
read-only |
Summary statistics about operation replication lag in milliseconds since the statistics were last reset |
|
|
read-only |
Summary statistics about the number of operations for each transaction since the statistics were last reset |
|
|
read-only |
Execution time summary statistics in nanoseconds for each cache for each operation type |
|
|
read-only |
The time at which the CoherenceAdapter was started |
|
|
read-only |
The name of the trail file currently being read |
|
|
read-only |
The position in the trail file of the last successfully-processed operation |
Operations
The CoherenceAdapterMXBean
MBean includes a resetStatistics
operation that resets all cache statistics.
The HotCache report includes operational settings and performance statistics. The statistical data is collected from the CoherenceAdapterMXBean
MBean and presented over time making it ideal for discovering performance trends and troubleshooting potential performance issues. The name of the HotCache report is timestamp-hotcache.txt
where the timestamp is in YYYYMMDDHH format. For example, a file named 2009013101-hotcache.txt
represents a HotCache report for January 31, 2009 at 1:00 a.m.
To view the HotCache report, you must enable Coherence reporting and you must configure the report-all report group. If you are new to Coherence reporting, see Using Oracle Coherence Reporting.
Table 2-3 describes the contents of the HotCache report.
Table 2-3 Contents of the HotCache Report
Column | Data Type | Description |
---|---|---|
|
|
A sequential counter to help integrate information between related files. This value resets when the reporter restarts, and is not consistent across members. However, it is helpful when trying to integrate files. |
|
|
A timestamp for each report refresh |
|
|
The user-given name of the HotCache event handler from the GoldenGate HotCache properties file |
|
|
The Coherence member name where the HotCache adapter runs |
|
|
The time when the Coherence HotCache adapter started |
|
|
The number of transaction operations processed |
|
|
The name of the Golden Gate trail file that contains transaction operations |
|
|
The position in the trail file of the last successfully-processed operation |
|
|
The average number of operations processed for each transaction |
|
|
The maximum number of operations processed for each transaction |
|
|
The minimum number of operations processed for each transaction |
|
|
The average number of entry processor invocations that are performed for each operation |
|
|
The maximum number of entry processor invocations that are performed for each operation |
|
|
The minimum number of entry processor invocations that are performed for each operation |
|
|
The average execution time for each operation since the last sample in nanoseconds |
|
|
The average execution time for each operation in nanoseconds |
|
|
The maximum execution time for each operation in nanoseconds |
|
|
The minimum execution time for each operation in nanoseconds |
|
|
The average execution time for each transaction in nanoseconds |
|
|
The maximum execution time for each transaction in nanoseconds |
|
|
The maximum execution time for each transaction in nanoseconds |
|
|
The average time in milliseconds between the commit of the database transaction and the processing of the last operation by the HotCache adapter |
|
|
The average time in milliseconds between the commit of the database transaction and the processing of the operation by the HotCache adapter |
|
|
The average time in milliseconds between the commit of the database transaction and the processing of the operation by the HotCache adapter since the last sample |
|
|
The minimum time in milliseconds between the commit of the database transaction and the processing of the operation by the HotCache adapter |
The HotCache tab in the Coherence-JVisualVM Plug-In provides a graphical view of HotCache performance statistics. If you are new to the Coherence-Java VisualVM plug-in, see Using the Coherence-JVisualVM Plug-In.
The HotCache statistical data is collected from the CoherenceAdapatMBean
MBean and presented over time in both tabular and graph form. The tab displays statistics for each GoldenGate HotCache member including detail about specific caches refreshed by that HotCache member. To view data for a specific member, select the member on the member table. To view data for a specific cache, select the cache on the cache table.
Use the HotCache tab to get a detailed view of performance statistics and to identify potential performance issues with cache update operations. The HotCache tab includes:
The minimum, maximum, and average time it takes to update a cache for each operation.
The minimum, maximum, and average time it takes to update a cache for all the operations in a transaction.
The total number of entry processor invocations that are performed for each operation.
The minimum, maximum, and average time for the last operation.
The minimum, maximum, and average operation replication lag time for the last operation since this MBean attribute value was last sampled. Replication lag is the amount of time between the commit of the database transaction and the processing of the operation by the HotCache adapter.
The minimum, maximum, and average operation replication lag time since the statistics were last reset.
The minimum, maximum, and average number of operations for each transaction.
The minimum, maximum, and average time for each operation type for each cache. Operations include: EVICT
, INSERT
, PK_CHANGE
, READ_FROM_DB
, REDUNDANT_INSERT
, REFRESH
, UPDATE
, and UPSERT
.