3 Using Oracle GoldenGate for Live Reporting
Topics:
3.1 Overview of the Reporting Configuration
The most basic Oracle GoldenGate configuration is a one-to-one configuration that replicates in one direction: from a source database to a target database that is used only for data retrieval purposes such as reporting and analysis. Oracle GoldenGate supports like-to-like or heterogeneous transfer of data, with capabilities for filtering and conversion on either system in the configuration (support varies by database platform).
Oracle GoldenGate supports different reporting topologies that enable you to custom-configure the processes based on your requirements for scalability, availability, and performance. This section contains things to take into consideration when choosing a reporting configuration.
Parent topic: Using Oracle GoldenGate for Live Reporting
3.1.1 Filtering and Conversion
Data filtering and data conversion both add overhead, and these activities are sometimes prone to configuration errors. If Oracle GoldenGate must perform a large amount of filtering and conversion, consider using one or more data pumps to handle this work. You can use Replicat for this purpose, but you would be sending more data across the network that way, as it will be unfiltered. You can split filtering and conversion between the two systems by dividing it between the data pump and Replicat.
To filter data, you can use:
-
A
FILTER
orWHERE
clause in aTABLE
statement (Extract) or in aMAP
statement (Replicat) -
A SQL query or procedure
-
User exits
To transform data, you can use:
-
The Oracle GoldenGate conversion functions
-
A user exit from the Extract or Replicat process that applies rules from an external transformation solution, then returns the manipulated data to Oracle GoldenGate
-
Replicat to deliver data directly to an ETL solution or other transformation engine
For more information about Oracle GoldenGate's filtering and conversion support, see:
Parent topic: Overview of the Reporting Configuration
3.1.2 Read-only vs. High Availability
The Oracle GoldenGate live reporting configuration supports a read-only target. See Configuring Oracle GoldenGate for Active-Active Configuration if the target in this configuration will also be used for transactional activity in support of high availability.
Parent topic: Overview of the Reporting Configuration
3.1.3 Additional Information
The following documentation provides additional information of relevance to configuring Oracle GoldenGate.
-
For additional system requirements, process configuration, and database setup requirements, see the Oracle GoldenGate installation and configuration document for your database type. These guides are listed in the Preface of this book.
-
For detailed instructions on configuring Oracle GoldenGate change capture and delivery groups, see Configuring Online Change Synchronization.
-
For additional tuning options for Oracle GoldenGate, see Tuning the Performance of Oracle GoldenGate.
-
For complete syntax and descriptions of the Oracle GoldenGate commands and parameters, see Reference for Oracle GoldenGate for Windows and UNIX.
Parent topic: Overview of the Reporting Configuration
3.2 Creating a Standard Reporting Configuration
In the standard Oracle GoldenGate configuration, one Extract group sends captured data over TCP/IP to a trail on the target system, where it is stored until processed by one Replicat group.
Refer to Figure 3-1 for a visual representation of the objects you will be creating.
Figure 3-1 Configuration Elements for Creating a Standard Reporting Configuration
Description of "Figure 3-1 Configuration Elements for Creating a Standard Reporting Configuration"
3.2.1 Source System
Configure the Manager process and Extract group on the source system.
To Configure the Manager Process
On the source, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
To Configure the Extract Group
Parent topic: Creating a Standard Reporting Configuration
3.2.2 Target System
Configure the Manager process and Replicat group on the target system.
To Configure the Manager Process
-
On the target, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the local trail.
To Configure the Replicat Group
Parent topic: Creating a Standard Reporting Configuration
3.3 Creating a Reporting Configuration with a Data Pump on the Source System
You can add a data pump on the source system to isolate the primary Extract from TCP/IP functions, to add storage flexibility, and to offload the overhead of filtering and conversion processing from the primary Extract.
In this configuration, the primary Extract writes to a local trail on the source system. A local data pump reads that trail and moves the data to a remote trail on the target system, which is read by Replicat.
You can, but are not required to, use a data pump to improve the performance and fault tolerance of Oracle GoldenGate.
Refer to Figure 3-2 for a visual representation of the objects you will be creating.
Figure 3-2 Configuration Elements for Replicating to One Target with a Data Pump
Description of "Figure 3-2 Configuration Elements for Replicating to One Target with a Data Pump"
Topics:
3.3.1 Source System
Configure the Manager process and Extract group on the source system.
To Configure the Manager Process
-
On the source, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the local trail.
To Configure the Primary Extract Group
-
On the source, use the
ADD EXTRACT
command to create an Extract group. For documentation purposes, this group is called ext.ADD EXTRACT
ext
, {TRANLOG | INTEGRATED TRANLOG}, BEGINtime
[option
[, ...]]See Reference for Oracle GoldenGate for detailed information about these and other
ADD EXTRACT
options that may be required for your installation. -
On the source, use the
ADD EXTTRAIL
command to create a local trail. The primary Extract writes to this trail, and the data-pump Extract reads it.ADD EXTTRAIL
local_trail
, EXTRACText
Use the
EXTRACT
argument to link this trail to the primary Extract group. The primary Extract group writes to this trail, and the data pump group reads it. -
On the source, use the
EDIT PARAMS
command to create a parameter file for the primary Extract group. Include the following parameters plus any others that apply to your database environment. For possible additional required parameters, see the Oracle GoldenGate installation and setup guide for your database.-- Identify the Extract group: EXTRACT
ext
-- Specify database login information as needed for the database: [SOURCEDBdsn_1
][,USERIDALIASalias
] -- Log all scheduling columns if using integrated Replicat LOGALLSUPCOLS -- Specify the local trail that this Extract writes to and -- encryption algorithm: ENCRYPTTRAILalgorithm
EXTTRAILlocal_trail
-- Specify tables and sequences to be captured: SEQUENCE [container
.|catalog
.]owner
.sequence; TABLE [container
.|catalog
.]owner
.table
;
To Configure the Data Pump Extract Group
3.3.2 Target System
Configure the Manager process and Replicat group on the target system.
To Configure the Manager Process
-
On the target, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the local trail.
To Configure the Replicat Group
3.4 Creating a Reporting Configuration with a Data Pump on an Intermediary System
You can use an intermediary system as a transfer point between the source and target systems. In this configuration, a data pump on the source system sends captured data to a remote trail on the intermediary system. A data pump on the intermediary system reads the trail and sends the data to a remote trail on the target. A Replicat on the target reads the remote trail and applies the data to the target database.
Figure 3-3 Configuration Elements for Replication through an Intermediary System
Description of "Figure 3-3 Configuration Elements for Replication through an Intermediary System"
When considering this topology, take note of the following:
-
This configuration is practical if the source and target systems are in different networks and there is no direct connection between them. You can transfer the data through an intermediary system that can connect to both systems.
-
This configuration can be used to add storage flexibility to compensate for deficiences on the source or target.
-
This configuration can be used to perform data filtering and conversion if the character sets on all systems are identical. If character sets differ, the data pump cannot perform conversion between character sets, and you must configure Replicat to perform the conversion and transformation on the target.
-
This configuration is a form of cascaded replication. However, in this configuration, data is not applied to a database on the intermediary system. See Creating a Cascading Reporting Configuration to include a database on the intermediary system in the Oracle GoldenGate configuration.
Topics:
Parent topic: Using Oracle GoldenGate for Live Reporting
3.4.1 Source System
Refer to Figure 10 for a visual representation of the objects you will be creating.
To Configure the Manager Process
-
On the source, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the trail.
To Configure the Primary Extract Group on the Source
-
On the source, use the
ADD EXTRACT
command to create an Extract group. For documentation purposes, this group is called ext.ADD EXTRACT
ext
, {TRANLOG | INTEGRATED TRANLOG}, BEGINtime
[option
[, ...]]See Reference for Oracle GoldenGate for detailed information about these and other
ADD EXTRACT
options that may be required for your installation. -
On the source, use the
ADD EXTTRAIL
command to create a local trail. The primary Extract writes to this trail, and the data-pump Extract reads it.ADD EXTTRAIL
local_trail
, EXTRACText
Use the
EXTRACT
argument to link this trail to the primary Extract group. The primary Extract group writes to this trail, and the data pump group reads it. -
On the source, use the
EDIT PARAMS
command to create a parameter file for the primary Extract group. Include the following parameters plus any others that apply to your database environment. For possible additional required parameters, see the Oracle GoldenGate installation and setup guide for your database.-- Identify the Extract group: EXTRACT
ext
-- Specify database login information as needed for the database: [SOURCEDBdsn_1
][, USERIDALIASalias
] -- Log all scheduling columns if using integrated Replicat LOGALLSUPCOLS -- Specify the local trail that this Extract writes to and -- encryption algorithm: ENCRYPTTRAILalgorithm
EXTTRAILlocal_trail
-- Specify tables and sequences to be captured: SEQUENCE [container
.|catalog
.]owner
.sequence; TABLE [container
.|catalog
.]owner
.table
;
To Configure the Data Pump on the Source
3.4.2 Intermediary System
Configure the Manager process and data pump on the intermediary system.
To Configure the Manager Process on the Intermediary System
-
On the intermediary system, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the trail.
To Configure the Data Pump on the Intermediary System
3.4.3 Target System
Configure the Manager process and Replicat group on the target system.
To Configure the Manager Process on the Target
-
On the target system, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the trail.
To Configure the Replicat Group on the Target
3.5 Creating a Cascading Reporting Configuration
Oracle GoldenGate supports cascading synchronization, where Oracle GoldenGate propagates data changes from the source database to a second database, and then on to a third database. In this configuration:
-
A primary Extract on the source writes captured data to a local trail, and a data pump sends the data to a remote trail on the second system in the cascade.
-
On the second system, Replicat applies the data to the local database.
-
Another primary Extract on that same system captures the data from the local database and writes it to a local trail.
-
A data pump sends the data to a remote trail on the third system in the cascade, where it is applied to the local database by another Replicat.
Note:
See Creating a Reporting Configuration with a Data Pump on an Intermediary System if you do not need to apply the replicated changes to a database on the secondary system.
Use this configuration if:
-
One or more of the target systems does not have a direct connection to the source, but the second system can connect in both directions.
-
You want to limit network activity from the source system.
-
You are sending data to two or more servers that are very far apart geographically, such as from Chicago to Los Angeles and then from Los Angeles to servers throughout China.
When considering this topology, take note of the following:
-
This configuration can be used to perform data filtering and conversion if the character sets on all systems are identical. If character sets differ, a data pump cannot perform conversion between character sets, and you must configure Replicat to perform the conversion and transformation on the target.
-
On the second system, you must configure the Extract group to capture Replicat activity and to ignore local business application activity. The Extract parameters that control this behavior are
IGNOREAPPLOPS
andGETREPLICATES
.
Parent topic: Using Oracle GoldenGate for Live Reporting
3.5.1 Source System
Refer to Figure 3-4 for a visual representation of the objects you will be creating.
To Configure the Manager Process on the Source
-
On the source, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the trail.
To Configure the Primary Extract Group on the Source
-
On the source, use the
ADD EXTRACT
command to create an Extract group. For documentation purposes, this group is called ext_1.ADD EXTRACT
ext_1
, {TRANLOG | INTEGRATED TRANLOG}, BEGINtime
[option
[, ...]]See Reference for Oracle GoldenGate for detailed information about these and other
ADD EXTRACT
options that may be required for your installation. -
On the source, use the
ADD EXTTRAIL
command to create a local trail.ADD EXTTRAIL
local_trail_1
, EXTRACText_1
Use the
EXTRACT
argument to link this trail to the ext_1 Extract group. -
On the source, use the
EDIT PARAMS
command to create a parameter file for the ext_1 Extract group. Include the following parameters plus any others that apply to your database environment. For possible additional required parameters, see the Oracle GoldenGate installation and setup guide for your database.-- Identify the Extract group: EXTRACT
ext_1
-- Specify database login information as needed for the database: [SOURCEDBdsn_1
][, USERIDALIASalias
] -- Log all scheduling columns if using integrated Replicat LOGALLSUPCOLS -- Specify the local trail that this Extract writes to -- and encryption algorithm: ENCRYPTTRAILalgorithm
EXTTRAILlocal_trail
-- Specify tables and sequences to be captured: SEQUENCE [container
.|catalog
.]owner
.sequence; TABLE [container
.|catalog
.]owner
.table
;
To Configure the Data Pump on the Source
Parent topic: Creating a Cascading Reporting Configuration
3.5.2 Second System in the Cascade
Configure the Manager process, Replicat group, and data pump on the second system in the cascade.
To Configure the Manager Process on the Second System
-
On the second system, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the trail.
To Configure the Replicat Group on the Second System
-
Create a Replicat checkpoint table (unless using Oracle integrated Replicat). See Creating a Checkpoint Table for instructions.
-
On the second system, use the
ADD REPLICAT
command to create a Replicat group. For documentation purposes, this group is called rep_1.ADD REPLICAT
rep_1
[, INTEGRATED | COORDINATED [MAXTHREADSnumber
]] , EXTTRAILremote_trail_1
, , BEGINtime
Use the
EXTTRAIL
option to link the rep_1 group to the remote trail remote_trail_1 that is on the local system.See Reference for Oracle GoldenGate for detailed information about these and other options that may be required for your installation.
-
On the second system, use the
EDIT PARAMS
command to create a parameter file for the Replicat group. Include the following parameters plus any others that apply to your database environment. For possible additional required parameters, see the Oracle GoldenGate installation and setup guide for your database.-- Identify the Replicat group: REPLICAT
rep_1
-- Specify database login information as needed for the database: [TARGETDBdsn_2
][, USERIDALIASalias
] -- Specify error handling rules: REPERROR (error, response) -- Specify tables for delivery and threads if using coordinated Replicat: MAP [container
.|catalog
.]owner
.table
, TARGETowner
.table
[, DEFtemplate
] [, THREAD (thread_ID)][, THREADRANGE (thread_range[, column_list])] ;Note:
For DB2 for i, you may need to use the
ADD TRANDATA
command on the target tables if they are not already journaled. Alternatively, you could use theSTRJRNPF
command to assign the tables to the appropriate journal. If the target tables are not required to be replicated by Oracle GoldenGate, theIMAGES(*AFTER)
option can be used withSTRJRNPF
. Since Oracle GoldenGate operates using transactions, all tables must be journaled to support transactions and this is not the default with DB2 for i.
To Configure an Extract Group on the Second System
-
On the source, use the
ADD EXTRACT
command to create an Extract group. For documentation purposes, this group is called ext_2.ADD EXTRACT
ext_2
, {TRANLOG | INTEGRATED TRANLOG}, BEGINtime
[option
[, ...]]See Reference for Oracle GoldenGate for detailed information about these and other
ADD EXTRACT
options that may be required for your installation. -
On the second system, use the
ADD EXTTRAIL
command to specify a local trail that will be created on the third system.ADD EXTTRAIL
local_trail_2
, EXTRACText_2
Use the
EXTRACT
argument to link this local trail to the ext_2 Extract group. -
On the second system, use the
EDIT PARAMS
command to create a parameter file for the ext_2 Extract group. Include the following parameters plus any others that apply to your database environment. For possible additional required parameters, see the Oracle GoldenGate installation and setup guide for your database.-- Identify the Extract group: EXTRACT
ext_2
-- Specify database login information as needed for the database: [SOURCEDBdsn_2
][, USERIDALIASalias
] -- Log all scheduling columns if using integrated Replicat LOGALLSUPCOLS -- Specify the local trail that this Extract writes to -- and encryption algorithm: ENCRYPTTRAILalgorithm
EXTTRAILlocal_trail_2
-- Ignore local DML, capture Replicat DML: IGNOREAPPLOPS GETREPLICATES -- Specify tables and sequences to be captured: SEQUENCE [container
.|catalog
.]owner
.sequence; TABLE [container
.|catalog
.]owner
.table
;
Note:
If replicating DDL operations, IGNOREAPPLOPS, GETREPLICATES
functionality is controlled by the DDLOPTIONS
parameter.
To Configure the Data Pump on the Second System
Parent topic: Creating a Cascading Reporting Configuration
3.5.3 Third System in the Cascade
Configure the Manager process and Replicat group on the third system in the cascade.
To Configure the Manager Process
-
On the third system, configure the Manager process according to the instructions in Configuring Manager and Network Communications.
-
In the Manager parameter file, use the
PURGEOLDEXTRACTS
parameter to control the purging of files from the trail.
To Configure the Replicat Group
Parent topic: Creating a Cascading Reporting Configuration