4 Choosing Capture and Apply Modes

This chapter contains information that helps you determine the appropriate capture and apply modes for your database environment.

Topics:

• Overview of Oracle GoldenGate Capture and Apply Processes
  The Oracle GoldenGate capture process is known as Extract. Each instance of an Extract process is known as a group, which includes the process itself and the associated files that support it.
• Deciding Which Capture Method to Use
  For an Oracle source database, you can run Extract in either integrated capture or classic capture mode.
• Deciding Which Apply Method to Use
  The Replicat process is responsible for the application of replicated data to an Oracle target database.
• Using Different Capture and Apply Modes Together
  You can use integrated capture and classic capture concurrently within the same source Oracle GoldenGate instance, and you can use integrated Replicat and nonintegrated Replicat concurrently within the same target Oracle GoldenGate instance. This configuration requires careful placement of your objects within the appropriate process group, because there is no coordination of DDL or DML between classic and integrated capture modes, nor between nonintegrated and integrated Replicat modes.
• Switching to a Different Process Mode
  You can switch between process modes. For example, you can switch from classic capture to integrated capture, or from integrated capture to classic capture.

Overview of Oracle GoldenGate Capture and Apply Processes

The Oracle GoldenGate capture process is known as Extract. Each instance of an Extract process is known as a group, which includes the process itself and the associated files that support it.

An additional Extract process, known as a data pump, is recommended to be used on the source system, so that captured data can be persisted locally to a series of files known as a trail. The data pump does not capture data but rather reads the local trail and propagates the data across the network to the target.

The Oracle GoldenGate apply process is known as Replicat. Each instance of a Replicat process is known as a group, which includes the process itself and the associated files that support it. Replicat reads data that is sent to local storage, known as a trail, and applies it to the target database.

The following diagram illustrates the basic Oracle GoldenGate process configuration.

Description of the illustration oracle_config.png

Note:

Oracle Databases must be in ARCHIVELOG mode so that Extract can process the log files.

Deciding Which Capture Method to Use

For an Oracle source database, you can run Extract in either integrated capture or classic capture mode.

Although you can use the classic capture mode, it is recommended that you use the integrated capture mode because classic capture has been deprecated and is not being enhanced for any future releases. It will be desupported in future releases and any classic Extract configuration will need to be migrated to integrated Extract.

The method that you use determines how you configure the Oracle GoldenGate processes and depends on such factors as:

• the data types involved

• the database configuration

• the version of the Oracle Database

The following explains these modes and the database versions that each mode supports.

• About Integrated Capture
  
• About Classic Capture
  

About Integrated Capture

In integrated capture mode, the Oracle GoldenGate Extract process interacts directly with a database logmining server to receive data changes in the form of logical change records (LCR). The following diagram illustrates the configuration of Extract in integrated capture mode.

Description of the illustration integratedcapture.png

Integrated capture supports more data and storage types as compared to classic capture, and the support is more transparent. For more information, see Summary of Supported Oracle Data Type and Objects Per Capture Mode.

The following are some additional benefits of integrated capture:

• Because integrated capture is fully integrated with the database, no additional setup is required to work with Oracle RAC, ASM, and TDE.

• Integrated capture uses the database logmining server to access the Oracle redo stream, with the benefit of being able to automatically switch between different copies of archive logs or different mirrored versions of the online logs. Thus integrated capture can transparently handle the absence of a log file caused by disk corruption, hardware failure, or operator error, assuming that additional copies of the archived and online logs are available

• Integrated capture enables faster filtering of tables.

• Integrated capture handles point-in-time recovery and RAC integration more efficiently.

• Integrated capture features integrated log management. The Oracle Recovery Manager (RMAN) automatically retains the archive logs that are needed by Extract.

• Integrated capture is the only mode that supports capture from a multitenant container database. One Extract can mine multiple pluggable databases within a multitenant container database.

• For a release 11.2.0.4 source database and later (with source compatibility set to 11.2.0.4 or higher), the capture of DDL is performed by the logmining server asynchronously and requires no special triggers, tables, or other database objects to be installed. Oracle GoldenGate upgrades can be performed without stopping user applications. The use of a DDL trigger and supporting objects is required when Extract is in integrated mode with an Oracle 11g source database that is earlier than version 11.2.0.4.

• Because integrated capture and integrated apply are both database objects, the naming of the objects follow the same rules as other Oracle Database objects, see Specifying Object Names in Oracle GoldenGate Input in Administering Oracle GoldenGate.

• Integrated Capture Deployment Options
  

Integrated Capture Deployment Options

The deployment options for integrated capture are described in this section and depend on where the mining database is deployed. The mining database is the one where the logmining server is deployed.

• Local deployment: For a local deployment, the source database and the mining database are the same. The source database is the database for which you want to mine the redo stream to capture changes, and also where you deploy the logmining server. Because integrated capture is fully integrated with the database, this mode does not require any special database setup.

• Downstream deployment: In a downstream deployment, the source and mining databases are different databases. You create the logmining server at the downstream database. You configure redo transport at the source database to ship the redo logs to the downstream mining database for capture at that location. Using a downstream mining server for capture may be desirable to offload the capture overhead and any other overhead from transformation or other processing from the production server, but requires log shipping and other configuration.

  When using a downstream mining configuration, the source database and mining database must be of the same platform. For example, if the source database is running on Windows 64-bit, the downstream database must also be on a Windows 64-bit platform. See Configuring a Downstream Mining Database and Example Downstream Mining Configuration to configure a downstream mining database.

• Downstream sourceless Extract deployment: In the Extract parameter file, replace the USERID parameter with NOUSERID. You must use TRANLOGOPTIONS MININGUSER. Extract obtains all required information from the downstream mining database. Extract is not dependent on any connection to the source database. The source database can be shutdown and restarted without affecting Extract.

  Extract will abend if it encounters redo changes that require data to be fetched from the source database.

  To capture any tables that are listed as ID KEY in the dba_goldengate_support_mode view, you need to have a FETCHUSERID or FETCHUSERIDALIAS connection to the support the tables. Tables that are listed as FULL do not require this. We also need to state that if a customer wants to perform SQLEXEC operations that perform a query or execute a stored procedure they cannot use this method as it is incompatible with NOUSERID because SQLEXEC works with USERID or USERIDALIAS.

About Classic Capture

In classic capture mode, the Oracle GoldenGate Extract process captures data changes from the Oracle redo or archive log files on the source system or from shipped archive logs on a standby system. The following diagram illustrates the configuration of an Extract in classic capture mode.

Note:

Classic capture has been deprecated from Oracle GoldenGate 18c (18.1.0) and higher releases.

Description of the illustration classiccapture.png

Classic capture supports most Oracle data types fully, with restricted support for the complex data types. Classic capture is the original Oracle GoldenGate capture method. You can use classic capture for any source Oracle RDBMS that is supported by Oracle GoldenGate, with the exception of the multitenant container database.

For more information, see Details of Support for Oracle Data Types and Objects.

Deciding Which Apply Method to Use

The Replicat process is responsible for the application of replicated data to an Oracle target database.

For an Oracle target database, you can run Replicat in parallel, non-integrated or integrated mode. Oracle recommends that you use the parallel Replicat unless a specific feature requires a different type of Replicat.

Topics:

• About Parallel Replicat
  
• About Non-integrated Replicat
  
• About the Integrated Replicat Mode
  

About Parallel Replicat

Parallel Replicat is a new variant of Replicat that applies transactions in parallel to improve performance.

It takes into account dependencies between transactions, similar to Integrated Replicat. The dependency computation, parallelism of the mapping and apply is performed outside the database so can be off-loaded to another server. The transaction integrity is maintained in this process. In addition, parallel Replicat supports the parallel apply of large transactions by splitting a large transaction into chunks and applying them in parallel.

The following table lists the features supported by the respective Replicats.

Feature	Parallel Replicat	Integrated Replicat	Coordinated Replicat	Classic Replicat
Batch Processing	Yes	Yes	Yes	Yes
Barrier Transactions	Yes	Yes	Yes	No
Dependency Computation	Yes	Yes	No	No
Auto-parallelism	Yes	Yes	No	No
DML Handler	Yes, Integrated mode	Yes	No	No
Procedural Replication	Yes, used for integrated Parallel Replicat (iPR)	Yes	No	No
Auto CDR	Yes, used by iPR only	Yes	No	No
Dependency-aware Transaction Split	Yes	No	No	No
Cross-RAC-node Processing	Yes	No	Yes	No
ALLOWDUPTARGETMAP See ALLOWDUPTARGETMAP | NOALLOWDUPTARGETMAP	No. Oracle Database with iPR	No, Oracle Database	Yes	Yes

Parallel Replicat supports all databases using the non-integrated option. Parallel Replicat only supports replicating data from trails with full metadata, which requires the classic trail format.

The components of parallel Replicat are:

• Mappers operate in parallel to read the trail, map trail records, convert the mapped records to the Integrated Replicat LCR format, and send the LCRs to the Merger for further processing. While one Mapper maps one set of transactions, the next Mapper maps the next set of transactions. The the trail information is split and the trail file is untouched because it orders trail information in order.

• Master processes have two threads, Collater and Scheduler. The Collater receives mapped transactions from the Mappers and puts them back into trail order for dependency calculation. The Scheduler calculates dependencies between transactions, groups transactions into independent batches, and sends the batches to the Appliers to be applied to the target database.

• Appliers reorder records within a batch for array execution. It applies the batch to the target database and performs error handling. It also tracks applied transactions in checkpoint tables.

About Non-integrated Replicat

In non-integrated mode, the Replicat process uses standard SQL to apply data directly to the target tables. In this mode, Replicat operates as follows:

• Reads the Oracle GoldenGate trail.

• Performs data filtering, mapping, and conversion.

• Constructs SQL statements that represent source database DML or DDL transactions (in committed order).

• Applies the SQL to the target through Oracle Call Interface (OCI).

The following diagram illustrates the configuration of Replicat in non-integrated mode.

Description of the illustration nonintegratedapply.png

Use non-integrated Replicat when you want to make heavy use of features that are not supported in integrated Replicat mode, see About the Integrated Replicat Mode.

You can apply transactions in parallel with a non-integrated Replicat by using a coordinated Replicat configuration.

About the Integrated Replicat Mode

In integrated mode, the Replicat process leverages the apply processing functionality that is available within the Oracle Database. In this mode, Replicat operates as follows:

• Reads the Oracle GoldenGate trail.

• Performs data filtering, mapping, and conversion.

• Constructs logical change records (LCR) that represent source database DML transactions (in committed order). DDL is applied directly by Replicat.

• Attaches to a background process in the target database known as a database inbound server by means of a lightweight streaming interface.

• Transmits the LCRs to the inbound server, which applies the data to the target database.

The following figure illustrates the configuration of Replicat in integrated mode.

Description of the illustration integratedapply.png

Within a single Replicat configuration, multiple inbound server child processes known as apply servers apply transactions in parallel while preserving the original transaction atomicity. You can increase this parallelism as much as your target system will support when you configure the Replicat process or dynamically as needed. The following diagram illustrates integrated Replicat configured with two parallel apply servers.

Description of the illustration integratedapply_detail.png

Integrated Replicat applies transactions asynchronously. Transactions that do not have interdependencies can be safely executed and committed out of order to achieve fast throughput. Transactions with dependencies are guaranteed to be applied in the same order as on the source.

A reader process in the inbound server computes the dependencies among the transactions in the workload based on the constraints defined at the target database (primary key, unique, foreign key). Barrier transactions and DDL operations are managed automatically, as well. A coordinator process coordinates multiple transactions and maintains order among the apply servers.

If the inbound server does not support a configured feature or column type, Replicat disengages from the inbound server, waits for the inbound server to complete transactions in its queue, and then applies the transaction to the database in direct apply mode through OCI. Replicat resumes processing in integrated mode after applying the direct transaction.

The following features are applied in direct mode by Replicat:

• DDL operations

• Sequence operations

• SQLEXEC parameter within a TABLE or MAP parameter

• EVENTACTIONS processing

• UDT Note, if the extract uses USENATIVEOBJSUPPORT to capture the UDT, then integrated Replicat will apply it with the inbound server, otherwise it will be handled by Replicat directly.

Because transactions are applied serially in direct apply mode, heavy use of such operations may reduce the performance of the integrated Replicat mode. Integrated Replicat performs best when most of the apply processing can be performed in integrated mode, see Monitoring and Controlling Processing After the Instantiation in Using Oracle GoldenGate for Oracle Database.

Note:

User exits are executed in integrated mode. The user exit may produce unexpected results, however, if the exit code depends on data in the replication stream.

• Benefits of Integrated Replicat
  
• Integrated Replicat Requirements
  

Benefits of Integrated Replicat

The following are the benefits of using integrated Replicat versus nonintegrated Replicat.

• Integrated Replicat enables heavy workloads to be partitioned automatically among parallel apply processes that apply multiple transactions concurrently, while preserving the integrity and atomicity of the source transaction. Both a minimum and maximum number of apply processes can be configured with the PARALLELISM and MAX_PARALLELISM parameters. Replicat automatically adds additional servers when the workload increases, and then adjusts downward again when the workload lightens.

• Integrated Replicat requires minimal work to configure. All work is configured within one Replicat parameter file, without configuring range partitions.

• High-performance apply streaming is enabled for integrated Replicat by means of a lightweight application programming interface (API) between Replicat and the inbound server.

• Barrier transactions are coordinated by integrated Replicat among multiple server apply processes.

• DDL operations are processed as direct transactions that force a barrier by waiting for server processing to complete before the DDL execution.

• Transient duplicate primary key updates are handled by integrated Replicat in a seamless manner.

• Integrated Replicat works with single or pluggable databases.

Integrated Replicat Requirements

To use integrated Replicat, the following must be true.

• The target Oracle Database must be Oracle 11.2.0.4 or later.

• Supplemental logging must be enabled on the source database to support the computation of dependencies among tables and scheduling of concurrent transactions on the target. Instructions for enabling the required logging are in Configuring Logging Properties. This logging can be enabled at any time up to, but before, you start the Oracle GoldenGate processes.

• Integrated Parallel Replicat is supported on Oracle Database 12.2.0.1 and greater.

Using Different Capture and Apply Modes Together

You can use integrated capture and classic capture concurrently within the same source Oracle GoldenGate instance, and you can use integrated Replicat and nonintegrated Replicat concurrently within the same target Oracle GoldenGate instance. This configuration requires careful placement of your objects within the appropriate process group, because there is no coordination of DDL or DML between classic and integrated capture modes, nor between nonintegrated and integrated Replicat modes.

Each Extract group must process objects that are suited to the processing mode, based on table data types and attributes. No objects in one Extract can have DML or DDL dependencies on objects in the other Extract. The same type of segregation must be applied to the Replicat configuration.

You can use the following capture and apply modes together:

• Classic capture (Oracle or non-Oracle source) and nonintegrated Replicat

• Classic capture (Oracle or non-Oracle source) and integrated Replicat

• Integrated capture and nonintegrated Replicat

• Integrated capture and integrated Replicat

The recommended Oracle GoldenGate configuration, when supported by the Oracle version, is to use one integrated capture on an Oracle source and one integrated Replicat per source database on an Oracle target. Integrated capture supports certain data types more completely than classic capture. One integrated Replicat configuration supports all Oracle data types either through the inbound server or by switching to direct apply when necessary, and it preserves source transaction integrity. You can adjust the parallelism settings to the desired apply performance level as needed.

If the target database is an Oracle version that does not support integrated Replicat, or if it is a non-Oracle Database, you can use a coordinated Replicat configuration. See Administering Oracle GoldenGate for more information.

Switching to a Different Process Mode

You can switch between process modes. For example, you can switch from classic capture to integrated capture, or from integrated capture to classic capture.

For instructions, see Performing Administrative Operations inAdministering Oracle GoldenGate.