4 Choosing Different Replicat Modes with Extract

This chapter contains information that helps you determine the appropriate Replicat modes to use with Extract for your database environment.

Topics:

4.1 Deciding Which Apply Method to Use

The Replicat process is responsible for the application of replicated data to an Oracle target database.

For an Oracle target database, you can run Replicat in parallel, non-integrated or integrated mode. Oracle recommends that you use the parallel Replicat unless a specific feature requires a different type of Replicat.

The following table lists the features supported by the respective Replicats.

Feature Parallel Replicat Integrated Replicat Coordinated Replicat Classic Replicat

Batch Processing

Yes

Yes

Yes

Yes

Barrier Transactions

Yes

Yes

Yes

No

Dependency Computation

Yes

Yes

No

No

Auto-parallelism

Note:

Auto-parallelism is disabled, by default. Only four threads are used in the default settings. If you want to change Replicat to use MIN_PARALLELISM and MAX_PARALLELISM then auto-parallelism is used.

Yes

Yes

No

No

DML Handler

Yes, Integrated mode

Yes

No

No

Procedural Replication

Yes, used for integrated Parallel Replicat (iPR)

Yes

No

No

Auto CDR

Yes, used by iPR only

Yes

No

No

Dependency-aware Transaction Split

Yes

No

No

No

Cross-RAC-node Processing

Yes

No

Yes

No

ALLOWDUPTARGETMAP

See ALLOWDUPTARGETMAP | NOALLOWDUPTARGETMAP

No. Oracle Database with iPR

No, Oracle Database

Yes

Yes

Topics:

4.1.1 About Parallel Replicat

Parallel Replicat is another variant of Replicat that applies transactions in parallel to improve performance.

It takes into account dependencies between transactions, similar to Integrated Replicat. The dependency computation, parallelism of the mapping and apply is performed outside the database so can be off-loaded to another server. The transaction integrity is maintained in this process. In addition, parallel Replicat supports the parallel apply of large transactions by splitting a large transaction into chunks and applying them in parallel.

Note:

For best performance for an OLTP workload, parallel Replicat in non-integrated mode is recommended.

Only Oracle database supports parallel Replicat and integrated parallel Replicat. However, parallel Replicat supports all databases when using the non-integrated option.

To use parallel Replicat, you need to ensure that you have the following values, which are also the default values:
  • Metadata in the trail (which means you can't use parallel Replicat if your trails are formatted below 12.1.

  • You must have schedulding columns in your trail file.

  • You must use UPDATERCORDFORMAT COMPACT.

With integrated parallel Replicat, the Replicat sends the LCRs to the inbound server, which applies the data to the target database, and in regular parallel Replicat, Oracle GoldenGate applies the LCR as a SQL statement directly to the database, similar to how the other non-integrated Replicats work.

The components of parallel Replicat are:
  • Mappers operate in parallel to read the trail, map trail records, convert the mapped records to the Integrated Replicat LCR format, and send the LCRs to the Merger for further processing. While one Mapper maps one set of transactions, the next Mapper maps the next set of transactions. The the trail information is split and the trail file is untouched because it orders trail information in order.

  • Master processes have two threads, Collater and Scheduler. The Collater receives mapped transactions from the Mappers and puts them back into trail order for dependency calculation. The Scheduler calculates dependencies between transactions, groups transactions into independent batches, and sends the batches to the Appliers to be applied to the target database.

  • Appliers reorder records within a batch for array execution. It applies the batch to the target database and performs error handling. It also tracks applied transactions in checkpoint tables.

Note:

Parallel Replicat requires that any foreign key columns are indexed.
4.1.1.1 Benefits of Parallel Replicat

The following are the benefits of using parallel Replicat.

  • Integrated Parallel Replicat enables heavy workloads to be partitioned automatically among parallel apply processes that apply multiple transactions concurrently, while preserving the integrity and atomicity of the source transaction. Both a minimum and maximum number of apply processes can be configured with the PARALLELISM and MAX_PARALLELISM parameters. Replicat automatically adds additional servers when the workload increases, and then adjusts downward again when the workload lightens.

  • Integrated Parllel Replicat requires minimal work to configure. All work is configured within one Replicat parameter file, without configuring range partitions.

  • High-performance apply streaming is enabled for integrated parallel Replicat by means of a lightweight application programming interface (API) between Replicat and the inbound server.

  • Barrier transactions are coordinated by integrated parallel Replicat among multiple server apply processes.

  • DDL operations are processed as direct transactions that force a barrier by waiting for server processing to complete before the DDL execution.

  • Transient duplicate primary key updates are handled by integrated parallel Replicat in a seamless manner.

  • Parallel Replicat can break a single large transaction into smaller chunks and apply those chunks in parallel. See the SPLIT_TRANS_RECS for details.

4.1.2 About Non-integrated Replicat

In non-integrated mode, the Replicat process uses standard SQL to apply data directly to the target tables. In this mode, Replicat operates as follows:

  • Reads the Oracle GoldenGate trail.

  • Performs data filtering, mapping, and conversion.

  • Constructs SQL statements that represent source database DML or DDL transactions (in committed order).

  • Applies the SQL to the target through Oracle Call Interface (OCI).

The following diagram illustrates the configuration of Replicat in non-integrated mode.

Use non-integrated Replicat when you want to make heavy use of features that are not supported in integrated Replicat mode, see About Integrated Replicat.

You can apply transactions in parallel with a non-integrated Replicat by using a coordinated Replicat configuration.

4.1.3 About Integrated Replicat

In integrated mode, the Replicat process leverages the apply processing functionality that is available within the Oracle Database. In this mode, Replicat operates as follows:

  • Reads the Oracle GoldenGate trail.

  • Performs data filtering, mapping, and conversion.

  • Constructs logical change records (LCR) that represent source database DML transactions (in committed order). DDL is applied directly by Replicat.

  • Attaches to a background process in the target database known as a database inbound server by means of a lightweight streaming interface.

  • Transmits the LCRs to the inbound server, which applies the data to the target database.

The following figure illustrates the configuration of Replicat in integrated mode.

Within a single Replicat configuration, multiple inbound server child processes known as apply servers apply transactions in parallel while preserving the original transaction atomicity. You can increase this parallelism as much as your target system will support when you configure the Replicat process or dynamically as needed. The following diagram illustrates integrated Replicat configured with two parallel apply servers.

Integrated Replicat applies transactions asynchronously. Transactions that do not have interdependencies can be safely executed and committed out of order to achieve fast throughput. Transactions with dependencies are guaranteed to be applied in the same order as on the source.

A reader process in the inbound server computes the dependencies among the transactions in the workload based on the constraints defined at the target database (primary key, unique, foreign key). Barrier transactions and DDL operations are managed automatically, as well. A coordinator process coordinates multiple transactions and maintains order among the apply servers.

If the inbound server does not support a configured feature or column type, Replicat disengages from the inbound server, waits for the inbound server to complete transactions in its queue, and then applies the transaction to the database in direct apply mode through OCI. Replicat resumes processing in integrated mode after applying the direct transaction.

The following features are applied in direct mode by Replicat:

  • DDL operations

  • Sequence operations

  • SQLEXEC parameter within a TABLE or MAP parameter

  • EVENTACTIONS processing

  • UDT

    Note:

    By default, UDT's are applied with the inbound server. Only if NOUSENATIVEOBJSUPPORT is in place, then Extract handling is done by Replicat directly.

Because transactions are applied serially in direct apply mode, heavy use of such operations may reduce the performance of the integrated Replicat mode. Integrated Replicat performs best when most of the apply processing can be performed in integrated mode, see Monitoring and Controlling Processing After the Instantiation in Using Oracle GoldenGate for Oracle Database.

Note:

User exits are executed in integrated mode. However, user exit may produce unexpected results, if the exit code depends on data in the replication stream.

Note:

Integrated Replicat requires that any foreign key columns are indexed.
4.1.3.1 Benefits of Integrated Replicat

The following are the benefits of using integrated Replicat versus nonintegrated Replicat.

  • Integrated Replicat enables heavy workloads to be partitioned automatically among parallel apply processes that apply multiple transactions concurrently, while preserving the integrity and atomicity of the source transaction. Both a minimum and maximum number of apply processes can be configured with the PARALLELISM and MAX_PARALLELISM parameters. Replicat automatically adds additional servers when the workload increases, and then adjusts downward again when the workload lightens.

  • Integrated Replicat requires minimal work to configure. All work is configured within one Replicat parameter file, without configuring range partitions.

  • High-performance apply streaming is enabled for integrated Replicat by means of a lightweight application programming interface (API) between Replicat and the inbound server.

  • Barrier transactions are coordinated by integrated Replicat among multiple server apply processes.

  • DDL operations are processed as direct transactions that force a barrier by waiting for server processing to complete before the DDL execution.

  • Transient duplicate primary key updates are handled by integrated Replicat in a seamless manner.

4.1.3.2 Integrated Replicat Requirements

To use integrated Replicat, the following must be true.

  • Supplemental logging must be enabled on the source database to support the computation of dependencies among tables and scheduling of concurrent transactions on the target. Instructions for enabling the required logging are in Configuring Logging Properties. This logging can be enabled at any time up to, but before, you start the Oracle GoldenGate processes.

  • Integrated Parallel Replicat is supported on Oracle Database 12.2.0.1 and greater.

4.2 About Extract

The Oracle GoldenGate Extract process interacts directly with a database logmining server to receive data changes in the form of logical change records (LCR). The following diagram illustrates the configuration of Extract.

Some of the additional features of Oracle GoldenGate Extract are:

  • Because Extract is fully integrated with the database, features such as Oracle RAC, ASM, and TDE work seamlessly with Oracle GoldenGate.

  • Extract uses the database logmining server to access the Oracle redo stream, with the benefit of being able to automatically switch between different copies of archive logs or different mirrored versions of the online logs. Thus, capture can transparently handle the absence of a log file caused by disk corruption, hardware failure, or operator error, assuming that additional copies of the archived and online logs are available

  • Capture enables faster filtering of tables.

  • Capture handles point-in-time recovery and RAC integration more efficiently.

  • Capture features integrated log management. The Oracle Recovery Manager (RMAN) automatically retains the archive logs that are needed by Extract.

  • Capture supports capture from a multitenant container database and from per-PDB capture mode.

  • Because capture and integrated apply are both database objects, the naming of the objects follow the same rules as other Oracle Database objects, see Specifying Object Names in Oracle GoldenGate Input in Administering Oracle GoldenGate.

4.2.1 Integrated Capture Deployment Options

The deployment options for integrated capture are described in this section and depend on where the mining database is deployed. The mining database is the one where the logmining server is deployed.

  • Local deployment: For a local deployment, the source database and the mining database are the same. The source database is the database for which you want to mine the redo stream to capture changes, and also where you deploy the logmining server. Because integrated capture is fully integrated with the database, this mode does not require any special database setup.

  • Downstream deployment: In a downstream deployment, the source and mining databases are different databases. You create the logmining server at the downstream database. You configure redo transport at the source database to ship the redo logs to the downstream mining database for capture at that location. Using a downstream mining server for capture may be desirable to offload the capture overhead and any other overhead from transformation or other processing from the production server, but requires log shipping and other configuration.

    When using a downstream mining configuration, the source database and mining database must be of the same platform. For example, if the source database is running on Windows 64-bit, the downstream database must also be on a Windows 64-bit platform. See Configuring a Downstream Mining Database and Example Downstream Mining Configuration to configure a downstream mining database.

  • Downstream sourceless Extract deployment: In the Extract parameter file, replace the USERID parameter with NOUSERID. You must use TRANLOGOPTIONS MININGUSER. Extract obtains all required information from the downstream mining database. Extract is not dependent on any connection to the source database. The source database can be shutdown and restarted without affecting Extract.

    Extract will abend if it encounters redo changes that require data to be fetched from the source database.

    To capture any tables that are listed as ID KEY in the dba_goldengate_support_mode view, you need to have a FETCHUSERID or FETCHUSERIDALIAS connection to the support the tables. Tables that are listed as FULL do not require this. We also need to state that if a customer wants to perform SQLEXEC operations that perform a query or execute a stored procedure they cannot use this method as it is incompatible with NOUSERID because SQLEXEC works with USERID or USERIDALIAS.

4.3 Using Different Replicat Modes with Extract

The recommended Oracle GoldenGate configuration, when supported by the Oracle version, is to use one Extract on an Oracle source and one integrated Replicat per source database on an Oracle target.

One integrated Replicat configuration supports all Oracle data types either through the inbound server or by switching to direct apply when necessary, and it preserves source transaction integrity. You can adjust the parallelism settings to the desired apply performance level as needed.

Each Extract group must process objects that are suited to the processing mode, based on table data types and attributes. No objects in one Extract can have DML or DDL dependencies on objects in the other Extract. The same type of segregation must be applied to the Replicat configuration.

If the target database is an Oracle version that does not support integrated Replicat, or if it is a non-Oracle database, you can use a coordinated or parallel Replicat configuration.