Improving AWT Throughput with Parallel Propagation to the Oracle Database

To improve throughput for an AWT cache group, you can configure multiple threads that act in parallel to propagate and apply transactional changes to the Oracle database. Parallel propagation enforces transactional dependencies and applies changes in AWT cache tables to Oracle Database tables in commit order.

Parallel propagation is supported for AWT cache groups with the following configurations:

AWT cache groups involved in an active standby pair replication scheme
AWT cache groups in a single TimesTen database (without a replication scheme configuration)
AWT cache groups configured with any aging policy

The following data store attributes enable parallel propagation and control the number of threads that operate in parallel to propagate changes from AWT cache tables to the corresponding Oracle Database tables:

ReplicationApplyOrdering enables parallel propagation by default.
ReplicationParallelism defines the number of transmitter threads on the source database and the number of receiver threads on the target database for parallel replication in a replication scheme. This value can be between 2 and 32 when used solely for parallel replication. The default is 1. In addition, the value of ReplicationParellelism cannot exceed half the value of LogBufParallelism.
CacheAWTParallelism, when set, determines the number of threads used in parallel propagation of changes from AWT cache tables to the Oracle Database tables. Set this attribute to a number from 2 to 31. The default is 1.

Parallel propagation for an AWT cache group is configured with one of the following scenarios:

ReplicationApplyOrdering is set to 0 and ReplicationParallelism is greater than 1.

If you do not set CacheAWTParallelism, the number of threads that apply changes to Oracle Database is 2 times the setting for ReplicationParallelism. For example, if ReplicationParallelism=3, the number of threads that apply changes to Oracle Database tables is 6. In this case, ReplicationParallelism can only be set from 2 to 16; otherwise, twice the value would exceed the maximum number of 31 threads for parallel propagation. If the value is set to 16, the maximum number of threads defaults to 31.
ReplicationApplyOrdering is set to 0, ReplicationParallelism is equal to or greater than 1, and CacheAWTParallelism is greater than 1. The value for CacheAWTParallelism must be greater than or equal to the value set for ReplicationParallelism and less than or equal to 31.

If CacheAWTParallelism is not specified, then ReplicationParallelism is used to determine the number of threads that are used for parallel propagation to Oracle Database. However, since this value is doubled for parallel propagation threads, you can only set ReplicationParallelism to a number from 2 to 16. If the value is set to 16, the maximum number of threads defaults to 31.

If both ReplicationParallelism and CacheAWTParallelism attributes are set, the value set in CacheAWTParallelism configures the number of threads used for parallel propagation. The setting for CacheAWTParallelism determines the number of apply threads for parallel propagation and the setting for ReplicationParallelism determines the number of threads for parallel replication. Thus, if ReplicationParallelism is set to 4 and CacheAWTParallelism is set to 6, then the number of threads that apply changes to Oracle Database tables is 6. This enables the number of threads used to be different for parallel replication and parallel propagation to Oracle Database tables.

Note:

See Configuring Parallel Replication in the Oracle TimesTen In-Memory Database Replication Guide. See ReplicationApplyOrdering, ReplicationParallelism, and CacheAWTParallelism in the Oracle TimesTen In-Memory Database Reference.

These data store attributes are interrelated. Table 7-1 shows the result with the combination of the various possible attribute values.

Table 7-1 Results of Parallel Propagation Data Store Attribute Relationships

ReplicationApply Ordering	ReplicationParallelism	CacheAWTParallelism	Number of Parallel Propagation Threads
Set to 0, which enables parallel propagation	Set to > 1 for multiple tracks and <= 16.	Not specified.	Set to twice the value of `ReplicationParallelism`.
Set to 0, which enables parallel propagation	Set to > 16 and <= 32 for multiple tracks.	Not specified.	Error is thrown. If `CacheAWTParallelism` is not set, then 2 times the value set in `ReplicationParallelism` specifies the number of threads. Thus, in this case, `ReplicationParallelism` cannot be greater than 16.
Set to 0, which enables parallel propagation	Set to > 1 and <= 32 for multiple tracks.	Set to >= to `ReplicationParallelism`.	Set to number specified by `CacheAWTParallelism`.
Set to 0, which enables parallel propagation	Set to > 1 and <= 32 for multiple tracks.	Set to < `ReplicationParallelism`.	Error is thrown at database creation. The `CacheAWTParallelism` must be set to a value greater than or equal to `ReplicationParallelism`.
Set to 0, which enables parallel propagation	Set to 1 or not specified. Single track.	Set to > 1	Set to number specified by `CacheAWTParallelism`.
Set to 1, which disables parallel propagation.	N/A	Set to > 1	Error is thrown at database creation, since parallelism is turned off, but `CacheAWTParallelism` is set to a value, expecting parallel propagation to be enabled.

Foreign keys in Oracle Database tables that are to be cached must have indexes created on the foreign keys. Consider these Oracle Database tables:

CREATE TABLE parent (c1 NUMBER PRIMARY KEY NOT NULL);
CREATE TABLE child (c1 NUMBER PRIMARY KEY NOT NULL, 
                    c2 NUMBER REFERENCES parent(c1));
CREATE TABLE grchild (c1 NUMBER PRIMARY KEY NOT NULL, 
                      c2 NUMBER REFERENCES parent(c1), 
                      c3 NUMBER REFERENCES parent(c1));

These indexes must be created:

CREATE INDEX idx_1 ON child(c2);
CREATE INDEX idx_2 ON grchild(c2);
CREATE INDEX idx_3 ON grchild(c3);

The following sections describe restrictions, configuration and checks for parallel propagation: