6 Configuring Change Synchronization
Online change synchronization extracts data and transmits it to a target. Here you see how to prepare Extract, Replicat, and Logger, and how to work with parameter files.
This topic includes the following:
Introduction
If your application is TMF-protected, perform change synchronization using Extract and Replicat. Non-TMF applications use Logger and Replicat to perform the same functions. You can configure Oracle GoldenGate to process changes from the following sources:
-
A TMF audit trail
-
An intermediate Oracle GoldenGate trail, created either by Logger or Extract
-
Directly from entry-sequenced files, or from BASE24,
TLF
orPTLF
files. -
Oracle GoldenGate logs generated from non-TMF applications
Change Synchronization for TMF Applications
TMF application change synchronization requires at least one Extract group, one trail, and one Replicat group.
Configuring Extract
To configure and run Extract, you must create an Extract group and an Extract parameter file.
Example 6-1 Sample Extract Parameter File
-- Extract parameter file for -- TCUSTMER and TCUSTORD changes EXTRACT EXTORD GETROLLBACKS EXTTRAIL \LA.$DATA03.JDSDAT.ET TABLE $DATA11.JDSSOU.TCUSTMER; TABLE $DATA11.JDSSOU.TCUSTORD;
The name of the parameter file is usually the same as the process group name. For more information on parameter guidelines, see "Working with Parameter Files".
Configuring Trails
Based on considerations such as performance, hard disk constraints, and data throughput speed, you can specify where you want your trails to reside. For example, if you are concerned about disk space in your Extract environment, you may choose to create your trails on the system where Replicat is installed.
To configure your trail, you must create an empty trail using GGSCI, then start its associated process. You can test the trail by checking to see data is being written to it. If an INFO ALL
command shows data being written to your trail, it is configured correctly.
To add an Oracle GoldenGate trail:
-
Determine if your trail will run locally or remotely. Base this decision on performance considerations vs. data throughput speed.
-
If you are not at the GGSCI prompt, start GGSCI.
TACL> RUN GGSCI
-
Add your trail using the following commands:
GGSCI> ADD EXTTRAIL trail_path, EXTRACT extract_group, [trail_size_limit], [limit_of_files_waiting_to_be_written] GGSCI> ADD RMTTRAIL trail_path, EXTRACT extract_group, [trail_size_limit]
For example, a local trail would read:
GGSCI> ADD EXTTRAIL \LA.$DATA03.JDSDAT.ET, EXTRACT EXTORD, MEGABYTES 5, MAXFILES 10
To test a trail:
-
Issue the GGSCI command
INFO ALL
. -
Check to make sure Extract and Replicat are both running, and checkpoint sizes show relative byte addresses.
Configuring Replicat
Replicat gathers data from your trail and delivers it to your target. A Replicat group contains the named Replicat itself, a Replicat parameter file, and checkpoint groups, as desired.
To configure Replicat:
-
From GGSCI, create a Replicat group:
GGSCI> ADD REPLICAT group_name, EXTTRAIL trail_name
For example:
GGSCI> ADD REPLICAT REPORD, EXTTRAIL $DATA03.JDSDAT.ET
-
Launch a NonStop text editor to create a Replicat parameter file (or use GGSCI):
TACL> TEDIT PARAMS REPORD
-
Enter your parameters as desired.
The name of the parameter file is usually the same as the process group name. For more information on parameter guidelines, see "Working with Parameter Files". Following is a sample Replicat parameter file.
-- Replicat parameter file for replicating -- TCUSTMER and TCUSTORD changes REPLICAT REPORD HANDLECOLLISIONS PURGEOLDEXTRACTS ASSUMETARGETDEFS DISCARDFILE \LA.$DATA11.GGSDISC.REPORD, PURGE MAP \LA.$DATA11.GGSSOU.TCUSTORD, TARGET \NY.$DATA03.GGSTAR.TCUSTORD; MAP \LA.$DATA11.GGSSOU.TCUSTMER,TARGET \NY.$DATA03.GGSTAR.TCUSTMER;
-
Start GGSCI, then start the Replicat you just configured.
GGSCI> START REPLICAT REPORD
-
Verify that Replicat is working and receiving data from Extract.
Change Synchronization for Non-TMF Applications
The Logger program, with the GGSLIB run-time library, extracts changed records from files that are not protected by TMF. A Logger process records database updates in a log trail, which feeds data to Replicat. Each Logger process in a set is named $GGLnn
, where nn
is a sequenced identifier beginning with 00
. For example, if you configure two Logger processes, they are named $GGL00
and $GGL01
. Each Logger group has one or more Logger process, a parameter file, one or more log trails, and file extraction lists.
Log trails are sets of files, written to disk, that hold data extracted and sent to a particular Logger. Each log trail is owned by one Logger process. The parameter file holds specific volume locations and the number and size of each log file.
A log trail's name is comprised of the volume and subvolume where Oracle GoldenGate Logger is installed, the process prefix, and a series of letters and numbers that grow depending on the number of log trails. For example, $DATA1.GLOGGGL.AA000000
means Oracle GoldenGate Logger is installed on volume Data1
, subvolume GLOG
, the process prefix is GGL
, and the trail itself is called AA000000
.
To configure and run change synchronization for non-TMF applications, you must:
- Create the
LOGPARM
parameter file. - Configure Logger and GGSLIB with the
ADD LOGGER
command. - Start Logger.
- Bind GGSLIB to the non-TMF application.
Creating the LOGPARM File
Just as Extract and Replicat are controlled by parameter files, so is Logger. Unlike either Extract or Replicat, you must create the LOGPARM
before you add your Logger to Oracle GoldenGate Manager.
To create a Logger parameter file:
Logger parameters are detailed in Reference for Oracle GoldenGate on HP NonStop Guardian. The following section outlines a sample Logger parameter file.
Sample LOGPARM File
This sample parameter file configures two log processes $GGL00
and $GGL01.
These process names have been explicitly set with the PROCESS
parameter, but when not set the names default to $GGLnn.
The system will increment nn from 00
so the default will generate the same process names in this instance.
Note:
Parameter position is important. As soon as a log entry is specified with the log parameter, it becomes the current log. Parameters entered below the current log parameter apply only to the current log. For instance, in the following example, all parameters after the LOG $D3.GGSLOG.AA
and before the LOG $D15.GGSLOG.BB
entry apply to LOG $D3.GGSLOG.AA
.
-
Creates a log trail
$D3.GGSLOG.AA
that contains10
files each sized at500
megabytes (for a total of 5,000 megabytes). The file names will beAA000000,
AA000001
, throughAA000009
. As new files are required, the oldest one is recycled and takes the next sequence number; in this case,AA000000
will becomeAA000010
. File space is pre-allocated by the GGSCI and Manager processes. -
Configures
$GGL00
to run onCPU 9
, with backupCPU 4
. -
Specifies that data written by the application in
$DATA4
will be logged to the log trail$D3.GGSLOG.AA
. -
Specifies that data written by the application in
$DATA5
will be logged to the log trail$D3.GGSLOG.AA
. -
Excludes
$DATA4.REPORTS.*
from being logged toAA
. -
Excludes
$DATA4.DAT.TRANSFL
from being logged toAA
.
Logger GGL01:
-
Creates a log trail
$D15.GGSLOG.BB
that contains100
files each sized at10
megabytes (for a total of 1,000 megabytes). The file names will beBB000000
,BB000001
, throughBB000009
. These files are recycled when needed. -
Configures
$GGL01
to run inCPU 3,
with backupCPU 2
. -
Specifies that data written by the application to files in any location should be written to the
BB
log trail, except$DATA4.REPORTS.*
and any data already captured by$GGL00
(in this case,$DATA4.*.*
and$DATA5.*.*
).$DATA4.DAT.TRANSFL
will be captured inBB
since it was implicitly included in $*.*.* and excluded nowhere for this logger.
Example 6-2 Sample Logger Parameter File
-- Logger configuration for two Loggers LOG $D3.GGSLOG.AA, PROCESS $GGL00, MEGABYTES 500, NUMFILES 10, SECURE "NUUU" CPU 9,4 FILE $DATA4.*.* FILE $DATA5.*.* EXCLUDEFILE $DATA4.REPORTS.* EXCLUDEFILE $DATA4.DAT.TRANSFL LOG $D15.GGSLOG.BB, PROCESS $GGL01, MEGABYTES 100, NUMFILES 10, SECURE "NUUU" CPU 3,2 FILE $*.*.* EXCLUDEFILE $DATA4.REPORTS.*
Logger GGL00:
Configuring Logger and GGSLIB
Run the ADD LOGGER
command to process the configuration in LOGPARM
. This step establishes a configuration for both Logger and GGSLIB and pre-allocates disk files for each Logger process to use for logging database updates.
Before starting Logger, Oracle GoldenGate must process and store its configuration. This step pre-allocates file space for each log trail to ensure extracted records can be stored.
To process the Logger configuration, enter the following command.
GGSCI> ADD LOGGER
Using Macros to Bind GGSLIB to a Non-TMF Application
Use the GGSCI BIND PROGRAMS
command to bind the GGSLIB library to your non-TMF application. This step also binds GGSLIB to existing user libraries the application calls.
TACL > RUN GGSCI GGSCI> BIND PROGRAMS
BIND PROGRAMS
prompts for a list of programs to bind with GGSLIB. GGSCI analyzes this list to see which files are already bound and which ones it must bind.
In this context, bound means that GGSLIB runs as a user library in the application program (BIND CHANGE LIBRARY GGSLIB
in program_name is performed). GGSLIB is not literally bound with the application program. If a program already calls a user library, that library is literally bound with GGSLIB to create a new library. The library will have the same name as the old user library.
Building GGSLIB
GGSLIB, built as part of installation, contains the BASELIB module that intercepts Guardian function calls made by the application. GGSLIB also contains C, CRE and COBOL run-time libraries that call Guardian functions. When bound to GGSLIB, these libraries attempt to call the operating system function, but actually call the Oracle GoldenGate function instead. GGSLIB in turn calls the intended operating system function transparently to the application. GGSLIB uses a shared extended memory segment for efficient configuration storage, and maintains a private memory segment for working storage variables.
Without the presence of these libraries, the C and COBOL run-time libraries would be called at the operating system level and would bypass Oracle GoldenGate intercept functions.
Therefore, build these libraries carefully. Keep the following libraries up-to-date with your latest operating system release and related IPMs. Not all of these libraries are required in the GGSLIB build if your application does not run COBOL74, COBOL85 or C routines. It is recommended, however, to bind each of these components that exist on your system into GGSLIB.
Library | Function |
---|---|
$SYSTEM.ZCOBOLRT.CLIBOBJ |
COBOL74 routines |
$SYSTEM.ZCOB85RT.C8LIB |
COBOL85 routines |
$SYSTEM.ZCRERTL.CFELIB |
Common Run-time Environment |
$SYSTEM.SYSTEM.CRELIB |
Common Run-time Environment |
$SYSTEM.SYSTEM.COBOLLIB |
More COBOL85 routines |
Private Memory and Stack Space
GGSLIB routines minimize stack space requirements. By doing so, programs are ensured that typical activities will have enough stack room left for themselves.
For its own working space, GGSLIB allocates a small private memory segment to handle in-transit I/O buffers and keep its own state variables. This requires approximately 250 words.
Alternate Methods of Binding GGSLIB to an Application
There are alternatives to using the Oracle GoldenGate macros (NLDLIB
for example) to bind the Oracle GoldenGate intercept library to your application. These alternatives may vary depending on your NonStop environment.
For non-native mode systems, a type 100 object file is produced using the TAL, COBOL, or C language compilers. Native mode Itanium systems use EPTAL, ECOBOL or CCOMP to compile type 800 objects.
Using the ?Search Directive
You can connect the application to the Oracle GoldenGate intercept library by using the ?SEARCH
directive in the compile. This copies the library into the application object file. The drawback to this method is that an upgrade to the Oracle GoldenGate application or the operating system will not be picked up by the built-in modules of these programs. A recompile is required to replace the modules.
Non-Native Environments
You can bind the intercept library to application programs in non-native environments by using:
-
The NonStop
BIND
utilityBIND CHANGE LIBRARY $vol.subvol.library IN application_object
-
A
/LIB /
parameter in the run statementRUN application_object/LIB $vol.subvol.library/
-
SET SERVER GUARDIAN-LIB
parameter if it is a Pathway server
Native Mode Itanium Systems
The native mode Itanium system does not require any special steps. The intercept library can be bound to the application by any of the following.
-
Using the TNS/E Link edit (ELD) utility change command
ELD -CHANGE LIBNAME '$vol.svol.library' application_object
-
A
/LIB /
parameter in the run statementRUN application_program/LIB Oracle_GoldenGate_library/
-
Using the server parameter
GUARDIAN-LIB.
Libraries for Native Applications
If your NonStop environment is running in native mode, you may decide to use native mode libraries so processes run more efficiently. You must use native mode modules and libraries if you are using the encryption or compression capabilities of Oracle GoldenGate. Oracle GoldenGate provides a TACL macro, NLDLIB
, for building the following native libraries:
-
BASELIBR: A relinkable, native version of
BASELIB
, a module that intercepts function calls made by the application. -
GGSDLL: A native version of
BASELIB
for use as a dynamically linked library (DLL
) on the operating systems. -
GGSLIBR: A relinkable, native
BASELIB
containing CRE and COBOL SRLs.Note:
Applications running on the operating systems that include native C, native COBOL, and pTAL require two intercept libraries. The one to be linked to the C and COBOL applications should reference the COBOL and CRE dynamic -link libraries, and the one for pTAL should not. This is due to the limitation that pTAL does not perform the necessary initialization of the run-time environment.
Running NLDLIB
Running the NLDLIB macro lets you create these libraries and combine them with the native mode Oracle GoldenGate BASELIBN library and certain Guardian system libraries. You can run NLDLIB as part of your initial installation routine or on its own.
You can also run the NLDLIB
macro from the TACL prompt providing arguments in the command line. This is not recommended, however, as it may produce unexpected results. Interactive responses help ensure the appropriate options for your environment.
Activating Authorization of Bound Libraries
You can add the Oracle GoldenGate SFGEXIT module to Safeguard to produce a warning for any program that opens non-audited files for update and does not have the Oracle GoldenGate intercept library bound to it. See "Authentication for Bound Programs" for more information.
Note:
Opens on SQL tables, unstructured files, and TMF protected files are always ignored. Opens from processes on remote nodes are also ignored.
You can enable the authorization event and supply optional PARAM
-TEXT
arguments when the program is added.
The syntax for the ADD
within Safecom is:
=ADD EVENT-EXIT-PROCESS OPENCHECK PROG $vol.subvol.SFGEXIT [, PNAME process_name] [, ENABLE-AUTHORIZATION-EVENT {ON | OFF}] [, ENABLE {ON | OFF}] [, PARAM-TEXT [, DETAIL] | [, OSOPENSUMMARY| OSOPENDETAIL | NOOSOPENS] [, AUDCFG filename [REJECT]] ]
Adding and Verifying the Authorization Event
The following steps show examples that add, set options, check the status, and remove the authorization event.
Using Different PARAM-TEXT Options
Other examples of setting options when adding the authorization event are shown below.
Working with Parameter Files
Parameters give you complete control over all aspects of Oracle GoldenGate, such as:
-
Data selection, mapping, and transformation
-
Replication
-
Error resolution
-
Logging
-
Status and error reporting
-
System resource usage
-
Startup and run-time activities
There can be only one active parameter file for each Manager, Extract, or Replicat. There are two types of parameters: global and file-specific.
-
Global parameters apply to all tables specified in the parameter file for synchronization. Some global parameters affect processing while others affect such things as memory utilization.
-
File or table-specific parameters control processing for tables specified with a
FILE
,TABLE
orMAP
statement. Table-specific parameters enable you to designate one set of processing rules for some tables, while designating other rules for other tables. There are two implementations for file-specific parameters:-
Toggling the parameter on and off around one or more
FILE
,TABLE
orMAP
statements. -
Adding the parameter within
MAP
statement so it applies only to that table or file.
-
Some parameters, such as HANDLECOLLISIONS
/NOHANDLECOLLISIONS
can be included in a MAP
statement or toggled ON
and OFF
. Others can be implemented using only one of the methods. For further details, see Oracle GoldenGate Parameters.
The ordering of parameters in a parameter file can be important.
-
A global parameter can appear anywhere in the parameter file, and it should only be listed in the file once. When listed more than once, only the last instance of the parameter is active. All other instances are ignored.
-
Table-specific parameters take effect in the order that each parameter is listed in the file.
Table 6-1 Basic Extract and Replicat Parameter Files
Sample Extract parameter file | Sample Replicat parameter file |
---|---|
EXTRACT NYTOLA DISCARDFILE =DISCARD_FILE, PURGE EXTTRAIL $DATA1.EXTDAT.XX FILE $DATA2.FINANCE.ACCOUNTS; |
REPLICAT NYTOLA DISCARDFILE =$DATA.GGSDISC.NYTOLA, PURGE ASSUMETARGETDEFS MAP $DATA2.FINANCE.ACCOUNT, TARGET $BACK.FINANCE.ACCOUNTS; |
Creating a Parameter File
From the subvolume where Oracle GoldenGate is installed, create a parameter file using the NonStop text editor. The name of the parameter file is usually the same as the process group name. For example, if you created the Extract group ADD EXTRACT NYTOLA
, you would create your parameter file by entering TEDIT PARAMS NYTOLA
.
To create a parameter file through GGSCI
-
From the Oracle GoldenGate installation location, run the GGSCI command-line user interface.
-
In GGSCI, issue the following command to open the default text editor.
GGSCI> EDIT PARAMS group_name
Where:
group_name is either
MGRPARM
(for the Manager process),LOGPARM,
or the name of the Extract or Replicat group for which the file is being created. The name of an Extract or Replicat parameter file must match that of its process group.Examples:
-
The following creates or edits the parameter file for an Extract group named
EXTORA
.GGSCI> EDIT PARAMS EXTORA
-
The following creates or edits the parameter file for the Manager process.
GGSCI> EDIT PARAMS MGRPARM
-
The following creates or edits the parameter file for the Manager process.
GGSCI> EDIT PARAMS LOGPARM
-
-
Using the editing functions of the editor, enter as many comment lines as you want to describe this file, making certain that each line is commented out by two hyphens (--). As an alternative, you can use the
COMMENT
parameter, which causes everything on the same line as theCOMMENT
parameter to be ignored. The syntax forCOMMENT
is:COMMENT comment_text
Note:
Do not put a dash or pound symbol before the
COMMENT
keyword. Do not useCOMMENT
if any column names in the tables contain the word “comment." Instead, use double hyphens (--). -
On non-commented lines, enter the parameters for your synchronization configuration, starting a new line for each parameter statement.
For parameters that accept table names, you can use an asterisk (*) wildcard to match any number of characters.
Parameters have the following general syntax:
parameter argument [, option] [&]
Where:
-
parameter is the parameter name.
-
argument is a required argument for the parameter. Some parameters take arguments, while others do not. Separate all arguments with commas, as in the following example:
USERID ggs, PASSWORD ggs123 RMTHOST sysb, MGRPORT 8040 RMTFILE $DATA05.GGSDAT.C1, PURGE
Note:
Use a maximum of six characters to name any volume that identifies files or tables in parameter files.
$DATA05
is supported, but$DATA011
is not. -
option is an optional argument.
-
&
enables you to continue a parameter's arguments on another line. Place it at the end of the line to be continued.Note:
Ampersands (
&
) are not always required to span more than one line, but it is a good practice to use ampersands when:-
A parameter is not terminated by a semicolon and the line extends beyond 79 characters
-
A line for any of the options used for the parameter extend beyond 79 characters.
-
-
Save and close the file.
-
Storing Parameter Files
By default, parameter files are stored in the GGSPARM
subvolume. If you are not going to use the default location, create the new location before starting Oracle GoldenGate. You can change this default location using an ADD DEFINE
parameter in the GLOBALS
parameter file, such as the one in the following example.
TACL> ADD DEFINE =GGS_PARAMS, CLASS DEFAULTS, VOLUME $VOL.SUBVOL
Once paired with a process, a parameter file must remain in its original location for Oracle GoldenGate to operate properly.
Viewing a Parameter File
You can view a parameter file by issuing the GGSCI VIEW PARAMS
command.
VIEW PARAMS filename
VIEW PARAMS
displays the file.
Table 6-2 summarizes the various ways in which you can scroll through its contents.
Table 6-2 Parameter File Scrolling Commands
Command | Description |
---|---|
return, n |
Next page |
/string |
Search for next occurrence of string in file |
number |
Go to line indicated by number |
l |
Go to last page of file |
b |
Go backwards one page in file |
q |
Quit display |
h |
Help |
Reference for Oracle GoldenGate on HP NonStop Guardian has a complete list of commands.
Changing a Parameter File
You make changes to an Oracle GoldenGate NonStop process parameter file by editing it using the NonStop text editor or some other compatible editor.
To ensure that all changes you make to the Manager parameter file are activated you must stop and restart the Manager process. To change an Extract or Replicat parameter file, make your changes, then verify them with the CHECKPARAMS
parameter as described in "Verifying a Parameter File".
Using OBEY and Macros in Parameters
You can leverage existing parameter files through the Oracle GoldenGate macros and the OBEY
command. To simplify the process, you can use Oracle GoldenGate macros for a variety of operations, including implementing multiple uses of a statement, consolidating multiple commands, or invoking other macros. You also can use OBEY
to direct Oracle GoldenGate to retrieve parameter settings from another parameter file. Upon encountering OBEY
, Oracle GoldenGate processes the parameters from the other file and then returns to the current file to process any remaining instructions.
See "Configuring Custom Operations" for more information about using macros and OBEY
files.
Verifying a Parameter File
Use the following procedure to confirm that the syntax in an Extract or Replicat parameter file is correct:
Substituting a Parameter
It is possible to assign different values to a parameter within a parameter file. One-off change synchronization runs that require specific parameters can process with the same parameter file as your default change synchronization routine; any difference in parameter requirements is handled by parameter substitution. This minimizes your need for multiple parameter files.
To include a run-time parameter within the parameter file, precede any intended parameter name with a question mark. Then, before running the Extract process, use the TACL PARAMS
command to pass the value.
When you are ready to run your special data run, specify the following from your TACL prompt:
TACL> PARAM EXTFILE $DATA2.GGS.EXTFILE TACL> PARAM TABNAME $DATA3.MYDB.ACCOUNTS TACL> PARAM REGION EAST TACL> RUN EXTRACT /IN PARMFL/
Extract will interpret the parameter as follows:
SOURCEISFILE EXTFILE $DATA2.GGS.EXTFILE TABLE $DATA3.MYDB.ACCOUNTS, WHERE (REGION = "EAST");
Note:
A question mark can also be used as a wildcard so care should be exercised in using PARAMS
and wildcards together. The program will process parameter substitutions first, before evaluating wildcards. It cannot distinguish, however, between ?DATA
as a parameter and ?DATA
as a wildcard, so it is important that the user selects parameter names that are never used as part of an actual file name.
Example 6-3 Parameter File Contents
SOURCEISFILE EXTFILE ?extfile TABLE ?tabname, WHERE (REGION = "?region");