8 Getting Started with the Oracle GoldenGate Process Interfaces

This chapter describes how Oracle GoldenGate users provide instructions to the processes through the Oracle GoldenGate Software command line interface (CLI), batch and shell scripts, and parameter files. Oracle GoldenGate CLI offers two options: Admin Client and GGSCI.

Topics:

8.1 Using Command Line Interfaces

To start either the Admin Client or GGSCI, you need to change the current working directory to the Oracle GoldenGate home directory (OGG_HOME).

Note:

The environment variable OGG_HOME and OGG_VAR_HOME must be set before starting the Admin Client or GGSCI.

Topics:

8.1.1 Using GGSCI

Run the ggsci executable file.

If you don't set the current working directory to the Oracle GoldenGate home directory, then errors similar to the following may occur:

GGSCI (20) 2> stop mgr 
Manager process is required by other GGS processes. Are you sure you want to stop it (y/n)?y
Transparent Integration with XAG is enabled. Sending the command STOP MANAGER to XAG...
ERROR: No Oracle GoldenGate instance is found in XAG resource list.

8.1.2 Using Wildcards in Command Arguments

You can use wildcards with certain Oracle GoldenGate commands to control multiple Extract and Replicat groups as a unit. The wildcard symbol that is supported by Oracle GoldenGate is the asterisk (*). An asterisk represents any number of characters. For example, to start all Extract groups whose names contain the letter X, issue the following command.

START EXTRACT *X*

8.1.3 Globalization Support for the Command Interface

All command input and related console output are rendered in the default character set of the local operating system. To specify characters that are not compatible with the character set of the local operating system, use Unicode notation. For example, the following Unicode notation is equivalent to the name of a table that has the Euro symbol as its name:

ADD TRANDATA \u20AC1

For more information, see Support for Escape Sequences for more information about using Unicode notation.

Note:

Oracle GoldenGate group names are case-insensitive.

8.1.4 Using Command History

The execution of multiple commands is made easier with the following tools:

  • Use the HISTORY command to display a list of previously executed commands.

  • Use the ! command to execute a previous command again without editing it.

  • Use the FC command to edit a previous command and then execute it again.

8.1.5 Storing and Calling Frequently Used Command Sequences

You can automate a frequently-used series of commands by using an OBEY file and the OBEY command. The OBEY file takes the character set of the local operating system. To specify a character that is not compatible with that character set, use the Unicode notation.

See Support for Escape Sequences for more information about using Unicode notation.

To use OBEY

  1. Create and save a text file that contains the commands, one command per line. This is your OBEY file. The name can be anything supported by the operating system. You can nest other OBEY files within an OBEY file.
  2. Run the Admin Client or GGSCI.
  3. (Optional) If using an OBEY file that contains nested OBEY files, issue the following command. This command enables the use of nested OBEY files for the current session and is required whenever using nested OBEY files. See Reference for Oracle GoldenGate for more information.
    ALLOWNESTED
    
  4. Call the OBEY file by using the OBEY command from the command line interface (Admin Client or GGSCI).
    OBEY file_name
    

    Where:

    file_name is the relative or fully qualified name of the OBEY file.

Example 8-1 OBEY command file

ADD EXTRACT myext, TRANLOG, BEGIN now
START EXTRACT myext

ADD REPLICAT myrep, EXTTRAIL /ggs/dirdat/aa
START REPLICAT myrep

INFO EXTRACT myext, DETAIL
INFO REPLICAT myrep, DETAIL

The following example illustrates an OBEY command file for use with the OBEY command. It creates and starts Extract and Replicat groups and retrieves processing information.

See Reference for Oracle GoldenGate for more information about the OBEY command.

8.2 Controlling Oracle GoldenGate Processes

The standard way to control Oracle GoldenGate processes is through the GGSCI interface. Typically, the first time that Oracle GoldenGate processes are started in a production setting is during the initial synchronization process (also called instantiation process). However, you will need to stop and start the processes at various points as needed to perform maintenance, upgrades, troubleshooting, or other tasks.

These instructions show basic syntax.

Topics:

8.2.1 Controlling Manager

Manager should not be stopped unless you want to stop replication processing.

To Stop Manager

  1. From the Oracle GoldenGate directory, run GGSCI.
  2. In GGSCI, issue the following command.

    {START | STOP [!]} MANAGER

    Where:

    The ! bypasses the prompt that confirms the intent to shut down Manager.

Note:

When starting Manager from the command line or GGSCI with User Account Control enabled, you will receive a UAC prompt requesting you to allow or deny the program to run.

8.2.2 Controlling Extract and Replicat

Here are basic directions for controlling Extract and Replicat processes.

To Start Extract or Replicat

START {EXTRACT | REPLICAT} group_name

Where:

group_name is the name of the Extract or Replicat group or a wildcard set of groups (for example, * or fin*).

To Stop Extract or Replicat Gracefully

STOP {EXTRACT | REPLICAT} group_name

Where:

group_name is the name of the Extract or Replicat group or a wildcard set of groups (for example, * or fin*).

To Stop Replicat Forcefully

STOP REPLICAT group_name !

The current transaction is aborted and the process stops immediately. You cannot stop Extract forcefully.

To End a Process that STOP Cannot Stop

KILL {EXTRACT | REPLICAT} group_name

Ending a process does not shut it down gracefully, and checkpoint information can be lost.

To Control Multiple Processes at Once

command ER wildcard specification

Where:

  • command is: KILL, START, or STOP
  • wildcard specification is a wildcard specification for the names of the process groups that you want to affect with the command. The command affects every Extract and Replicat group that satisfies the wildcard. Oracle GoldenGate supports up to 100,000 wildcard entries.

8.2.3 Deleting Extract and Replicat

This section contains basic directions for deleting Extract and Replicat processes. See Reference for Oracle GoldenGate for additional command options.

To Delete an Extract Group

  1. Run GGSCI.

  2. Issue the DBLOGIN command as the Extract database user (or a user with the same privileges). You can use either of the following commands, depending on whether a local credential store exists.

    DBLOGIN [SOURCEDB dsn] {USERID user, PASSWORD password [encryption_options] | USERIDALIAS alias [DOMAIN domain]}
    
  3. Stop the Extract process.

    STOP EXTRACT group_name

  4. Issue the following command.

    DELETE EXTRACT group_name

  5. (Oracle) Unregister the Extract group from the database.

    UNREGISTER EXTRACT group_name,database_name

To Delete a Replicat Group

  1. Stop the Replicat process.

    STOP REPLICAT group_name

  2. Issue one of the following commands from GGSCI to log into the database.
    DBLOGIN [SOURCEDB dsn] {USERID user, PASSWORD password [encryption_options] | USERIDALIAS alias [DOMAIN domain]}
    

    Where:

    • SOURCEDB dsn supplies the data source name, if required as part of the connection information.

    • USERID user, PASSWORD password specifies an explicit database login credential.

    • USERIDALIAS alias [DOMAIN domain] specifies an alias and optional domain of a credential that is stored in a local credential store.

    • encryption_options is one of the options that encrypt the password.

  3. Issue the following command to delete the group.

    DELETE REPLICAT group_name

Deleting a Replicat group preserves the checkpoints in the checkpoint table (if being used). Deleting a process group also preserves the parameter file. You can create the same group again, using the same parameter file, or you can delete the parameter file to remove the group's configuration permanently.

8.3 Automating Commands

Oracle GoldenGate supports the issuing of commands through scripts or jobs. This section describes these options for UNIX- or Linux-based platforms and the IBMi platform.

On a UNIX or Linux system, or within a runtime environment that supports UNIX or Linux applications, you can issue Oracle GoldenGate commands from a script such as a startup script, shutdown script, or failover script by running GGSCI and calling an input file. The script file must be encoded in the operating system character set. Unicode notation can be used for characters that are not supported by the operating system character set. Before creating a script, see Globalization Support for the Command Interface.

To Input a Script

Use the following syntax from the command line of the operating system.

ggsci < input_file

Where:

  • The angle bracket (<) character pipes the file into the GGSCI program.

  • input_file is a text file, known as an OBEY file, containing the commands that you want to issue, in the order they are to be issued.

Note:

To stop the Manager process from a batch file, make certain to add the ! argument to the end of the STOP MANAGER command. Otherwise, GGSCI issues a prompt that requires a response and causes the process to enter into a loop.

8.3.1 Issuing Commands Through the IBM i CLI

Oracle GoldenGate for IBM DB2 for i includes a set of native IBM i commands that enables the operation of the most common Oracle GoldenGate programs from the IBM i command-line interface (CLI). Because these commands are native, they do not need to be run from a PASE environment. With this support, it is possible to issue commands interactively or by using the typical job submission tools such as SBMJOB to operate Oracle GoldenGate non-interactively.

The commands are as follows and correspond to the Oracle GoldenGate programs of the same name. They reside in the Oracle GoldenGate installation library.

DEFGEN

EXTRACT

KEYGEN

LOGDUMP

MGR

REPLICAT

8.4 Specifying Object Names in Oracle GoldenGate Input

The following rules apply when specifying object names in parameter files (such as in TABLE and MAP statements), column-conversion functions, commands, and in other input.

Topics:

8.4.1 Specifying Filesystem Path Names in Parameter Files on Windows Systems

On Windows systems, if the name of any directory in a filesystem path name begins with a number, the path must be specified with forward slashes, not backward slashes, when listing that path in Oracle GoldenGate input, such as parameter files or commands. This requirement prevents Oracle GoldenGate from interpreting the name as an octal escape sequence. For example, the following paths contain a directory named \2014 that will be interpreted as the octal sequence \201:

C:\ogg\2014\install\dirdat\aa
C:\ogg\install\2014\dirdat\aa

The preceding path can be used with forward slashes as follows:

C:/ogg/2014/install/dirdat/aa
C:/ogg/install/2014/dirdat/aa

For more information, see Support for Escape Sequences.

8.4.2 Supported Database Object Names

Object names in parameter files, command, and other input can be any length and in any supported character set. For supported character sets, see Supported Character Sets.

Oracle GoldenGate supports most characters in object and column names. Specify object names in double quote marks if they contain special characters such as white spaces or symbols.

The following lists of supported and non-supported characters covers all databases supported by Oracle GoldenGate; a given database platform may or may not support all listed characters.

Topics:

8.4.2.1 Supported Special Characters

Oracle GoldenGate supports all characters that are supported by the database, including the following special characters. Object names that contain these special characters must be enclosed within double quotes in parameter files.

Character Description

/

Forward slash (See Specifying Names that Contain Slashes)

*

Asterisk (Must be escaped by a backward slash when used in parameter file, as in: \*)

?

Question mark (Must be escaped by a backward slash when used in parameter file, as in: \?)

@

At symbol (Supported, but is often used as a resource locator by databases. May cause problems in object names)

#

Pound symbol

$

Dollar symbol

%

Percent symbol (Must be %% when used in parameter file)

^

Caret symbol

( )

Open and close parentheses

_

Underscore

-

Dash

<space>

Space

8.4.2.2 Non-supported Special Characters

The following characters are not supported in object names and non-key column names.

Character Description

\

Backward slash (Must be \\ when used in parameter file)

{ }

Begin and end curly brackets (braces)

[ ]

Begin and end brackets

=

Equal symbol

+

Plus sign

!

Exclamation point

~

Tilde

|

Pipe

&

Ampersand

:

Colon

;

Semi-colon

,

Comma

' '

Single quotes

" "

Double quotes

'

Accent mark (Diacritical mark)

.

Period

<

Less-than symbol (or beginning angle bracket)

>

Greater-than symbol (or ending angle bracket)

8.4.3 Specifying Names that Contain Slashes

If a table name contains a forward-slash character (/) in any part of its name, that name component must be enclosed within double quotes unless the object name is from an IBM i platform . The following are some examples:

"c/d" 
"/a".b 
a."b/"

If the name contains a forward slash that is not enclosed within double quotes, Oracle GoldenGate treats it as a name that originated on the IBM i platform (from a DB2 for i database). The forward slash in the name is interpreted as a separator character.

8.4.4 Qualifying Database Object Names

Object names must be fully qualified in the parameter file. This means that every name specification must be qualified, not only those supplied as input to Oracle GoldenGate parameter syntax, but also names in a SQL procedure or query that is supplied as SQLEXEC input, names in user exit input, and all other input supplied in the parameter file.

Oracle GoldenGate supports two-part and three-part object names, as appropriate for the database.

Topics:

8.4.4.1 Two-part Names

Most databases require only two-part names to be specified, in the following format:

owner.object

For example: HR.EMP

Where:

owner is a schema or database, depending on how the database defines a logical namespace that contains database objects. object is a table or other supported database object.

The databases for which Oracle GoldenGate supports two-part names are as follows, shown with their appropriate two-part naming convention:

  • Db2 for i: schema.object and library/file(member)

  • Db2 LUW: schema.object

  • Db2 on z/OS: schema.object

  • MySQL: database.object

  • Oracle Database (non-CDB databases): schema.object

  • SQL Server: schema.object

  • Teradata: database.object

8.4.4.2 Three-part Names

Oracle GoldenGate supports three-part names for the following databases:

  • Oracle container databases (CDB)

Three-part names are required to capture from a source Oracle container database because one Extract group can capture from more than one container. Thus, the name of the container, as well as the schema, must be specified for each object or objects in an Extract TABLE statement.

Specify a three-part Oracle CDB name as follows:

container.schema.object

For example: PDBEAST.HR.EMP

8.4.4.3 Applying Data from Multiple Containers or Catalogs

To apply data captured from multiple source containers or catalogs to a target Oracle container database, both three- and two-part names are required. In the MAP portion of the MAP statement, each source object must be associated with a container or catalog, just as it was in the TABLE statement. This enables you (and Replicat) to properly map data from multiple source containers or catalogs to the appropriate target objects. In the TARGET portion of the MAP statement, however, only two-part names are required. This is because Replicat can connect to only one target container or catalog at a time, and schema.owner is a sufficient qualifier. Multiple Replicat groups are required to support multiple target containers or catalogs. Specify the target container or catalog with the TARGETDB parameter.

8.4.4.4 Specifying a Default Container or Catalog

You can use the SOURCECATALOG parameter to specify a default catalog for any subsequent TABLE, MAP, (or Oracle SEQUENCE) specifications in the parameter file.

The following example shows the use of SOURCECATALOG to specify the default Oracle PDB named pdbeast for region and jobs objects, and the default PDB named pdbwest for appraisal objects. The objects in pdbeast are specified with a fully qualified three-part name, which does not require a default catalog to be specified.

TABLE pdbeast.hr.emp*; 
SOURCECATALOG pdbeast 
TABLE region.country*; 
TABLE jobs.desg*; 
SOURCECATALOG pdbwest 
TABLE appraisal.sal*;

8.4.5 Specifying Case-Sensitive Database Object Names

Oracle GoldenGate supports case-sensitive names. Follow these rules when specifying case-sensitive objects.

  • Specify object names from a case-sensitive database in the same case that is used to store them in the host database. Keep in mind that, in some database types, different levels of the database can have different case-sensitivity, such as case-sensitive schema but case-insensitive table. If the database requires quotes to enforce case-sensitivity, put quotes around each object that is case-sensitive in the qualified name.

    Correct: TABLE "Sales"."ACCOUNT"

    Incorrect: TABLE "Sales.ACCOUNT"

  • Oracle GoldenGate converts case-insensitive names to the case in which they are stored when required for mapping purposes.

Table 8-1 provides an overview of the support for case-sensitivity in object names, per supported database. Refer to the database documentation for details on this type of support.

Table 8-1 Case Sensitivity of Object Names Per Database

Database Requires quotes to enforce case-sensitivity? Unquoted object name Quoted object name

DB2

Yes. Differentiates between case-sensitive and case-insensitive by use of quotes.

Case-insensitive, stores in upper case

Case-sensitive, stores in mixed case

MySQL

(Case-sensitive database)

No

  • Always case-sensitive, stores in mixed case

  • The names of columns, triggers, and procedures are case-insensitive

No effect

No effect

Oracle Database

Yes. Differentiates between case-sensitive and case-insensitive by use of quotes.

Case-insensitive, stores in upper case

Case-sensitive, stores in mixed case

SQL Server

(Database created as case-sensitive)

No

Always case-sensitive, stores in mixed case

No effect

No effect

SQL Server

(Database created as case-insensitive)

No

Always case-insensitive, stores in mixed case

No effect

No effect

Teradata

No

Always case-insensitive, stores in mixed case

No effect

No effect

Note:

For all supported databases, passwords are always treated as case-sensitive regardless of whether the associated object name is quoted or unquoted.

8.4.6 Using Wildcards in Database Object Names

You can use wildcards for any part of a fully qualified object name, if supported for the specific database. These name parts can be the following: the container, database, or catalog name, the owner (schema or database name), and table or sequence name. For specifics on how object names and wildcards are supported, see the Oracle GoldenGate installation and configuration guide for that database.

Where appropriate, Oracle GoldenGate parameters permit the use of two wildcard types to specify multiple objects in one statement:

  • A question mark (?) replaces one character. For example in a schema that contains tables named TABn, where n is from 0 to 9, a wildcard specification of HQ.TAB? returns HQ.TAB0, HQ.TAB1, HQ.TAB2, and so on, up to HQ.TAB9, but no others. This wildcard is not supported for the DB2 LUW database nor for DEFGEN. This wildcard can only be used to specify source objects in a TABLE or MAP parameter. It cannot be used to specify target objects in the TARGET clause of TABLE or MAP.

  • An asterisk (*) represents any number of characters (including zero sequence). For example, the specification of HQ.T* could return such objects as HQ.TOTAL, HQ.T123, and HQ.T. This wildcard is valid for all database types throughout all Oracle GoldenGate commands and parameters where a wildcard is allowed.

  • In TABLE and MAP statements, you can combine the asterisk and question-mark wildcard characters in source object names only.

Topics:

8.4.6.1 Rules for Using Wildcards for Source Objects

For source objects, you can use the asterisk alone or with a partial name. For example, the following source specifications are valid:

  • TABLE HQ.*;

  • TABLE PDB*.HQ.*;

  • MAP HQ.T_*;

  • MAP HQ.T_*, TARGET HQ.*;

The TABLE, MAP and SEQUENCE parameters take the case-sensitivity and locale of the database into account for wildcard resolution. For databases that are created as case-sensitive or case-insensitive, the wildcard matches the exact name and case. For example, if the database is case-sensitive, SCHEMA.TABLE is matched to SCHEMA.TABLE, Schema.Table is matched to Schema.Table, and so forth. If the database is case-insensitive, the matching is not case-sensitive.

For databases that can have both case-sensitive and case-insensitive object names in the same database instance, with the use of quote marks to enforce case-sensitivity, the wildcarding works differently. When used alone for a source name in a TABLE statement, an asterisk wildcard matches any character, whether or not the asterisk is within quotes. The following statements produce the same results:

TABLE hr.*;
TABLE hr."*";

Similarly, a question mark wildcard used alone matches any single character, whether or not it is within quotes. The following produce the same results:

TABLE hr.?;
TABLE hr."?";

If a question mark or asterisk wildcard is used with other characters, case-sensitivity is applied to the non-wildcard characters, but the wildcard matches both case-sensitive and case-insensitive names.

  • The following TABLE statements capture any table name that begins with lower-case abc. The quoted name case is preserved and a case-sensitive match is applied. It captures table names that include "abcA" and "abca" because the wildcard matches both case-sensitive and case-insensitive characters.

    TABLE hr."abc*";
    TABLE hr."abc?";
    
  • The following TABLE statements capture any table name that begins with upper-case ABC, because the partial name is case-insensitive (no quotes) and is stored in upper case by this database. However, because the wildcard matches both case-sensitive and case-insensitive characters, this example captures table names that include ABCA and "ABCa".

    TABLE hr.abc*;
    TABLE hr.abc?;
8.4.6.2 Rules for Using Wildcards for Target Objects

When using wildcards in the TARGET clause of a MAP statement, the target objects must exist in the target database. (The exception is when DDL replication is being used, which allows new schemas and their objects to be replicated as they are created.)

For target objects, only an asterisk can be used. If an asterisk wildcard is used with a partial name, Replicat replaces the wildcard with the entire name of the corresponding source object. Therefore, specifications such as the following are incorrect:

TABLE HQ.T_*, TARGET RPT.T_*;
MAP HQ.T_*, TARGET RPT.T_*;

The preceding mappings produce incorrect results, because the wildcard in the target specification is replaced with T_TEST (the name of a source object), making the whole target name T_T_TESTn. The following illustrates the incorrect results:

  • HQ.T_TEST1 maps to RPT.T_T_TEST1

  • HQ.T_TEST2 maps to RPT.T_T_TEST2

  • (The same pattern applies to all other HQ.T_TESTn mappings.)

The following examples show the correct use of asterisk wildcards.

MAP HQ.T_*, TARGET RPT.*;

The preceding example produces the following correct results:

  • HQ.T_TEST1 maps to RPT.T_TEST1

  • HQ.T_TEST2 maps to RPT.T_TEST2

  • (The same pattern applies to all other HQ.T_TESTn mappings.)

8.4.6.3 Fallback Name Mapping

Oracle GoldenGate has a fallback mapping mechanism in the event that a source name cannot be mapped to a target name. If an exact match cannot be found on the target for a case-sensitive source object, Replicat tries to map the source name to the same name in upper or lower case (depending on the database type) on the target. Fallback name mapping is controlled by the NAMEMATCH parameters. For more information, see Reference for Oracle GoldenGate.

8.4.6.4 Wildcard Mapping from Pre-11.2.1 Trail Version

If Replicat is configured to read from a trail file that is a version prior to Oracle GoldenGate 11.2.1, the target mapping is made in the following manner to provide backward compatibility.

  • Quoted object names are case-sensitive.

  • Unquoted object names are case-insensitive.

The following maps a case-sensitive table name "abc" to target "abc". This only happens with a trail that was written by pre-11.2.1 Extract for SQL Server databases with a case-sensitive configuration. In this example, if the target database is Oracle Database or DB2 fallback name mapping is performed if the target database does not contain case-sensitive "abc" but does have table ABC. (See Fallback Name Mapping.)

MAP hq."abc", TARGET hq.*;

The following example maps a case-insensitive table name abc to target table name ABC. Previous releases of Oracle GoldenGate stored case-insensitive object names to the trail in upper case; thus the target table name is always upper cased. For case-insensitive name conversion, the comparison is in uppercase, A to Z characters only, in US-ASCII without taking locale into consideration.

MAP hq.abc, TARGET hq.*;
8.4.6.5 Asterisks or Question Marks as Literals in Object Names

If the name of an object itself includes an asterisk or a question mark, the entire name must be escaped and placed within double quotes, as in the following example:

TABLE HT."\?ABC";
8.4.6.6 How Wildcards are Resolved

By default, when an object name is wildcarded, the resolution for that object occurs when the first row from the source object is processed. (By contrast, when the name of an object is stated explicitly, its resolution occurs at process startup.) To change the rules for resolving wildcards, use the WILDCARDRESOLVE parameter. The default is DYNAMIC.

8.4.6.7 Excluding Objects from a Wildcard Specification

You can combine the use of wildcard object selection with explicit object exclusion by using the EXCLUDEWILDCARDOBJECTSONLY, CATALOGEXCLUDE, SCHEMAEXCLUDE, MAPEXCLUDE, and TABLEEXCLUDE parameters.

8.4.7 Differentiating Case-Sensitive Column Names from Literals

By default, Oracle GoldenGate follows SQL-92 rules for specifying column names and literals. In Oracle GoldenGate parameter files, conversion functions, user exits, and commands, case-sensitive column names must be enclosed within double quotes if the database requires quotes around a name to support case-sensitivity. For example:

"columnA"

Case-sensitive column names in databases that do not require quotes to enforce case-sensitivity must be specified as they are stored in the database. For example:

ColumnA

Literals must be enclosed within single quotes. In the following example, Product_Code is a case-sensitive column name in an Oracle database, and the other strings are literals.

@CASE ("Product_Code", 'CAR', 'A car', 'TRUCK', 'A truck')