6 Running Veridata GoldenGate Parameter Processing

This chapter explains how to use the Veridata GoldenGate Parameter Processing (VGPP) command line tool to use Oracle GoldenGate parameter files.

This chapter includes the following sections:

6.1 Overview of the Command Line Interface

An Oracle GoldenGate parameter file contains all of the information required to extract or apply replicated data. The data propagated through Oracle GoldenGate replication is controlled by settings in the Extract and/or Replicat parameter files. Typically, the Extract parameter file specifies the tables to be replicated and Replicat parameter file controls the mapping of columns, as well as further restricting the tables. The VGPP command line utility accepts one or two Oracle GoldenGate parameter files as input. One of these parameter files must be a Replicat parameter file while the other optional file must be an Extract parameter file.

Oracle GoldenGate replication can capture information about the transaction responsible for changing the data as well as the actual changed data. However, Oracle GoldenGate Veridata can only detect the current state of the source and target databases so it cannot support any Oracle GoldenGate parameters relating to transactional changes. For example, the INSERTDELETES parameter. VGPP uses the MAP statements from a GoldenGate REPLICAT file to generate Veridata compare pairs. Other information in the parameter file is used to determine the relevant MAP statements. Optionally, the TABLE statements from the EXTRACT parameter can be used to restrict the compare pairs that are generated.

Oracle GoldenGate Veridata only support single column mapping. Thus, the column mapping assumes a one to one mapping between source columns and target columns.

You can:

  • Reuse parameter Oracle GoldenGate Replicat and Extract configurations in Oracle GoldenGate Veridata.

  • Avoid creating separate Oracle GoldenGate Veridata Replicat and Extract parameter files.

6.2 Running VGPP

The VGPP program can be run by anyone who has the correct operating system permissions to run it.

  1. Ensure that the parameter files you want to use are on the system where the Oracle GoldenGate Veridata is installed. If you need to copy the files from another system, these files should be copied as binary files so that the CHARSET parameter remains valid.

  2. Run the command shell of the operating system.

  3. Navigate to the VERIDATA_DOMAIN_HOME/veridata/bin directory.

  4. Use the following syntax to run the VGPP program.

Syntax

veridata_param_process{.bat|.sh} required_parameter [optional_parameter]

Required Parameters

The following are required; otherwise an error is returned.

[-noscripting |
-create |
-replace|
-update ]
[-wlport port ] |
-wluser user_name |
[-p <propfile>]
[-o <outputFile>] 
[replicat_param_filename]

The -wluser specifies the WebLogic Server user name to connect to the WebLogic Server. This WebLogic Server user should have the veridataCommandLineUser privilege to access and execute command-line operations. The user should also have the veridataAdministrator or veridataPowerUser privilege to successfully run jobs and to use the Veridata import and export utilities.

Optional Parameter

This is the optional parameter:

[extract_param_filename]

Table 6-1 VGPP Runtime Arguments

Argument Description

{ -noscripting | -create | -replace | -update}

Specifies that veridata_scripting will not be run with the generated configuration file. The -create, -replace, and -update options indicate that veridata_scripting should be run with the generated configuration file. In either case, the generated scripting configuration file can be used as input to the veridata_scripting.

 
-wluser

Specifies the WebLogic Server user name that authenticates and connects to the server.

 
-wlport

Specifies the WebLogic Server port number. The default listening port is 8830.

-p

Specifies a properties file containing additional information required for the Veridata configuration.

-o

Specifies the output file containing the generated Veridata scripting file. The default is replicat_name_scripting.xml; replicat_name is the value of the REPLCAT parameter.

The optional extract_param_filename specifies an EXTRACT parameter file containing source information for the comparison.

The replicat_param_filenameis the REPLICAT parameter containing the target information.

6.2.1 Using a Property File

When the VGPP program is run, an optional property file can be specified. This file contains information that is not available in the Oracle GoldenGate parameter file and is required to generate a valid Veridata comparison configuration. The following are some of properties (information) that you can specify.

Table 6-2 Optional Parameters

Property Name Comments

source.connection.name

The name of the Veridata agent/manager connection. This may be the name of an existing Veridata connection.

The default is the Extract name. This is the only source connection property needed to reference an existing connection.

source.connection.port

The port for the source agent. This is required when the connection does not already exist.

source.connection.host

The host name where the source agent is running. This is required when the connection does not exist.

source.connection.user

This defaults to the user information in the extract parameter file. This is required when the connection does not exist.

source.connection.password

This defaults to the user information in the extract parameter file. This is required when the connection does not already exist. If property name is specified without a value, the scripting utility will prompt for the value when the scripting configuration is loaded into Veridata.

source.catalog

This is valid for Sybase, SQL Server, and Oracle consolidated databases.

For Sybase and SQL Server, it is the database containing the source tables.

For Oracle, it specifies the Oracle PDB to use when processing an Extract parameter. Statements not associated with this PDB are ignored. The default value is the first PDB reference in the file. The reference can be a SOURCECATALOG parameter or the first part of a three-part name in a TABLE statement.

extract.useansiquotes

Indicates whether or not the Extract parameter file follows the ANSI quotation specification. This is a Boolean value. The default value is true. This is the same as the GoldenGate core GLOBALS parameters USEANSISQLQUOTES | NOUSEANSISQLQUOTES.

extract.charset

The character set for the extract parameter file. This overrides any charset specified in the extract parameter file.

Extract.trail

The trail file name to use when more than one trail file is specified in an extract parameter file. The default is the first trail file specified in the extract parameter file. When an extract parameter file contains multiple rmttail entries with the same name, Veridata will use the first occurrence.

target.connection.name

The name of the Veridata agent/manager connection. This may be the name of an existing Veridata connection.

The default is the Replicat name. This is the only target connection property required to reference an existing connection.

target.connection.port

The port for the target agent. This is required when the connection does not already exist.

target.connection.host

The host name where the target agent is running. This is required when the connection does not exist.

target.connection.user

This defaults to the user information in the parameter file

target.connection.password

This defaults to the user information in the extract parameter file. If property name is specified without a value, the scripting utility prompts for the value when the scripting configuration is loaded into Veridata.

Target.catalog

This is valid for Sybase and SQL Server. It is the database containing the target tables.

replicat.useansiquotes

Indicates whether or not the replicat parameter file follows the ANSI quotation specification. This is Boolean value. The default value is true. This is the same as the GoldenGate core GLOBALS parameters USEANSISQLQUOTES | NOUSEANSISQLQUOTES.

replicat.charset

The character set for the replicat parameter file. This overrides any CHARSET specification in the replicat parameter file.

6.3 Parameter Handling

This section describes the handling of all of the parameters allowed in an Oracle GoldenGate Extract or Replicat parameter file. Each keyword is either supported, unsupported, or ignored. A supported parameter is used to generate the Veridata configuration. An unsupported parameter is something that interferes with the Veridata configuration generation. When an unsupported parameter (INSERTDELETES) is specified, subsequent MAP parameters are ignored. An ignored parameter specifies a feature that is not applicable to Veridata configuration generation.

The following table contains the known parameters and the expected handling: if a parameter is not listed, it is ignored.

Table 6-3 Parameter Handling

GoldenGate Parameters Veridata Support

CATALOGEXCLUDE

This parameter is ignored. Veridata only processes items from a single catalog.

CHARMAP

Unsupported.

CHARSET

Supported. This parameter is supported in parameter files and include/obey files. Veridata does not process GLOBALS files.

COLMATCH

Supported.

COMMENT | --

Supported.

DICTIONARY

Supported for NSK extract and replicat.

EXPANDDDL

Supported for NSK extract and replicat.

EXCLUDEWILDCARDOBJECTSONLY

Supported.

EXTRACT

Supported.

EXTTRAIL

Supported.

FILE | TABLE

Supported for NSK extract.

INCLUDE

Supported. When the file is not found by the specified path, VGPP will look for the file name in the same directory as the parameter file.

INSERTALLRECORDS

Unsupported. Ignore all subsequent MAP statements.

INSERTDELETES | NOINSERTDELETES

Unsupported | Supported. Ignore all MAP statements following an INSERTDELETES command until a NOINSERTDELETES command is found.

INSERTUPDATES | NOINSERTUPDATES

Unsupported| Supported: Ignore all MAP statements between the INSERTUPDATES and the NOINSERTUPDATES.

MACRO

Supported.

MACROCHAR

Supported.

MAP

Supported.

MAPEXCLUDE

Supported.

OBEY

Supported. The same as INCLUDE.

REPLICAT

Supported.

RMTTRAIL

Supported.

SCHEMAEXCLUDE

Supported.

SOURCECATALOG

Supported for Oracle consolidated databases.

TABLE | MAP

Supported. The details are explained in section 3.4.4

TABLEEXCLUDE

Supported.

UPDATEDELETES | NOUPDATEDELETES

Unsupported | Supported.

UPDATEINSERTS | NOUPDATEINSERTS

Unsupported | Supported.

USEANSISQLQUOTES | NOUSEANSISQLQUOTES

Supported.

6.4 Map and Table Statement Handling

Veridata will generate a compare pair element in the scripting configuration file for each Map statement in the Replicat parameter file. The generated scripting file will list the specific table mappings first, followed by the wildcard mappings, and finally the excluded mappings. This matches the behavior of the Oracle GoldenGate Replicat where specific mappings take precedence over wildcard mappings.

When the same source and target table specification appears in multiple MAP statements, the first occurrence will be used for the compare pair specification. The multiple occurrences can occur when the MAP statements use thread specifications and range filters.

The following table lists all of the keywords for the MAP and TABLE statements and support level in VGPP. MAP statements containing unsupported keywords will not generate a Veridata comparison configuration. Items marked with maybe indicate that more information is needed in order to determine the value for Veridata.

Table 6-4 Map and Table Statement Handling

Keyword Veridata Support

TARGET

Supported.

COLMAP

Supported. Only simple source column to target column mapping is supported. Target columns mapped to functions or literals is excluded from the comparison configuration.

The USEDEFAULTS keyword is supported.

The BINARYINPUT keyword is ignored.

COLS

Supported.Results in an explicit column list in the generated compare pair configuration.

COLSEXCEPT

Supported. If an explicit column mapping does not exist, this results in system mapped columns with a list of omitted columns.

COMPARECOLS

  

COORDINATED

Ignored.

DICTIONARY

Supported for NSK.

TARGETDICT

Supported for NSK.

DEF

Supported for NSK.

TARGETDEF

Supported for NSK.

EVENTACTIONS

Ignored.

EXCEPTIONSONLY

Unsupported.

EXITPARAM

Ignored.

FETCHBEFOREFILTER

  

FETCHCOLS|FETCHCOLSEXCEPT

  

FETCHMODCOLS|FETCHMODCOLSEXCEPT

  

FILTER

Ignored.

GETBEFORECOLS

Ignored.

HANDLECOLLISIONS | NOHANDLECOLLISIONS

  

INSERTALLRECORDS

Unsupported.

INSERTAPPEND | NOINSERTAPPEND

  

KEYCOLS

Supported.

MAPEXCEPTION

Ignored.

REPERROR

  

RESOLVECONFLICT

Ignored.

SQLEXEC

  

SQLPREDICATE

  

THREAD

Ignored.

THREADRANGE

Ignored.

TOKENS

Ignored.

TRIMSPACES | NOTRIMSPACES

Supported.

TRIMVARSPACES | NOTRIMVARSPACES

Supported.

WHERE

Ignored.