5 Using the Veridata Import and Export Utilities

This chapter describes how to use the Veridata Import and Export utilities. In addition to using the Oracle GoldenGate Veridata Web User Interface, you can use the import and export utilities provided with the Veridata installation to define portions of your configuration.

This chapter includes the following sections:

5.1 Introduction to the Import and Export Utilities

Using the import and export utilities, you can create XML documents that are used to configure Oracle GoldenGate Veridata. The DTD (Document Type Definition) that governs these XML documents is stored in the ORACLE_HOME/veridata/clilib/lib/veridata-scripting.jar file.

The import utility allows you to configure database connections, comparison groups including compare pairs, comparison jobs, and profiles. It takes an XML document as input then creates comparison objects in Veridata. Typically, the XML document matches the inputs on the configuration pages in the user interface.

The export utility helps you to either selectively or completely export the compare configuration data to an XML file. It can be used to export configurations from your current Veridata configuration or from other Veridata configurations using the -repoUrl option. Additionally, you can use it to export configurations between different Veridata repository types using the import functionality. For example, from a SQL Server configuration to an Oracle configuration.

You should have an understanding of basic XML and its rules.

These utilities provide the following advantages:

  • It can reduce the time required to define repetitive tasks

  • It allows you to create reusable configurations

  • It can ensure that your test configuration mirrors the one you use for production

5.1.1 Supported Configurations

Oracle GoldenGate Veridata import and export utilities support configuring:

  • Database connections

  • Comparison groups (jobs, groups, and compare pairs)

  • Profiles

5.2 Running the Import and Export Utilities

The import and export utilities run from the DOMAIN_HOME/veridata/bin directory of the Oracle GoldenGate Veridata installation location. The Windows programs are veridata_export.bat and veridata_import.bat; the UNIX and Linux scripts are veridata_export.sh and veridata_import.sh.

5.2.1 Using the Export Utility

The syntax for running the export utility is:

veridata_export[.sh | .bat] -export filePath -wluser commandlineUusername   [-wlport portNo] [–jobs jobName | - groups groupName | -connections connName | -profiles profileName | -all | -exportPassword] [[-repoUrl jdbc_url] [-u username>][-schema schema_name][-vdtPath VERIDATA_PRODUCT_HOME]]
  • -wlport: Represents the port for Veridata web server. The default value is 8830.

  • -wluser: Specifies a user commandlineUsername with Veridata configuration privileges and command-line privileges.

  • One of these optional operations can be requested at run time:

    • -jobs: Export all jobs, by name, including the associated groups, connections and profiles. You can specify one or more jobs by separating the names with a space, such as job1 job2 job3. If no job name is specified, all jobs with associated objects are exported.

    • -groups: Export all groups in the repository or add group names separated by a space, such as group1 group2 group3.

    • -connections: Export all connections in the repository or add connections separated by a space, such as conn1 conn2 conn3.

    • -profiles: Export all profiles in the repository or add profiles separated by a space, such as profile1 profile2 profile3.

    • -all: Export objects that are not part of any job. Takes precedence over all optional operations. This is the default when no other options are specified.

    • -exportPassword: Export the passwords for connections. By default, passwords for connections are not exported.

    • -repoUrl: Set the remote Veridata repository database JDBC URL for the export to use. You must set the -u option when using -repoUrl.

      For Oracle Database:
        jdbc:oracle:thin:@hostname/ip:port:SID 
      

      or

       jdbc:oracle:thin:@hostname/ip:port/serviceName
      
      For SQL Server:
       jdbc:weblogic:sqlserver://hostname/ip:port;databaseName= databaseName
      

      The repoUrl may require double quotes.

    • -u: Set the remote Veridata database username from which the configuration export is requested. Use with the -repoUrl option.

    • -schema: Set the remote Veridata schema name from which the configuration export is requested.

    • -vdtPath veridataLocation: Set the Veridata domain location for the 12c release and later. For 11g release, it is the installation location.

    • -help: Provides command line syntax.

I f you want to export data from a Veridata repository database that is not part of your existing installation, you must provide the URL, username, and schema name. You will be prompted to enter the external Veridata repository password during run time. The Connection passwords are not exported by default, use the -exportPassword option for exporting passwords.

5.2.2 Using the Import Utility

The syntax for running the import utility is:

veridata_import[.sh | .bat] [-wlport portNo] -wluser commandlineUusername [-create | -update | -delete | -replace] configuration.xml
  • -wlport: Represents the port for Veridata web server. The default value is 8830.

  • -wluser: Specifies a user commandlineUsername with Veridata configuration privileges and command-line privileges.

  • One of these optional operations can be requested at run time:

    • -create: All items listed in the configuration are new. If any item in the list exists in the repository, nothing is added.

    • -update: New items are added to the repository and existing items are modified. Items existing in the repository and not listed in the configuration are deleted.

    • -delete: All named items that exist for the configuration are removed from the repository.

    • -replace: All items listed to be replaced in the configuration are replaced as specified.

  • configuration.xml: The name of the import XML configuration file that you created to describe the configuration. This is a required option.

5.2.3 Processing the Configuration

The import utility first parses the configuration.xml file attempting to complete the entire file before aborting due to the errors. Any errors it finds are logged in the DOMAIN_HOME/veridata/logs/veridata_import.log. If it does not abort because of errors, it makes a second parsing pass, this time processing the configuration.

Matching Object Names

Database object names, such as catalogs, schema, tables, indexes, and columns will be matched according to these rules:

  • The matching is case insensitive

  • The hyphen (-) is considered a match to the underscore (_) to support matching Enscribe DDL and SQL columns

  • Wildcard expressions for table names and source column names match against the exact name and against the uppercase version of the name.

  • Quoted names for schemas and wildcards match everything within the quotations must be matched exactly. A wildcard character within quotes is treated as an ordinary character. An example of a quoted name as it would appear in the XML is:

    source-table=""CHAR_TYPES"*"
    

    This would match CHAR_TYPES, CHAR_TYPES2, and CHAR_TYPES_NOTNULL.

  • Filters can either include or exclude schemas and tables. If include filters are used, at least one filter must be matched before a table can be included in a compare pair. If exclude filters are used, a table is excluded if it matches any exclude filter. Include filters can include a COLFILTER element that contains a list of columns to include or exclude. When a table matches a include filter, the include filter's COLFILTER is used to specify the columns for the generated compare pair. The schema and table name can use wildcards.

    For NonStop Enscribe files, file pattern filters are used. The file pattern is any valid NonStop file name pattern.

  • A compare pair may have a column specification with the Boolean attribute "optional". When this attribute is true, the column is only included in the compare pair if the source table includes the specified source column.

Determining Key Columns

The key columns are selected in the following order:

  1. Explicit key column definitions if they are available. In this case if source-pkey and target-pkey compare-pair element attributes are set it will generate an error.

  2. Columns in the index specified by source-pkey and target-pkey attributes of the compare-pair element. The number of columns and all data types must match and the data types must be compatible.

  3. Columns in the system-selected primary key.

Generating Compare Pairs

Compare pair generation has the following characteristics:

  • Generating from wild cards works the same as the user interface generation except that regular expressions can be used.

  • Compare pairs are processed in the order specified in the configuration.xml file

  • The compare pairs generated by a single compare pair element are generated in alphabetical order of the source table name.

  • When compare pairs are generated by more than one compare pair element, the first one will be used.

As a general rule, the order of the compare pair elements should be:

  1. Compare pairs with specialized configuration requirements, such as user-defined keys.

  2. Compare pairs that match general patterns.

  3. Exclusions of compare pairs that would otherwise match general patterns.

5.3 Configuration File Element Reference

The configuration is defined by the top level configuration element and several nested elements. Most of these elements have attributes that define their characteristics, such as the operation attribute for the configuration element or the port attribute for the connection element.

The following is the high-level element hierarchy in the configuration XML file. For more information about an element and its attributes, click the element name in the hierarchy.

configuration
    connection
       conn-properties
    group
       description
       filter
       sql-partition
       enscribe-key
       compare-pair
          enscribe-info
          enscribe-key
          sql-partition
          column
          excluded-column
          delta-config
    job
    profile
    

Appendix B provides a sample configuration file.

Appendix C provides a description of the profile parameters that you can use to configure your profiles.

configuration

The root element is configuration.

The following elements can be nested within the configuration element:

Table 5-1 configuration Elements

Elements Description
connection

One or more Veridata database connection definitions.

group

One or more Veridata comparison group definitions.

job

One or more comparison job definitions.

profile

One or more profile definitions.


The following attributes describe the configuration element:

Table 5-2 configuration Attributes

Attribute Description
validation

Specifies the type of validation that is used for the configuration. The options are:

"required" - All compare pairs must be successfully validated before any pairs are added to the repository. This is the default value.

"omit-failures" - Successfully validated compare pairs are added to the repository and compare pairs that cannot be validated are ignored.

"none" - Compare pairs are added to the repository without any validation. If this option is selected, the Oracle GoldenGate Veridata Web User Interface should be used to review and fix validation problems.

operation

Specifies how data is applied to the repository. The options are:

"create" - All items listed in the configuration are new. If any item in the list exists in the repository, nothing is added. This can be used to prevent unintended modification to existing repository items. This is the default value.

"update" - New items are added to the repository and existing items modified. Items existing in the repository and not listed in the configuration are deleted.

"delete" - All named items in the configuration are removed from the repository.

You can use a command line flag to override the value entered for this attribute.

wildcard

Specifies the pattern matching method that is used. The options are:

"ggs" - Use the typical Oracle GoldenGate pattern using an asterisk (*). See the Oracle GoldenGate Veridata Web User Interface help for details on this type of matching. This is the default value.

"regex" - Use regular expressions for matching.


Example

The following example adds compare pairs that can be validated and ignores those that cannot; uses regular expressions for wild carding; and uses the "create" default to adds all items as new items, adding nothing if any item already exists.

<configuration validation="omit-failures" wildcard="regex">
.
.
.
</configuration>

column

The column element defines a set of columns to be included or excluded from the compare pair. The column element has no nested elements or text data.

The following attributes describe the column element:

Table 5-3 column attributes

Attribute Description
source-name

A regular expression that defines a set of source column names. This value is required.

target-name

A regular expression that defines a set of target column names. It can include references to groups captured by the source-name expression.

exclude

Indicates whether or not the matched columns should be excluded from the compare pair. The options are:

"true" - The matched columns should be excluded.

"false" - The matched columns should be included. This is the default.

type

Indicates the type of the column. The options are:

"key" - The column is used as a key.

"hash" - The column is compared using a hash value. This is the default value.

"literal"- The column is a literal value.

format

Specifies a format to override the comparison format that would normally be used.

scale

Specifies a scale to override the default scale for the comparison.

precision

Specifies a precision to override the default precision used for the comparison.

timezone

Specifies a time zone to override the default time zone of the comparison.

optional

Indicates whether the column mapping is optional. For example, mapping will not fail if the base tables do not have the column patterns specified. Default is "false".


colfilter

The colfilter element defines a set of columns to be included or excluded. It is used to specify the names of the columns to use as filtering criteria.

The following element describes the colfilter element:

Table 5-4 colfilter Element

Attribute Description
colfiltercol

Specifies a set of columns to be included or excluded.


The following attribute describes the colfilter element:

Table 5-5 colfilter Attribute

Attribute Description
type

Specifies whether to include the columns or exclude them. The options are include or exclude; the default is include. This is a required attribute.


Example

This example excludes COL3 and COL5 for the table TABLE_NAME from the generated compare pair.

<filter type="include" table="TABLE_NAME">
    <colfilter type ="exclude">
        <colfiltercol name="COL3" />
        <colfiltercol name="COL5" />
    </colfilter>
</filter>

colfiltercol

The colfiltercol element defines a set of columns to be included or excluded. It is used to specify the names of the columns to use as filtering criteria.

The following attribute describes the colfiltercol element:

Table 5-6 colfiltercol Attribute

Attribute Description
name

A regular expression that defines a set of source column names. This is a required attribute.


compare-pair

The compare-pair element specifies a set of compare pair items. As in the Oracle GoldenGate Veridata Web User Interface, the compare pairs default to system mapped keys and columns.

The following elements can be nested within the compare-pair element:

Table 5-7 compare-pair Elements

Element Description
enscribe-info

One or more sets of information used when comparing NonStop Enscribe files.

sql-partition

One or more specifications of a subset of rows within the table.

enscribe-key

One or more specifications of a subset of records within an Enscribe file.

key-column

A set of columns to be used as the user-defined key for the comparison.

column

One or more definitions of a set of columns to be included.

excluded-column

Defines a set of columns to excluded from the a compare pair when the compare pair uses system mapped columns.

delta-config

Defines the delta processing configuration for the compare pair. The maximum is to add it once per compare pair.


The following attributes describe the compare-pair element:

Table 5-8 compare-pair Attributes

Attribute Element
name

An expression defining the name of the compare pair. This expression can include groups captured with source-table expressions and target table group $0.

source-table

A regular expression that defines the table or tables to be compared. See "Regular Expression Grouping" later in this section for more detail. The default is to match all tables.

target-table

A regular expression that defines the target tables for the comparison. This may contain references to groups captured by the source table expression. The default is $0 for the full source table name.

source-schema

The name of the default schema for the source tables referenced for the compare pair. The default is the value specified for the group. For SQL/MP, this is the subvolume of the SQL catalog. This is not used with Enscribe files.

target-schema

The name of the default schema for the target tables referenced for the compare pair. The default is the value specified for the group. For SQL/MP, this is the subvolume of the SQL catalog. This is not used with Enscribe files.

source-catalog

The default catalog for the source tables referenced in this compare pair. For SQL/MP, this is the volume of the SQL catalog. This is not used for Oracle, DB2, Enscribe, or Teradata.

target-catalog

The default catalog for the source tables referenced in this compare pair. For SQL/MP, this is the volume of the SQL catalog. This is not used for Oracle, DB2, Enscribe, or Teradata.

exclude

Indicates whether or not the compare pair should be included in the group element. This can be used to remove a compare pair generated by an earlier compare pair element. The options are:

"true" - Exclude the compare pair.

"false" - Include the compare pair. This is the default.

source-file-pattern

The default file pattern for the source if the data source is Enscribe or SQL/MP.

target-file-pattern

The default file pattern for the target if the data target is Enscribe or SQL/MP.

source-pkey

The name of the unique index to use as the source portion of the user-specified primary key. The default is no user-specified index name.

target-pkey

The name of the unique index to use as the target portion of the user-specified primary key. The default is the value of the source-pkey.

delta-processing

Indicates whether or not delta processing is enabled for this compare pair. The options are:

"true" - delta processing is enabled.

"false" - delta processing is not enabled. This is the default.

profile-name

The name of the profile to use when running the compare-pair comparison.

system-key

If the compare pair has no column elements and no specified source-pkey, Oracle GoldenGate Veridata will select the most appropriate primary key or unique index to use. The options are:

"true" - Oracle GoldenGate Veridata selects the key if it is not defined. This is the default.

"false" - Oracle GoldenGate Veridata does not select the key.

system-columns

Indicates that the compare pair contains column elements with the type attribute set to key, so the generated compare pair will have user-defined columns for the key. The options are:

"true" - Compare pair has key column elements. This is the default.

"false" - Compare pair does not have key column elements.

wildcard

Specifies the pattern matching method that is used. The options are:

"ggs" - Use the typical Oracle GoldenGate pattern that matches an asterisk (*) to any number of characters.

"regex" - Use regular expressions for matching.

"default" - Use the setting for the configuration. This is the default.


Regular Expression Grouping

Regular expression grouping can be used to capture the parts of the source table names to be used for matching the target table name. You can do this by changing the wildcard attribute should be changed to regex. Groups to be matched are referenced as $1, $2, $3 and so on. Group $0 matches the entire source table name.

Examples of matching groups include:

  • P(.*) - Matches table names that begin with P. It captures the variable portion in $1. This matches table PROSPECTS.

  • [^PV].* - Matches table names that do not begin with P or V. This does not match the table PROSPECTS, but does match the table REGIONS.

  • ([P-R])(.*) - Matches table names starting with P, Q, or R and captures the initial letter in group $1 and the rest of the name in group $2. Groups are defined by parenthesis pairs. Group numbers are defined by the count of left parenthesis. Group $1 starts at the first left parenthesis and group $2 starts at the second parenthesis.

Captured groups ($n) are then used in expressions for selecting the target tables.

Example

The following example describes the key-only compare-pair. It's source tables are defined in the "test" schema and target tables in the "other" schema. It creates a compare pair in which the source table name begins with S and target table name begins with T. For example, S_TABLE and T_TABLE, where S_TABLE is a table in schema "test" and T_TABLE is table in schema "other". It also excludes all non-key columns in the generated compare pairs.

<configuration> 
    <connection name="source" host="somehost"
               .... use-ssl="true">
        <description>
             <![CDATA[
                Group SQL Scripting Source Connection
              ]]>
          </description>
     </connection>
...
...
</configuration>

connection

The connection element defines a connection to a source or target comparison database through an Oracle GoldenGate Veridata agent.

The following elements can be nested within the connection element:

Table 5-9 connection Elements

Element Description
description

Provides a description of the connection.

conn-properties

Defines the connection properties for a connection.


The following attributes describe the connection element:

Table 5-10 connection Attributes

Attribute Description
name

A name that identifies the connection. This is a required attribute.

host

The name of the system on which the Oracle GoldenGate Veridata agent is running.

port

The port number of the system on which the agent is running.

user

The user name the agent uses to connect to the database.

password

The password the agent uses to connect to the database.

repairUser

The database user with privileges to perform repair operations. See Installing and Configuring Oracle GoldenGate Veridata.

repairPassword

The password for the repairUser.

agent-timeout

The amount of time Oracle GoldenGate Veridata will wait before timing out when sending requests to the agent.

truncate-spaces

Either "true" or "false" to indicate whether or not spaces will be removed from the end of character columns. The default is "true" to truncate spaces.

fetch-size

(Oracle only) The number of rows fetched in each batch.

use-ssl

Defines using SSL communication between the Veridata Agent and the Server. The default is "true".


Example

The following example identifies the connection named source.

<configuration>
   <connection name="source" host="somehost"
    port="7850" user="somename" password="somepw"repairUser="veridata1" repairPassword="veridata1" agent-timeout="4000" truncate-spaces="false" fetch-size="3" use-ssl="true">
      <description>
             <![CDATA[
                Group SQL Scripting Source Connection
            ]]>
...
...
      </description>
   </connection>
.
.
</configuration>

conn-properties

The conn-properties element provides additional connection to a source or target comparison database elements.

The following attributes can be nested within the conn-properties element:

Table 5-11 conn-properties

Element Description

datatype-name

Specifies the data type for which properties have changed.

format

Specifies the Veridata comparison format to be used for comparison.

precision

Specifies the precision to be applied to the comparison.

scale

Specifies the scale to be applied to the comparison.

timezone

Timezone name is same as in the Veridata GUI.


delta-config

The delta-config element defines the delta processing configuration for the specified compare pair. It can be used once per compare pair. This element can appear once or not at all depending on the type of configuration you want. When the source or target configuration specified, the corresponding column-name attribute and query element are mandatory.

The following elements describe the delta-config:

Table 5-12 delta-config Elements

Attribute Description

source-config

Provides source side configuration for delta processing.

target-config

Provides target side configuration for delta processing.

query

Specifies the query for delta processing.


Example

This example creates a compare pair with delta processing enabled. Delta processing is enabled on COL1 of SYSMAPPING1 table for both source and target side. The SQL query is defined within the "query" tag.

<configuration validation="required">
   .
   .
   <group name="testGroup" source-conn="sourceConn" target-conn="targetConn" source-schema="sourceSchema" target-schema="targetSchema">
        <compare-pair source-table="SYSMAPPING1" target-table="SYSMAPPING1" name="sameTables" delta-processing="true" >
                  <delta-config>
                        <source-config column-name="COL1">
                              <query><![CDATA[ SELECT MAX(COL1) from SYSMAPPING1 ]]></query>
                        </source-config>
                        <target-config column-name="COL1">
                              <query><![CDATA[ SELECT MAX(COL1) from SYSMAPPING1 ]]></query>
                        </target-config>
                  </delta-config>
            </compare-pair> 
    </group>
      .
      .
</configuration>

description

The description element is free-form text that can be used to attach a description to the containing element. It has no associated attributes.

Example

The following example provides a description for the connection named source.

<configuration>
   <connection name="source" host="somehost"
    port="7850" user="somename" password="somepw"
      <description>
         <![CDATA[
           This connection is used when the Veridata agent connects
           to the source.
         ]]>
      </description>
   </connection>
.
.
.
</configuration>

enscribe-info

The enscribe-info element provides additional information used to compare NonStop Enscribe records at the field level.

The following elements can be nested within the enscribe-info element:

Table 5-13 enscribe-info Elements

Element Description
expandddl

Describes the rules that are used when applying the DDL.


The following attributes describe the enscribe-info element:

Table 5-14 enscribe-info Attributes

Attribute Description

side

Indicates whether the information applies to the source or the target table. The options are:

"source" to specify the source table. This is the default.

"target" to specify the target table.

dictionary

The volume and subvolume containing the data dictionary.

record

The name of the record in the data dictionary.


enscribe-key

The enscribe-key element defines the key that is to be used for Enscribe files. The enscribe-key element defines a delta processing that can used in a where clause on the initial comparison query.

The following attributes describe the enscribe-key:

Table 5-15 enscribe-key Attributes

Attribute Description
name

A name that identifies the key. This is a required attribute.

start-key

The key that is to be used to begin reading the Enscribe file. This is a required entry.

end-key

The key of the last Enscribe record that should be read. This is a required entry.

format

Specifies the format of the Enscribe key. The options are:

"ascii" - The format of the key is ASCII. This is the default.

"hexadecimal" - The format of the key is hexadecimal.

side

Indicates whether the partition should be applied at the source database, the target database, or both databases.

default

Indicates whether this is the default partition. This is equivalent to the "use at run time" indicator on the UI. The default is both.


Examples

<enscribe-key name = "Part1" end-key ="1000" format ="hexadecimal" default ="false" side="source"/>
<enscribe-key name = "Part1" start-key ="001" format ="hexadecimal" default ="false" side="target"/>
<enscribe-key name = "Both" start-key ="001" end-key ="1000" default ="true"/> 

excluded-column

The excluded-column element defines a set of columns to be excluded from a compare pair when the compare pair uses system mapped columns.

The following attribute describes the excluded-column element:

Table 5-16 excluded-column Attributes

Attribute Description
name

A regular expression that defines a set of source column names. This is a required attribute.


expandddl

The expandddl element describes the rules used when applying the DDL.

The following attributes describe the expandddl element:

Table 5-17 expandddl Attributes

Attribute Description
expandGroupArrays

Whether or not to expand group arrays. The options are:

"true" to expand the array. This is the default.

"false" not to expand the array.

redefined-columns

Whether or not to include redefined columns. The options are:

"include" - Includes redefined columns

"omit" - Leaves out redefined columns. This is the default.

resolvedups

Specifies how to resolve duplicates that result when the array is expanded. The options are:

"appendIndex" - Adds a unique numeric index to the end of the duplicate. This is the default.

"appendAlphaIndex" - Adds an alpha character index to the end of the duplicate.

"prependGroup" - Prefixes the name of the array group to the duplicate.

ddl-separator

The character separator for defining array output into columns. An example is the dash used in FIELDX-3, which is the third occurrence of FIELDX in the array. The options are:

"none" - There is no separator. This is the default.

"dash" - Use a dash (-) as the separator.

"bracket" - Use brackets [] as the separator.

"underscore" - Use underscore (_) as the separator.

"double-underscore" - Use double underscore (__) as the separator.

zero-fill-length

Prepends zeros to adjust the number of the occurrence. The value is the number of digits enclosed in quotation marks. "0" is the default.

fix-long-names

Whether to fix the names that result from resolving duplicates if they exceed the max-col-name-length. The options are:

"true" - Fix the names that exceed the maximum. This is the default.

"false" - Do not change the names that exceed the maximum.

max-col-name-length

The maximum length allowed for a column name. The entry is a number within quotation marks. The default is "120".


filter

The filter element defines a set of schemas and tables to either be included or excluded.

When using include filters, at least one filter must be matched before a table can be included in a compare pair. When a table matches a include filter, the include filter's colfilter is used to specify the columns for the generated compare pair.

When using exclude filters, a table is excluded if it matches any exclude filter. Include filters can include a colfilter element, which contains a list of columns to include or exclude.

Instead of schema and table filters, NonStop platforms use file pattern filters. The file pattern is any valid NonStop platform file name pattern.

The schema and table name can use wildcards.

The following attribute describes the filter element:

Table 5-18 filter Attributes

Attribute Description
type

Specifies either to include or exclude schemas and tables. Valid values are include or exclude.

catalog

Specifies the default catalog name.

schema

Specifies the schema name.

table

Specifies the table name.

file-pattern

For NonStop platforms only, specifies the file patter filter.


Example

When the source and target schemas have CHAR_TYPES3, INT_TYPE1, and INT_TYPE2 tables, then the following filters only create compare pairs for tables CHAR_TYPES1 and CHAR_TYPES3. The CHAR_TYPES2 table is excluded because of exclude filter and INT_TYPE1 and INT_TYPE2 are excluded because they were not part of include filter.

<group 
      ..
        <filter type="include" table="CHAR_TYPES*" />
        <filter type="exclude" table="CHAR_TYPES2" />
        <compare-pair source-table="*" target-table="*">
        </compare-pair>
      ..
</group>

group

The group element defines a set of compare pairs that all have the same source and target database connections. These compare pairs also have other properties in common.

The following elements can be nested within the group element.:

Table 5-19 Group Elements

Element Description
description

Provides a description of the group.

filter

One or more filter specifications, which allows table name filtering at the group level.

sql-partition

One or more specifications of a subset of rows within the table.

enscribe-key

One or more specifications of a subset of records within an Enscribe file.

compare-pair

Defines one or more compare pairs. The compare-pair elements are added to the group in the order they are specified. If the same compare pair fits the criteria of another specification in the group, the first compare pair will be used.


The following attributes describe the group element:

Table 5-20 Group Attributes

Attribute Description
name

A name that identifies the group. This value is required.

source-conn

The name of the connection to the source database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is required if it references an existing connection in the repository.

target-conn

The name of the connection to the target database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is required if it references an existing connection in the repository.

source-schema

The name of the default schema for the source tables referenced in the compare pairs that make up the group.

target-schema

The name of the default schema for the target tables referenced in the compare pairs that make up the group.

source-catalog

The default catalog for the source tables referenced in this group.

target-catalog

The default catalog for the target tables referenced in this group.

validation

Specifies the type of validation that will be used for the configurations. The options are:

"required" - All compare pairs must be successfully validated before any pairs are added to the repository.

"omit-failures" - Successfully validated compare pairs are added to the repository and compare pairs that cannot be validated are ignored.

"none" - Compare pairs are added to the repository without any validation. If this option is selected the Oracle GoldenGate Veridata Web User Interface should be used to review and fix validation problems.

"default" - Use the type of validation specified for a higher level, such as the configuration element. This is the default.

source-file-pattern

The default file pattern for the source if the data source is Enscribe or SQL/MP.

target-file-pattern

The default file pattern for the target if the data target is Enscribe or SQL/MP.


Example

<group name="weekly-tables" source-conn="source" target-conn"="target">
   <description>
   .
    .
    .
   </description>
   <sql-partition>
    .
    .
   </sql-partition>
   <compare-pair>
    .
    .
    .
</compare-pair>
</group>

job

The job element defines an Oracle GoldenGate Veridata comparison job.

The following elements can be nested within the job element:

Table 5-21 job Elements

Element Description
description

Provides a description of the job.

group

The name of the group associated with the job. This can be a new group or a previously defined group.


The following attributes describe the job element:

Table 5-22 job Attributes

Attribute Description
name

A name that identifies the job. This is a required attribute.

source-conn

The name of the connection to the source database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is required if it references an existing connection in the repository.

The job source-conn is used to override the source connection specified for the groups included in the job.

target-conn

The name of the connection to the target database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is used to override the target connection for the groups included in the job.

profile

The default profile to use when running the job.


Example

<job name="all-groups" profile="server-sort">
    <group name="all-tables"/>
    <group name="selected-tables"/>
</job> 

profile

The profile element defines the connection properties of a comparison job connection.

The following elements can be nested within the profile element:

Table 5-23 profile Elements

Element Description
description

Provides a description of the profile.

profile-general

Defines the profile parameters that control the output options.

sorting-method

Defines the profile parameters that control the sorting method and memory management. The data is sorted to match keys (or a key specification) so that the correct source and target rows are compared.

initial-compare

Defines the profile parameters that control the parameters for the job that performs the initial compare step

confirm-out-of-sync

Specifies the profile parameters that control the parameters for the job that performs the confirmation step

repair

Specifies the profile parameters that control the parameters for the repair job.


The following attributes describe the profile element:

Table 5-24 profile Attributes

Attribute Description
name

A name that identifies the profile. This is a required attribute.


Example

This example creates profile named "userDefinedProfile". The parameter names like "oos-format", "sort-method" are described in the table (link for table is in another pin)

<configuration validation="required">
   .
   .
<profile name="userDefinedProfile">
    <profile-general>
        <param name="oos-format" value="xml" />
        <param name="oos-xml-chunk-size" value="1000" />
    </profile-general>
    <sorting-method>
        <param name="sort-method" value="server" />
    </sorting-method>
</profile>
      .
      .
</configuration>

key-column

The key-column element defines a set of columns to be used as the user defined key for the comparison job.

The following attributes describe the key-column element:

Table 5-25 profile Attributes

Attribute Description
source-name

A regular expression that defines a set of source column names. This value is required.

target-name

A regular expression that defines a set of target column names. It can include references to groups captured by the source-name expression.

format

Specifies a format to override the comparison format that would normally be used.

scale

Specifies a scale to override the default scale for the comparison.

precision

Specifies a precision to override the default precision used for the comparison.

timezone

Specifies a time zone to override the default time zone of the comparison.


profile-general

The profile-general element provides parameters to control the output options.

The data is sorted to match keys (or a key specification) so that the correct source and target rows are compared.

The following elements can be nested within the profile-general element:

Table 5-26 profile-general Element

Element Description
param

Defines the parameter to change for the profile.


sorting-method

The sorting-method element provides parameters for sorting method and memory management. The data is sorted to match keys (or a key specification) so that the correct source and target rows are compared.

The following elements can be nested within the sorting-method element:

Table 5-27 sorting-method Element

Element Description
param

Defines the parameter to change for the profile.


initial-compare

The initial-compare element provides parameters for the process that performs the initial compare step.

The following elements can be nested within the initial-compare element:

Table 5-28 initial-compare Element

Element Description
param

Defines the parameter to change for the profile.


confirm-out-of-sync

The confirm-out-of-sync element provides parameters for the process that performs the confirmation step.

The following elements can be nested within the confirm-out-of-sync element:

Table 5-29 confirm-out-of-sync Element

Element Description
param

Defines the parameter to change for the profile.


param

The param element defines the parameters that are used for configuring profile options.

The following attributes describe the repair element:

Table 5-30 param Attributes

Attribute Description
name

The name of the parameter. This is a required attribute.

value

The value of the parameter


repair

The repair element provides parameters for the repair process.

The following elements can be nested within the repair element:

Table 5-31 repair Element

Element Description
param

Defines the parameters that are used to configure the profile options.


sql-partition

The sql-partition element defines a boolean SQL expression that can be used in a where clause in the initial comparison query.

The following attributes describe the sql-partition element:

Table 5-32 sql-partition Attributes

Attribute Description
name

A name that identifies the partition. This is a required attribute.

side

Indicates whether the partition should be applied at the source database, the target database, or both databases. The default is "both".

default

Indicates whether this is the default partition. This is equivalent to the "use at run time" indicator on the UI. The default is "false".


Example

<sql-partition name="replicate" default="true" side="source">
  <![CDATA[ replicated='false']]>
</sql-partition>
<sql-partition name="replicate" default="true" side="source">
  <![CDATA[ replicated='true']]>
</sql-partition>