Configuring Workflow Objects

2 Configuring Workflow Objects

Oracle GoldenGate Veridata supports connections, groups, compare pairs, profiles, and jobs. You can create, modify, edit, delete, and use all the features by using the User Interface.

2.1 Overview

To begin using Oracle GoldenGate Veridata, you need to create some objects that identify the data that you want to compare and which help you to manage your work. Create these objects in the following order:

Configure datasource connections: Oracle GoldenGate Veridata Server must be able to connect to an Oracle GoldenGate Veridata Agent for each database that contains source and target data that you want to compare. A connection is defined by a host, the port number of an Oracle GoldenGate Veridata Agent (or Manager, if a C-agent), and the datasource that is accessed by the agent. Connections must be created before any other objects are created.
Configure groups: You must configure at least one compare group that is linked to a set of source and target datasource connections. A group is a logical container for organizing the objects that you want to compare.
Configure compare pairs: You must configure one or more compare pairs for each group that you create. A compare pair is a set of corresponding source and target tables or files. Compare pairs can be created right away after you create a group, or you can edit the group later to add them. Configuring compare pairs will be the most time-consuming task, but you can spread the work across multiple sessions and save it as you go along. You can create a link to your work-in-progress in the Favorites Manager, so that you can return to it quickly.
Configure profiles: A profile contains settings for runtime parameters and can be applied globally to a job or to a specific compare pair as an override to the job profile. Profile parameters control such attributes as the sorting method to be used, thread and memory usage, report output, and so forth. Defining run profiles is optional, because Oracle GoldenGate Veridata includes a default profile that contains settings that apply to most usage scenarios. However, as you gain experience with Oracle GoldenGate Veridata, you may want to customize the default profile or create your own custom profiles.
Configure jobs: A job is a logical container for one or more compare groups and is the unit of work by which comparison processing is executed. Within one or more jobs, you can manage and run large volumes of compare groups across numerous databases and systems, and you can control the timing of those comparisons.

Parent topic: Configuring Workflow Objects

2.2 Configuring Connections

To get started with Oracle GoldenGate Veridata, you must define a connection to the source and target databases that contain the data that you want to compare. Oracle GoldenGate Veridata Server uses the connection information to communicate with Oracle GoldenGate Veridata Agent.

A connection is defined by:

A host where Oracle GoldenGate Veridata Agent is running
The port number for Oracle GoldenGate Veridata Agent on that host
The datasource that is associated with this agent

Connections are managed from the Connection Configuration page. To access this page, click Connection Configuration under Configuration in the navigation pane.

All connections that exist within the Oracle GoldenGate Veridata repository are shown in the Existing Connections list on this page. You must have the Administrator or Power User role to do the following from this page:

Parent topic: Configuring Workflow Objects

2.2.1 Creating a Connection

Click New. The New Connection Assistant prompts for the following:
- A name for the connection, and an optional description.
- Host Name or IP Address: The DNS (Domain Name Server) host name or IP address of the host where the Oracle GoldenGate Veridata Agent is installed. See the Oracle GoldenGate Veridata Agent System Requirements for installation instructions, if needed.
- Port: The port number that is assigned to the agent (or the Manager process, if a C-agent). To find out the port number of a Java agent, view the server.port parameter in the agent.properties file within the agent's installation directory. To find out the port number for a C-agent Manager, run the GGSCI program from the agent's installation directory, and then use the INFO MANAGER command.
- DataSource Type: Select the type of datasource from the Datasource Type drop-down list. The list contains the following datasource types:
  - Oracle
  - NSK
  - SQL Server
  - Sybase
  - DB2
  - Informix
  - Hive
  - Teradata
  - MySQL
  - PostgreSQL
- Use SSL for communication: Select the Use SSL for communication checkbox for secure communication between the Veridata agent and the server.
- A user name and password for connecting to the datasource (if required by the database).
- (Optional) A separate user can be configured for executing repair operations at the target database. This user needs permission to update as well as to query the tables.
Click Test Connection to confirm the supplied information is correct. If you have selected the Use SSL for communication checkbox in the previous screen, SSL will be used for verifying the datasource connection.

Parent topic: Configuring Connections

2.2.2 Edit a connection

Click the name of the connection in the Existing Connections list, or select it in the Select column and then click Edit.
The Edit Connection page is displayed.

Parent topic: Configuring Connections

2.2.3 Delete a connection

Before deleting a connection, you must un-link it from any groups and jobs to which it is linked, or delete the group or job if appropriate.
To delete a connection, select it in the Select column of the Existing Connections list, and then click the Delete button. Only one connection can be deleted at a time.

Parent topic: Configuring Connections

2.2.4 Filter the Existing Connections list

Expand Filters to reveal filter options. Strings are matched character for character.

Name Like: Use a string to filter by connection name.
Description Like: Use a string to filter by matching strings in the description.
Datasource Type: Select from the drop-down list to filter by the datasource type.

Click Apply Filter to display the selected connections.

Parent topic: Configuring Connections

2.2.5 Editing Connection Settings

Use the Edit Connection page to edit connection information. To access this page:

In the navigation pane, click Connection Configuration.
In the Existing Connections list, click the name of the connection that you want to edit. The Edit Connection page is displayed.

You must have the Administrator or Power User role to edit a connection.

Connection Settings tab

This tab modifies the behavior of the agent.

A check mark under Use Default indicates that a parameter is set to the default value.

The current setting is shown under Value.

If the parameter is a toggle, a check mark under Value indicates that it is enabled.

To change a setting

Clear the Use Default check box.
Make your change under Value.
Click Save.

Parameter Details

Agent Message Timeout: Specifies a time interval, in seconds, after which the Oracle GoldenGate Veridata Server abends if it has not received a message from the Oracle GoldenGate Veridata Agent.
Truncate Trailing Spaces When Comparing Values: Space (U+0020) and Ideographic Space (U+3000) are the truncate targets. This parameter works only for String or Binary columns. You can configure the truncated length with the truncate_spaces_len entry in the veridata.cfg file of the Oracle GoldenGate Veridata Server. This parameter also truncates the column padding character if the padding character is one of the targe spaces. This parameter does not trim trailing spaces on LOB data.
Initial Compare Fetch Batch Size: Sets the number of rows that are fetched at once for the initial comparison. Increasing the batch size may increase throughput, as compared to standard database access.
- Exclusively, for the Oracle and PostgreSQL database, the default value is 1000 and it fetches a batch size of 1000 rows; the minimum value that you can enter is 100. Any lesser value that you enter automatically changes to 100. The maximum value is 100000.
- For databases other than Oracle and PostgreSQL (SQL Server, Sybase, DB2), the default value is 0, minimum is 0, and maximum value is 1000.
Use Source or Target Columns as Key Columns When Generating Compare Pairs: If the source table has primary keys or unique index defined, but the target doesn’t, then Oracle GoldenGate Veridata uses the same columns (as the source) as unique identifiers for the target. Similarly, if the source doesn’t have the primary key or index, but target has them, then key columns of target are used for source. If either (source or target) of the columns of the Primary Key or index is not present on the target side, then the primary key or index isn’t considered.

Note:
The behavior of this option in the source column differs from its behavior in the target column.
Use All Columns as Key Columns When Generating Compare Pairs:
Enables Automatic mapping for the source and target connections. If this check box is disabled either at the source or the target connection, then the automatic mapping is also disabled for that group. If you enable this option to map all columns from source and target, then the mapping is considered only when both source and target table don't have primary or unique keys.

Connection Pair Parameters

Parent topic: Configuring Connections

2.2.6 Editing Connection Details

Use the Edit Connection page to edit connection information. The settings on this page are applied globally to all comparisons that are run from the associated data source, unless an override is permitted elsewhere.

You must have the Administrator or Power User role to edit a connection.

Connection Details tab

This tab captures the information that is required for the Oracle GoldenGate Veridata Agent to connect to a database.

Parameter Details

Name: The name cannot be changed.
Description: A description is optional and can be edited as needed.
Host Name or IP address: Either the Domain Name Server (DNS) host name or IP address of the system. This information can be changed.
Port: The port number of the Oracle GoldenGate Veridata Agent on the system. The port number can be changed if the one listed is not the correct port that was specified when the agent was installed.
Datasource Type: The type of database that is accessed by this connection. This attribute cannot be changed.
(Optional) Click Verify to verify that the connection will be successful. The connection will also be verified by Oracle GoldenGate Veridata at runtime.
Catalog: (Does not apply to all databases) The database that contains the objects that are to be compared.
User: The name of an existing user that has connection access to the database. The user can be changed. For required permissions, see Securing Access to Oracle GoldenGate Veridata by Defining User Roles.
Password: The user's password.
(Optional) If you configured a separate user for executing repair operations at the target database then you can change this user's name and password.
(Optional) Click Test Connection to verify that the credentials are valid.

When you are finished making your changes, click Save to save them to the repository.

Parent topic: Configuring Connections

2.2.7 Editing connection properties

Use the Edit Connection page to edit connection information. To access this page:

In the navigation pane, click Connection Configuration.
In the Existing Connections list, click the name of the connection that you want to edit. The Edit Connection page is displayed.

You must have the Administrator or Power User role to edit a connection.

Connection Properties tab

This tab defines rules for how each data type in the underlying database is interpreted and mapped if compared to data from a different type of database. This tab sets global values for all instances of a data type. To override the format mapping of a specific column in any given table, go to the Column Mapping Configuration page for the compare pair and then use the User Defined column mapping method.

The supported data types are displayed with default mappings to Oracle GoldenGate comparison formats. In cases where the automatic mapping is not sufficient, you can select another supported format.

To change a format setting:

Clear the Use Default box.
Make a selection under Comparison Formats.
Depending on the data type and format that you specified for Comparison Formats, you might need to supply or select additional information in the Precision, Scale, and Timezone columns.
Click Save.

Note:
If there's a need to treat NULL values and spaces as same then, the comparison format string_en can be used if listed in the supported formats for the selected datatype. This works only when either the Source or the Target database is Oracle.

Other connection configuration tabs

Editing Connection Details

Editing Connection Settings

Parent topic: Configuring Connections

2.3 Configuring Groups

Groups are logical containers for one or more compare pairs. They help you to organize and partition large or diverse sets of data into more manageable units. Groups are linked to jobs when jobs are created. Any group can be linked to one or more jobs, allowing you complete control over how and when data is compared.

Note:

A group is associated with a set of connections to the source and target data. Before creating a group, you must create these connections.

Groups are managed from the Group Configuration page. To access this page, click Group Configuration in the navigation pane.

All groups that are defined within the Oracle GoldenGate Veridata repository are shown in the Existing Groups list on this page. You must have the Administrator or Power User role to create, edit, or delete a group. You can do the following tasks from this page:

Parent topic: Configuring Workflow Objects

2.3.1 Filter the Existing Groups list

Expand Filters and type a search string in one of the following boxes. Strings are matched character for character.

Name Like: filters by group name.
Description Like: filters based on a matching string in the description.
Source Connection Names Like: filters by the name of the source connection.
Target Connection Names Like: filters by the name of the target connection.

Click Apply Filter to display the selected groups.

Parent topic: Configuring Groups

2.3.2 Create a group

To create a group, click New. The New Group Assistant is displayed. The assistant will prompt for:

A name and description.
Connection information

Parent topic: Configuring Groups

2.3.3 Edit a group

To edit a group, click the name of the group in the Existing Groups list, or select it in the Select column and then click Edit. The Edit Group page is displayed.

Parent topic: Configuring Groups

2.3.4 Delete a group

Before deleting a group, you must remove it from any jobs to which it was linked (or delete the job if appropriate.) To delete a group, select it in the Select column of the Existing Groups list, and then click the Delete button. Only one group at a time can be deleted.

Parent topic: Configuring Groups

2.4 Configuring Compare Pairs

A compare pair is the logical relationship between a source table or file and a target table or file for the purpose of comparing their data. Compare pairs are linked to groups. As a reason, all of the source and target objects that you configure into compare pairs for any given group must be accessible from the datasource connections that are associated with that group.

Note:

Before creating compare pairs, you must create a group to contain them.

Use the Compare Pair Configuration page to view, create, modify, and save compare pairs and their column mappings. To access this page:

Click the name of a group within any page that displays groups. To access this page from the navigation pane, click Group Configuration, then the name of the group for which you want to configure compare pairs. This displays the Edit Group page.
Click Go to Compare Pair Configuration...
- The Group Information section at the top of this page displays information about the group that you are working with.
- The tabs on this page contain the tools that you need to work with compare pairs.

The Existing Compare Pairs tab is your start point. It displays any compare pairs that currently exist in the selected group. The columns in the Existing Compare Pairs list show details about those compare pairs.

Modifying Existing Compare Pairs

To control delta processing (For more information about delta processing, see Using Delta Processing).

Editing a Compare Pair

Deleting a Compare Pair

To create new compare pairs

You must have the Administrator or Power User role to create compare pairs.

You can use the following methods to create compare pairs.

Pattern Mapping

Use the Pattern Mapping tab to map numerous source and target objects at once by using:
- An exact name match (for example TAB1=TAB1).
- A SQL percent (%) wildcard or an asterisk (*) wildcard to map differently named source and targets whose naming conventions support wildcarding.
To create compare pairs using pattern mapping see Mapping Objects Using a Pattern.
Manual Mapping

Use the Manual Mapping tab to map source objects to target objects one at a time. This method is useful when source and target names differ vastly and cannot be mapped by using wildcards.

Note:

The manual mapping tab is the tab, where you configure row partitions for new or existing compare pairs.

To create compare pairs using manual mapping, see Mapping Objects Manually.
Automatic Key Column Mapping

In the absence of unique identifiers, such as the primary keys and unique indices, which can uniquely identify each row, you need to manually define any column or columns to be used as a unique identifier. These mappings are automated by options in the Manual Mapping tab and the Pattern Mapping tab. You can enable automatic key column mapping either at the Connection Level or at the Compare Pair Level. If you have enabled it at the Connection Level, then any group using this Connection has the feature enabled. You need to enable the Use All Columns as Key Columns When Generating Compare Pairs checkbox at both the source as well as the target levels. If the checkbox is disabled at any one level, then the automatic key column mapping is disabled for the corresponding group. See Editing Connection Settings.

The following parameters in Oracle GoldenGate Veridata support Automatic Key Column mapping:

Column Mapping When No Keys are Specified

Use Source Key Columns for Target

Value of this key can be true/false. When this key set to true and use-all-columns key set to false, Oracle GoldenGate Veridata maps the Source Table Primary Keys to Target Table when the Target table doesn't have any Primary Key.

Use Target Key Columns for Source

Value of this key can be true/false. When this key set to true and use-all-columns key set to false, Oracle GoldenGate Veridata maps the Target Table Primary Keys to Source Table when the Source table does not have any Primary Key.

Use All Columns as Key Columns

Value of this key can be true/false. When set to true, all columns except LOBs are mapped as Primary keys when either Source or Target table OR both do not have any Primary Keys defined.

Connection Pair
Use Source Key Columns for Target
Value of this key can be true/false. When this key set to true and use-all-columns key set to false, the following actions occur:
- Oracle GoldenGate Veridata maps the Source Table Primary Keys to Target Table when the Target table does not have any Primary keys.
- Oracle GoldenGate Veridata maps the Target Table Primary Keys to Source Table when the Source table does not have any Primary Key.
  These configurations can be overridden at compare-pair level.
Use All Columns as Key Columns

Value of this key can be true/false. When set to true, all columns are mapped as Primary keys when either Source or Target table OR both don't have any Primary Keys defined.
These configurations can be overridden at Compare-pair level.
Combination Mapping

You can use a combination of both mapping methods, if needed. Use the pattern method first, and then use the manual method for any objects that remain unmapped, or to make revisions such as to add row partitions.

Existing Compare Pairs List Column Details
To filter the list
To control delta processing
Editing a Compare Pair
Configuring Partitions in Oracle GoldenGate Veridata
Oracle GoldenGate Veridata supports selecting a subset of rows for comparison by Manual row Partioning (SQL predicate statement or Enscribe partition range) or Oracle GoldenGate Veridata Automatic Row Partitioning. You can select either Manual Row Partition or Automatic Row Partition but not both.
Deleting a Compare Pair
Mapping Objects Using a Pattern
Mapping Objects Manually
Reviewing and Saving Compare Pairs
Viewing details for a compare pair that is finished

Parent topic: Configuring Workflow Objects

2.4.1 Existing Compare Pairs List Column Details

Compare Pair Name: This is either the default name format of <source>=<target> or a user-defined name. In either case, if you hover the mouse cursor over a compare pair name, the actual source and target object names are displayed.
Column Mapping: Links to the Column Mapping Configuration page, where you can create or edit the mappings between the source and target columns of a compare pair.
Source Catalog (Some platforms): The source metadata catalog or database.
Source Schema: The owner of the source database objects that are to be compared.
Target Catalog (Some platforms): The target metadata catalog or database.
Target Schema: The owner of the target database objects that are to be compared.
Row Partition: If any row partitions (subsets) are defined, there is a notification here. Otherwise, this field is blank.
Profile: If a run profile exists for a compare pair, it is shown here. Otherwise, this field is blank and the default profile will be used during comparisons. (A profile can be specified for an individual compare pair on the Manual Mapping tab.)
Validation Status: Shows whether or not the columns of the source and target objects are suitable for being compared, based on the results of any previous validation that was performed.
Key Mapping Method: Can be either System Generated or User Defined, depending on the method that was chosen to map the key columns for this compare pair.
Column Mapping When No Keys are Specified: Can be either System Generated or User Defined, depending on the method that was chosen to map the non-key columns for this compare pair.

Parent topic: Configuring Compare Pairs

2.4.2 To filter the list

Expand Filters to expose filter options.
- Compare Pairs with Status filters on one of the following:
  - Validated means that the source and target columns are compatible and suitable for comparison.
  - Preliminary Validation Failed means that a preliminary validation (done from a configuration page and not by a runtime process) failed.
  - Runtime Validation Failed means that the compare pair failed the runtime validation when a job was started.
- The remaining options filter based on a string that matches a string in:
  - a compare pair name
  - a table or file name
  - a profile name
Click Apply Filter to display the selected compare pairs.

Parent topic: Configuring Compare Pairs

2.4.3 To control delta processing

On any platform, you can enable or disable delta processing for any or all compare pairs from the Existing Compare Pairs tab. Select their names in the Select column (or select Page to select all pairs on the current page), and then click Enable Delta Processing or Disable Delta Processing. When delta processing is enabled, a yellow triangle delta symbol appears next to the name of the affected compare pair whenever the pair is displayed. For more information about delta processing, see Using Delta Processing.

Parent topic: Configuring Compare Pairs

2.4.4 Editing a Compare Pair

Use the Compare Pair Configuration page to edit compare pairs. To access this page:

Click the name of a group on any page that displays groups. This displays the Edit Group page.
In the Existing Compare Pairs list, click the name of the compare pair that you want to edit. This activates the Manual Mapping tab of the Compare Pair Configuration page.

You also can go directly to this page by clicking the name of any compare pair on any page that displays them.

You must have the Administrator or Power User role to edit compare pairs.

To edit compare pair properties

You can change any of the following. After you make a change, click Save.

To change the name of the compare pair

Type the new name in the Compare Pair Name field. The name cannot contain spaces but can contain underscores and equal signs. It is case-sensitive. You cannot modify the Name for compare-pairs that have the Automatic Row Partition set or for Existing Database Table Partition set.
To control delta processing

From the drop-down list, select either Enabled or Disabled. To use delta processing, server-side sorting must be enabled.

To change the profile that is associated with this compare pair

In the Profile field, select the name of the profile. Alternatively, you can click the Browse button next to Profile to browse for a profile using the Select a Profile page, which contains filtering options.

To add or change a Manual Row Partition

Next to Row Partitions, click Configure.

To edit the column mapping:

Make the Existing Compare Pairs tab active.
In the Column Mapping column, click Edit next to the compare pair whose column mapping you want to change.
Make your changes on the Column Mapping Configuration page.

Note:

Manual Row Partition cannot be set or edited for Automatic partitioned compare pairs. The Configure button is not visible for a compare-pair having Automatic Row Partition set as follows:

Figure 2-1 Manual Mapping Tab - Configure button not Visible

The Configure button is not visible when you enter Automatic in the Row Partitions field under Compare Pair Details.

Parent topic: Configuring Compare Pairs

2.4.5 Configuring Partitions in Oracle GoldenGate Veridata

Oracle GoldenGate Veridata supports selecting a subset of rows for comparison by Manual row Partioning (SQL predicate statement or Enscribe partition range) or Oracle GoldenGate Veridata Automatic Row Partitioning. You can select either Manual Row Partition or Automatic Row Partition but not both.

Using partitions allows you to compare source and target tables or files that have the same structure but a different number of rows. For example, you could compare a production table to a data warehouse table that may contain more rows because of historical data. The usage of partitions also speeds throughput by splitting the load into multiple processing streams. In Oracle GoldenGate Veridata user interface, the partitions have the following labels:

Manual: Specifies Manual Row Partition is set for the compare-pair
Automatic - Specifies Automatic Row Partition is set for the compare-pair
Not Specified - Specifies that No Partition is set for the compare-pair

This section contains the following topics:

Manual Row Partitions
In case of Manual Row Partitioning, you can use SQL Predicates or Enscribe Partition range to compare subset of data from a source and target table. Manual Row Partitioning is applicable for all databases. You can also configure any number of partions to generate compare pairs for the table.
Automatic Row Partitioning
If you want to split the large table comparison into multiple partitions, then you can use Automatic Row Partitioning. The partitioning is possible only when both the source database and target database are Oracle.
Mapping Database Table Partitions for Manual and Automatic Row Partitioning
If you have the database tables partitions defined then, those partitions can be mapped and can be split further using Manual Row Partitions or Automatic Row Partitions. Compare pairs are generated per Table Partitions based on the selected mappings.

Parent topic: Configuring Compare Pairs

2.4.5.1 Manual Row Partitions

In case of Manual Row Partitioning, you can use SQL Predicates or Enscribe Partition range to compare subset of data from a source and target table. Manual Row Partitioning is applicable for all databases. You can also configure any number of partions to generate compare pairs for the table.

For example, suppose a target SQL table FIN2 has an ID column that contains values 1 through 2000 (so 2,000 rows), but the source table FIN only has 1,000 rows with ID values 1 through 1000, a possible configuration for the target table row partition can be:

Compare Pair Name	Source Table/Target Table	Partition criteria
FinPart1	Fin/Fin2	ID < 500
FinPart2	Fin/Fin2	ID between 500 and 1000

To configure partitions:

For any given job run, one compare pair configuration can support:
- One Manual Row Partition for the source table or file and one Manual Row Partition for the target table or file.
  
  or
- One Manual Row Partition for either of the source objects or the target objects.
You can add any number of Manual Row Partitions to a compare pair, but only one can be active at runtime.

To compare multiple Manual Row Partitions concurrently

To compare multiple Manual Row Partitions for the same source and target objects during the same job run, create a new compare pair for each partition and include all of those compare pairs in the run.

After partitions are defined by means of unique compare pairs, multiple independent comparisons are possible in parallel or over the course of time, for example one Manual Row Partition per night.

Creating Manual Row Partitions

Parent topic: Configuring Partitions in Oracle GoldenGate Veridata

2.4.5.1.1 Creating Manual Row Partitions

Partitions are created in the Compare Pair Row Partitions editor. The appearance of this editor changes slightly depending on whether you are configuring a table-based or file-based compare pair.

To open the Compare Pair Row Partitions editor:

Navigate to the Manual Mapping tab of the Compare Pair Configuration page.
Go to the Row Partitions field.
Click the Configure button to display the Compare Pair Row Partitions editor.
Select one of the following tasks to work with the partitions.
Click OK to close the Manual Row Partitions editor. The Manual Row Partitions box of the Compare Pair Configuration page now contains the word Manual.

To create a new Manual Row Partition:

Under Source or Target, click New. The editor expands to display the Row Partition Configuration area.

Note:
You can cancel your work and start over at any time by clicking Cancel that is within this area of the editor. To exit the editor completely, click Cancel that is at the bottom-most portion of the editor page.
In the Name box, type a name for this partition. Use one word that can include underscores, hyphens, and other standard keyboard special characters.
The SQL Predicate Statement is a condition expression, where the WHERE clause is used to evaluate to a boolean value, either true or false.

The Use At Runtime checkbox is to make the row partition take effect for execution of compare and repair. If you don't select this checkbox, then though the row partition gets saved, it doesn't get activated.
Click Apply. The name of the new partition appears in the Row Partitions list above the work area, and the partition criteria is displayed.
Repeat these steps for any additional partitions that you want to create.
Click OK to close the Compare Pair Row Partitions editor. The Row Partitions box of the Compare Pair Configuration page now contains the word Manual.

To create a partition from a copy:

You can create partitions for one source or target object, and then apply those same partition(s) to the corresponding object with one mouse click. To allow for minor differences in the copied partition, you can edit it as needed.

Create the initial row partition for the source or target object.
Click Copy All From Target or Copy All From Source, depending on where you created the initial partition. The copied partition inherits the Use At Runtime setting of the original partition.
Click OK to close the Compare Pair Row Partitions editor. The Manual Row Partitions box of the Compare Pair Configuration page now contains the word Manual.

To edit a Manual Row Partition:

In the Select column under Source or Target, select the partition that you want to change.
Click Edit.
Under Row Partition Configuration, change the name or partition information as needed.

Note:
You can cancel your work and start over at any time by clicking the Cancel button that is within this area of the editor. To exit the editor completely, click the Cancel button that is at the bottom-most portion of the editor page.
(Optional) Click Use At Runtime to make this partition the active one for the compare pair.
Click Apply.
Click OK to save the partition statement and return to the Compare Pair Configuration page.

To delete a Manual Row Partition:

In the Select column, select the partition that you want to delete.
Click Remove.
Remove other partitions as needed.
Click OK to return to the Compare Pair Configuration page.

Parent topic: Manual Row Partitions

2.4.5.2 Automatic Row Partitioning

If you want to split the large table comparison into multiple partitions, then you can use Automatic Row Partitioning. The partitioning is possible only when both the source database and target database are Oracle.

The automatic row partition creates configurable partitions and generates compare pairs for each generated partition.

To configure Automatic Row Partitioning:

In the Compare Pair Configuration page, click Manual Mapping.
For the selected table pair, select the Automatic Row Partitions check box.
Enter a numeric value between 2 to 100 in the text field before generating compare-pair. This value decides the number of compare-pairs to be created. Each auto generated compare-pair compares the subset of data from the selected source and target table.

Figure 2-2 Automatic Row Partitioning
After all partitions are mapped, details of compare-pairs generated are displayed in Preview Tab as follows:

Figure 2-3 Automatic Row Partitioning - Preview tab

Note:

The Automatic Row Partitions check box is available only in case of Manual Mapping. You cannot edit compare-pair names that are created using Automatic Row Partitioning.

Parent topic: Configuring Partitions in Oracle GoldenGate Veridata

2.4.5.3 Mapping Database Table Partitions for Manual and Automatic Row Partitioning

If you have the database tables partitions defined then, those partitions can be mapped and can be split further using Manual Row Partitions or Automatic Row Partitions. Compare pairs are generated per Table Partitions based on the selected mappings.

The database tables partitions are shown only when both source and target are Oracle and the tables have already partitions defined at database.

To map existing database Table partitioning:

From the Manual Mapping tab, select Source Schema and Target Schema under Datasource information, and select the schemas that have to be configured. On selection of schema and Table, if the tables have partitions defined at database then they will be displayed in UI under Source Table Partitions and Target Table Partitions as follows:

Figure 2-4 Manual Mapping Table Partitioning
On selection of Table Partitions of source and target Table, the Add Mapping and Remove Mapping buttons are enabled.
Click Add Mapping to map the selected database Table Partitions. You can also map multiple database Table Partitions before generating the compare-pair.
Click Remove Mapping if you want to remove the mapped database Table Partitions.
If you want to map more than one database Table Partition then, you are not allowed to set Manual Row Partition on the same page. However, you can generate the compare-pair and later add Manual Partition to each compare pair by editing the compare-pair.
You can select auto partitioning if one or more database Table partitions are selected and multiple compare pairs are generated for each selected partitions. For example, in the Figure 2-4, 2 database Table Partitions are mapped; and if you select Automatic Row Partitions value as 5, then the total compare pair that gets generated is 10 (2 database Table Partitions * 5 Automatic Row Partitions).

Note:

Compare pair names cannot be edited when Mapped with Database Table Partitions.

Parent topic: Configuring Partitions in Oracle GoldenGate Veridata

2.4.6 Deleting a Compare Pair

Use the Compare Pair Configuration page to delete compare pairs.

You must have the Administrator or Power User role to delete compare pairs.

To delete compare pairs

Make the Existing Compare Pairs tab active.
(Optional) Expand Filters to filter the Existing Compare Pairs list as needed, and then click Apply Filter. You can filter by:
- Compare Pairs with status
  - Validated means that the source and target columns are compatible and suitable for comparison.
  - Preliminary Validation Failed means that the compare pair has not been processed in a job yet, but a preliminary validation failed.
  - Runtime Validation Failed means that the compare pair failed the runtime validation when a job was started.
  More on validation...
- Compare Pair Like
  
  The default name format is <source_table/file>=<target_table/file> but a compare pair can have a user-defined name instead. The filter returns names containing the specified string.
- Source Table Name Like and Target Table Name Like
  
  To determine the source and target table or file names if a user-defined name is shown, hover the mouse cursor over the name of the compare pair in the list. A hint appears showing the actual names.
- Profile Name Like
  
  There will be a profile name shown if one was specified to override the default profiles for jobs that contain this compare pair.
In the Select column, select the compare pairs that you want to delete, or select the Page check box to delete all of the compare pairs on the page. Use the page management tools to make selections on other list pages as needed.
Click Delete. You will be prompted to confirm your request.

Parent topic: Configuring Compare Pairs

2.4.7 Mapping Objects Using a Pattern

If your source and target object names are identical, or the naming conventions support using wildcards, you can create compare pairs by using the pattern mapping method. Pattern mapping can save a great deal of time compared to manual mapping, because it generates multiple compare pairs at once. You can combine the pattern mapping and manual mapping methods as needed.

To use pattern mapping, use the Pattern Mapping tab of the Compare Pair Configuration page.

You must have the Administrator or Power User role to configure compare pairs.

To use pattern mapping, complete the six tasks in the following sections in order.

Confirm group information
Select the Datasource
Specify the database structures that contain the tables to be mapped. The selections vary by database type.
Select a Pattern Mapping Method
Select and Specify Compare Pair Details
You can enable or disable automatic mapping either at the Connection Level (Connection Settings tab) or at the Compare Pair level (Compare Pair Configuration page). If this mapping is enabled at the Connection Level, then any group using this Connection will have the feature enabled. If the feature is disabled at the Connection level, then it also be disabled at Compare Pair level. You can override automatic mapping at the Compare Pair Generation stage.
Generate the mappings
Save the compare pairs to the repository

Parent topic: Configuring Compare Pairs

2.4.7.1 Confirm group information

Under Group Information, verify that the group you have chosen to contain the compare pairs is displayed at the top of the page.

Parent topic: Mapping Objects Using a Pattern

2.4.7.2 Select the Datasource

Specify the database structures that contain the tables to be mapped. The selections vary by database type.

Windows, UNIX, and Linux-based databases

Under Datasource Information, select the Source and Target catalogs or PDB (if applicable to the database) and schemas that contain the tables that you want to map as compare pairs. You can select from a drop-down list or click the Browse button to open a selection dialog that has filtering options.

Note:

As a prerequisite for the Oracle GoldenGate Veridata Server to display all the pluggable databases in the Source/Target Catalogs drop-down list, the database user must have access to the following tables:

V$DATABASE
DBA_PDBS

To use the filtering options, do the following:

Supply a search string to filter the list, and then click Apply Filter. The filter returns a list of names.
In the Select column of the list, click the name of the appropriate schema or catalog/PDB for this compare pair.
Click the Select button.

Enscribe databases

In the File Pattern box under Datasource Information, type the Source and Target volume, sub-volume, and file patterns of the source and target files that you want to map as compare pairs. Use the format of volume.subvolume.file_pattern (for example, volume.subvolume.*).
Click the Refresh Source Files/Tables and Refresh Target Files/Tables buttons.

Parent topic: Mapping Objects Using a Pattern

2.4.7.3 Select a Pattern Mapping Method

Under Pattern Compare Pair Mapping, select one of the following methods for matching source table names to target table names:

Table Mapping Method
- Map Source and Target Tables Using Exact Names.
  
  This pattern matches names character-for-character, so each source and target name must be identical. This pattern is useful for comparing production and failover databases, for example.
- Map Source and Target Tables Using SQL % Wildcard Pattern...
  
  To use this method, supply a wildcard string in Where Source Names Like and Not Like, and Where Target Names Like and Not Like that include the percent symbol (%) as the wildcard. A % in the target matches the text that is matched by the % in the source. By default, the Like option is selected for both the source and target tables. You can select Not Like if needed during the compare-pair generation.
  
  Example:
  
  Assume source tables of:
```
SOURCE_TABLE_1
SOURCE_TABLE_2
MY_SOURCE_TABLE_1
MY_SOURCE_TABLE_2
DUMMY_TABLE
```
  Assume target tables of:
```
TARGET_TABLE_1
TARGET_TABLE_2
MY_TARGET_TABLE_1
MY_TARGET_TABLE_2
DUMMY_TABLE
MY_DUMMY_TABLE
```
  Some possible Like pattern matches are:
  
  Source pattern: SOURCE_%
  
  Target pattern: TARGET_%
  
  Matches: SOURCE_TABLE1=TARGET_TABLE1, SOURCE_TABLE2=TARGET_TABLE2
  
  Source pattern: MY_SOURCE_%
  
  Target pattern: MY_TARGET_%
  
  Matches: MY_SOURCE_TABLE1=MY_TARGET_TABLE1, MY_SOURCE_TABLE2=MYTARGET_TABLE2
  
  Source pattern: SOURCE_%
  
  Target pattern: MY_TARGET_%
  
  Matches: SOURCE_TABLE1=MY_TARGET_TABLE1, SOURCE_TABLE2=MYTARGET_TABLE2
  
  Source pattern: DUMMY_TABLE
  
  Target pattern: %_TABLE
  
  Matches: DUMMY_TABLE=DUMMY_TABLE
  
  Source pattern: DUMMY_TABLE
  
  Target pattern: TARGET_%_
  
  Matches: None
  
  To preview the objects that are selected with the wildcards, click the Preview link that is shown after Map Source and Target Tables Using SQL % Wildcard Pattern. This opens the Pattern Mapping Preview page.
  
  Some possible NOT LIKE pattern matches are:
  
  Source pattern: SOURCE_%
  
  Target pattern: TARGET_%
  
  Matches: DUMMY_TABLE=DUMMY_TABLE
- Map Source and Target Tables Using Oracle GoldenGate * Wildcard Pattern...
```
SOURCE_TABLE_1
SOURCE_TABLE_2
MY_SOURCE_TABLE_1
MY_SOURCE_TABLE_2
DUMMY_TABLE
```
  Assume target tables of:
```
TARGET_TABLE_1
TARGET_TABLE_2
MY_TARGET_TABLE_1
MY_TARGET_TABLE_2
DUMMY_TABLE
MY_DUMMY_TABLE
```
  Possible Oracle GoldenGate wildcard matches are:
  
  Source pattern: SOURCE_*
  
  Target pattern: TARGET_*
  
  Matches: None
  
  Source pattern: *
  
  Target pattern: MY_*
  
  Matches: DUMMY_TABLE=MY_DUMMY_TABLE
  
  Source pattern: DUMMY_TABLE
  
  Target pattern: *
  
  Matches: DUMMY_TABLE=DUMMY_TABLE
  
  Source pattern: *
  
  Target pattern: *
  
  Matches: DUMMY_TABLE=DUMMY_TABLE
  
  The wildcard resolution is case-sensitive. To preview the objects that are selected with the wildcards, click the Preview link that is shown after Map Source and Target Tables Using GoldenGate * Wildcard Pattern. This opens the Pattern Mapping Preview page.
Map Table Partitions Using Exact Names: Select the Map Table Partitions Using Exact Names check box if you want to map the table partitions defined at database. For example, if there are 10 partitions in Source Table matching 10 table partitions in Target Table with same table partition names, then 10 compare pairs are created. If no table partitions exists then, one compare-pair per table gets generated.

Note:
The Map Table Partitions Using Exact Names check box is available only when both Source and Target database are Oracle.

After all partitions are mapped, details of compare-pairs generated are displayed in Preview Tab. In the following screen shot, the tables SALES_SRC and SALES_TGT have 2 partitions defined at database SALES_Q1_2006 and SALES_Q2_2006, which is mapped to each compare-pair:

Figure 2-5 Mapping Table Partitions Using Exact Names

For more information about partioning in Manual Mapping, see Mapping Database Table Partitions for Manual and Automatic Row Partitioning and Automatic Row Partitioning.
Exclude Views: Select the Exclude Views check box if you need to exclude all the Views (when applicable to the database) while generating compare pairs. When you click the Preview tab, the Exclude Views option is displayed.
Column Mapping When No Keys are Specified: You can enable or disable automatic mapping either at the Connection Level (Connection Settings tab) or at the Compare Pair level (Compare Pair Configuration page). If this mapping is enabled at the Connection Level, then any group using this Connection will have the feature enabled. If the feature is disabled at the Connection level, then it also be disabled at Compare Pair level. You can override automatic mapping at the Compare Pair Generation stage. See Automatic Mapping in Configuring Compare Pairs.
- Use Source Key Columns for Target: Enables automatic mapping for the target table. This option is available on the Compare Pair Generation page. If you disable the option for the target side, and if there is no primary key or unique index on the target side and the source has primary key or unique index, then only the mapping of columns happen but, automatic mapping doesn’t happen on the target table.
- Use Target Key Columns for Source:
  Enables automatic mapping for the source table. This option is available on the Compare Pair Generation page. If you disable the option for Source side, and if there is no primary key or unique index on the source side and target has primary key or unique index, then only the mapping of columns happen but, automatic mapping doesn’t happen on the source table.
- Use All Columns as Key Columns When Generating Compare Pairs: Enables automatic mapping for all columns. Ensure that this option is selected both at the source as well as the target groups. If the checkbox is disabled either at the source or the target levels, then the automatic mapping is disabled for the corresponding groups.

Parent topic: Mapping Objects Using a Pattern

2.4.7.4 Select and Specify Compare Pair Details

You can enable or disable automatic mapping either at the Connection Level (Connection Settings tab) or at the Compare Pair level (Compare Pair Configuration page). If this mapping is enabled at the Connection Level, then any group using this Connection will have the feature enabled. If the feature is disabled at the Connection level, then it also be disabled at Compare Pair level. You can override automatic mapping at the Compare Pair Generation stage.

This topic contains the details of the configurations in the Manual Mapping tab.

Selecting a Compare Pair

In the Selected Compare Pair section, specify the Source Table Name and the Target Table Name in the respective fields.

In the Column Mapping When No Keys are Specified section, you can select the following:
- Use Source Key Columns for Target: Enables automatic mapping for the target table. If you enable this option and if the source table contains primary key, but the target table doesn't, then, auto mapping of key columns is done based on source table primary keys.
- Use Target Key Columns for Source:
  Enables automatic mapping for the source table. If you enable this option and if the target table has primary keys, but the source table doesn't, then, auto mapping of key columns is done based on the target table primary keys.
- Use All Columns as Key Columns: Enables automatic mapping for all groups. Ensure that this option is selected both at the source as well as the target group. If you have enabled this option, then all columns from source and target tables get mapped only when both source and target tables don't have primary keys

See Automatic Mapping in Configuring Compare Pairs.

Under Compare Pair Details, specify the following:

Compare Pair Naming Format

Specifies the format for naming compare pairs. The default is *=*. This format uses an equal sign to link source and target object names. Example default names: customers=customers2, \FIN.$DATA.FN.CUST1=\FIN.$DATA.FN.CUST1. Case sensitivity is observed. Oracle GoldenGate Veridata first looks for a case-sensitive name match. Failing that, it looks for a case-insensitive name match. Objects that fail on both counts are not mapped.

You can change the naming format if desired.

To specify a name:

A user-defined name cannot contain spaces but can contain underscores and equal signs. It is case-sensitive. If you will be generating a large number of compare pairs, use a naming convention that is intuitive. Oracle GoldenGate Veridata will display the actual object names if you hover the mouse over the name of a compare pair.

Tip:

To condense the width of columns that display the names of compare pairs, you can use one asterisk as the name format if the source and target objects have identical names.

Description of the illustration enscribe_pattern_tip.jpg

The result is that just one name is displayed under Compare Pair Name columns throughout the application, instead of <name>=<name>. This is especially useful for long Enscribe names.

Description of the illustration enscribe_pattern_tip_result.jpg

Profile

You can leave this field blank (to use the default job profile) or you can specify a profile for this compare pair, which overrides the job profile.

To select a profile:

Select the profile from the drop-down list or click the Browse button to open the Select a Profile page, which has filtering options.
To use Select a Profile
- Supply a search string to filter the list by profile name, and then click Apply Filter.
- Select the correct profile from the list, and then click Select.

Delta Processing

For more information about delta processing, see Using Delta Processing.

Select Enabled to use the delta processing feature. By default it is disabled. To use delta processing, server-side sorting must be enabled.

Key Mapping Method and Column Mapping Method

System Generated mapping is the default column mapping method for keys and comparison columns. You can work more with column mapping after you generate the mapping and save the compare pair to the repository.

Parent topic: Mapping Objects Using a Pattern

2.4.7.5 Generate the mappings

When you are finished with the preceding tasks, click Generate Mappings. A message at the top of the page indicates whether the operation succeeded or failed.

If there are any duplicate mappings, you are notified after you click Save. These can be removed later.

Parent topic: Mapping Objects Using a Pattern

2.4.7.6 Save the compare pairs to the repository

Until now, the new compare pairs are not saved to the repository. Saving your work frequently is a good practice. Whether your Oracle GoldenGate Veridata Web User Interface session is terminated intentionally or unintentionally, you will lose the compare pairs that you created unless they are saved to the repository. You can always return to any compare pair to perform additional configuration tasks.

The Preview tab displays compare pairs that were generated successfully, but not yet saved. To save your work, go to the Preview tab now.

Note:

To make it easy to return to your work on a compare pair, save the task as a Favorites shortcut.

Parent topic: Mapping Objects Using a Pattern

2.4.8 Mapping Objects Manually

If your source and target naming conventions do not support using the pattern mapping method, or if you would rather do the mapping yourself, use the manual mapping method to create compare pairs. Manual mapping is performed by pairing source and target tables or files one at a time by name. Manual mapping can be used in conjunction with pattern mapping as needed.

Use the Manual Mapping tab of the Compare Pair Configuration page to manually map objects. This tab is also used to configure row subsets.

You must have the Administrator or Power User role to configure compare pairs.

To use manual mapping, complete the six tasks in the following sections in order.

Confirm group information
Select the Datasource
Specify the database structures that contain the tables to be mapped. The selections vary by database type.
(Optional) Filter the lists
Map a table or file
Specify compare pair details
Generate the mappings
Save the compare pairs to the repository

Parent topic: Configuring Compare Pairs

2.4.8.1 Confirm group information

The group that you have chosen to contain the compare pairs is displayed at the top of the page, along with the source and target connections. Verify that these details are correct before proceeding.

Parent topic: Mapping Objects Manually

2.4.8.2 Select the Datasource

Specify the database structures that contain the tables to be mapped. The selections vary by database type.

Windows, UNIX, and Linux-based databases

Note:

V$DATABASE
DBA_PDBS

To use the filtering options, do the following:

Supply a search string to filter the list, and then click Apply Filter. The filter returns a list of names.
In the Select column of the list, click the name of the appropriate schema or catalog/PDB for this compare pair.
Click the Select button.

Enscribe databases

In the File Pattern box under Datasource Information, enter the Source and Target volume, sub-volume, and file patterns of the source and target files that you want to map as compare pairs. Use the format of volume.subvolume.file_pattern (for example, volume.subvolume.*).
Click the Refresh Source Files/Tables and Refresh Target Files/Tables buttons.

Parent topic: Mapping Objects Manually

2.4.8.3 (Optional) Filter the lists

By default, the Manual Compare Pair Mapping list shows all of the source and target objects that are contained by the specified datasources. You can filter the list at any time in the process of creating compare pairs. For example, after you finish mapping several compare pairs, it might be helpful to reduce the size of the list by using the filter to show only unmapped objects. This saves you navigation time.

To filter the list:

Expand Filters to show filter options.
- Show All: Shows all tables or files that are contained by the specified datasources. Those already mapped in another compare pair are denoted by a check mark, but you can still select them for this pair if desired. However, make certain to give this compare pair a different name.
- Show Mapped: Shows only previously mapped tables or files.
- Show Not Mapped: Shows only tables or files not mapped in any other compare pair.
- Names Like: Shows specific tables or files whose names contain the supplied string.
- Exclude Views: Shows all tables with no VIEWS in the specified datasources.
Click Apply Filter to enact the filter and update the view.

Parent topic: Mapping Objects Manually

2.4.8.4 Map a table or file

In the list of objects that can be mapped, there will be a check mark in the Mapped column for any object that is already mapped (unless a filter was used to filter out mapped objects).

Map one source object and one target object at a time:

Click the Select button next to the source object.
Click the Select button next to the target object.

As you select objects, their names are placed in the Compare Pair Name field under Compare Pair Details for further configuration of this pair.

Parent topic: Mapping Objects Manually

2.4.8.5 Specify compare pair details

So far, you have mapped object names, but there are still other configuration options that might apply to this compare pair.

Under Compare Pair Details, specify the following:

Compare Pair Name

You can keep the default name format of <source>=<target> or use another name of your choosing. Example default names:

 customers=customers2

 \FIN.$DATA.FN.CUST1=\FIN.$DATA.FN.CUST1

To specify a name:

A user-defined name cannot contain spaces but can contain underscores and equal signs. It is case-sensitive. If you will be generating a large number of compare pairs, use a naming convention that is intuitive. The Oracle GoldenGate Veridata Web User Interface displays the actual object names if you hover the mouse over the name of a compare pair.

Delta Processing

Delta processing can be used for all supported databases. Select Enable Delta Processing to use the delta processing feature. By default it is disabled. To use delta processing, server-side sorting must be enabled.

For more information about delta processing, see Using Delta Processing.

Profile

You can leave this field blank (to use the default job profile) or you can specify a profile for this compare pair that will override the job profile.

To select a profile:

Select the profile from the drop-down list or, to use filtering options, click the Browse button to open the Select a Profile page.
To use Select a Profile:
1. Supply a search string to filter the list by profile name, and then click Apply Filter.
2. Select the correct profile from the list, and then click Select.

Row Partitions

You can specify which rows to include or not to include in a comparison by specifying a SQL predicate statement or an Enscribe partition.

For more information on row partitions, see Configuring Partitions in Oracle GoldenGate Veridata.

Key Mapping Method and Column Mapping Method

System Generated mapping is the default column mapping method for keys and comparison columns. You can work more with column mapping after you save the compare pair to the repository

Parent topic: Mapping Objects Manually

2.4.8.6 Generate the mappings

When you are finished with the preceding tasks, click Generate Compare Pair. A message at the top of the page indicates whether the operation succeeded or failed.

If there are any duplicate mappings, you will be notified after you click Save. These can be removed later.

Parent topic: Mapping Objects Manually

2.4.8.7 Save the compare pairs to the repository

The Preview tab displays compare pairs that were generated successfully, but not yet saved. To save your work, select the Preview tab now.

Note:

To make it easy to return to your work on a compare pair, save the task as a Favorites shortcut.

Parent topic: Mapping Objects Manually

2.4.9 Reviewing and Saving Compare Pairs

Use the Preview tab to review, edit, and save or discard the compare pairs that you generate on the pattern-mapping or manual-mapping tab of the Compare Pair Configuration page.

Saving your work frequently is a good practice. If your Oracle GoldenGate Veridata User Interface session is terminated intentionally or unintentionally, you will lose the compare pairs on the Preview tab unless they are saved to the repository. You can always return to any compare pair after saving it, to perform additional configuration tasks.

Parent topic: Configuring Compare Pairs

2.4.10 Viewing details for a compare pair that is finished

Use the Details for Finished Compare Pair page to view near-real-time information about a compare pair that is finished being processed.

To access this page:

Under Finished Jobs in the navigation pane, select View By Compare Pair.
In the Compare Pair Name column of the Finished Compare Pairs list, click the name of the pair for which you want to view details.

The upper portion of the page shows specific information about the compare pair, including:

Compare Pair Name: the name of the compare pair.
Source Table and Target Table: the names of the underlying source and target objects.
Compare Pair Run ID: the identifier used by Oracle GoldenGate Veridata Server while running a comparison. It is useful for locating messages related to this comparison in the veridata.log file, which is used for troubleshooting purposes when resolving support cases.
Compare Pair Start Time: the time when processing started for this compare pair.
Compare Pair Run Duration: how long it took to process this compare pair.
Run Status: the outcome of this comparison (Waiting, Canceled, Canceling, Running, Finished)
Comparison Status: the status of the comparison (see Possible comparison status conditions.) If delta processing is enabled for this compare pair, the delta symbol is displayed.
Overall Rows Compared: the total number of rows in the source and target objects that were compared during this run of the compare pair.
Overall Rows With OOS: the number of rows in the source and target objects that are out-of-sync.

Using the charts

The charts on this page help you to determine what types of operations are causing the most out-of-sync conditions and which phases of processing are taking the most time to complete.

Overall Rows Compared pie chart

Plots the result from the Overall Rows Compared field according to the possible outcomes of the comparison: In-Sync or Out-Of-sync.

Operations Out-Of-Sync pie chart

Plots the number of out-of-sync rows according to their operation type:

Inserts: source table has the row, but the target does not.

Updates: source table has different row values from the target.

Deletes: target table has the row, but the source does not.

Compare Pair Performance table

Lists performance statistics based on each phase of a run: the sorting phase (for source and target), the initial comparison phase, and the confirmation phase. Note: sorting statistics are only available if the job profile specifies server-side sorting (row sorts are performed by Oracle GoldenGate Veridata, not by the database engines).

Start Time: The time that a run phase started.

Run Duration: The amount of time that it took to complete a run phase.

Rows Processed: The number of rows that were processed in a run phase.

Rows Per Second: The performance rate of a run phase, in terms of rows processed.

Bytes Processed: The overall volume of data, in bytes, that was processed in a run phase.

Rows Per Second: The performance rate of a run phase, in terms of bytes processed.

Compare Pair Performance History chart

Plots the performance of each of the run phases as a bar chart. To use this chart:

From View Step, select a run phase to plot.

Initial comparison step: Select Initial Compare.

Confirmation step: Select Confirm Out-Of-Sync.

Sorting process: Select Sorting.

From View Performance History By, select the criterion for this performance analysis. This sets the units of measure for the vertical axis of the chart. You can measure:

Number of rows or bytes that were processed

Number of rows or bytes per second that were processed

Run duration (the time spent processing the rows)

Other actions from here

To view a report of the results for this run of this compare pair, click View Comparison Report.

To view details about out-of-sync rows, including column values, click View Out-Of-Sync Rows. This link only appears if the compare pair has out-of-sync rows.

To edit this compare pair, click Compare Pair Configuration at the top of the page to go to the Compare Pair Configuration page.

Parent topic: Configuring Compare Pairs

2.5 Configuring Column Mappings

This section provides explains how to use column mapping for compare pairs.

Note:

You must create a compare pair before you can do column mapping. See Configuring Compare Pairs.

To access the Column Mapping Configuration page:

Click the name of a group on any page that displays groups. To access this page from the navigation pane, click Group Configuration, then the name of the group that you want to verify. This displays the Edit Group page.
Click Go to Compare Pair Configuration.
Click the Edit... link for the compare pare you are interested in.

To view an existing column mapping

Use the Existing Column Mapping tab to view the current mappings for the key columns and the comparison columns, and which method is being used to map them. This tab also shows the source and target columns that were excluded from being mapped, either explicitly by a user or because Oracle GoldenGate Veridata could not find a corresponding column with the same name.

To test the validity of the current mapping

It is possible for object metadata to change from the time that a column mapping was created to the time that you view it on the Existing Column Mapping tab. To confirm that the mapping is still valid, click Validate Existing Column Mapping.

See Validating column mappings.

To create or change a column mapping

Use the Column Mapping tab to create or change a column mapping.

You must have the Administrator or Power User role to create or edit column mappings.

Parent topic: Configuring Workflow Objects

2.5.1 Choosing a column mapping method

There are two methods for mapping key columns and comparison columns:

System-generated: Column mappings are configured automatically by Oracle GoldenGate Veridata at runtime based on current object metadata.
User-defined: Column mappings are configured manually by an Oracle GoldenGate Veridata user who has the Administrator or Power User role.

All new compare pairs default to System Generated for key columns and for comparison columns. You can change to a different mapping method at any time.

How these methods apply to keys

System Generated: If you know that the objects in a compare pair both contain a primary key or a unique index, you can leave the key mapping method set to the default of System Generated. The key columns will be mapped automatically. To map keys, Oracle GoldenGate Veridata finds all indexes on the source and target objects and tries to find a primary key on each one. If primary keys are not found, Oracle GoldenGate Veridata tries to use the smallest index (least number of columns), and then it maps the columns that have identical names and comparison formats. Any columns that cannot be matched are excluded from the configuration.
User Defined: If an object has neither a primary key or unique key, you can use the User Defined method to map key columns manually, one by one. You can also use the user-defined method to override existing keys or indexes, but the columns that you select to use as a key must ensure the uniqueness of rows. Also avoid using source and target indexes that have different precision levels or other characteristics that can reduce the accuracy of row selection, especially in a heterogeneous environment.

How these methods apply to comparison columns

System Generated: If the source and target comparison columns have the same names and comparison formats, you can leave the comparison column mapping method set to the default of System Generated. Oracle GoldenGate Veridata will map those columns automatically at runtime. Non-matching columns are excluded from the configuration. By default, this method includes all of the columns in a comparison. This method defaults to the hash comparison method. You can change the comparison method later by editing the compare pair.
User Defined: Use this method to map source and target columns manually and to control the comparison method.

You can combine these methods to speed up the mapping process. If most of the column names support system-generated mapping, you can use it and then switch to the user-defined method to map the remaining columns, or to exclude columns from the comparison. For example, you can exclude columns if you know that their values never change or if you expect their values to be out-of-sync.

Parent topic: Configuring Column Mappings

2.5.2 Validating column mappings

Validation is a preliminary test to determine whether or not the source and target table structures are compatible, and that they both have primary or unique key columns that match. You can perform a manual validation at any time. Oracle GoldenGate Veridata always performs a validation at runtime.

To perform the validation

Click Validate Column Mapping on the Existing Compare Pairs tab of the Compare Pair Configuration page. Make certain to place a check mark in the Select column for all of the pairs that you want to validate. The result is displayed under Validation Status.

To access the Compare Pair Configuration page:

Click the name of a group on any page that displays groups. To access this page from the navigation pane, click Group Configuration, then the name of the group that you want to verify. This displays the Edit Group page.
Click Go to Compare Pair Configuration.

You can also perform validation when you are configuring column mapping on the Column Mapping Configuration page.

You can also perform validation when you are configuring column mapping on the Column Mapping Configuration page. On the Compare Pair Configuration page, click Edit in the Column Mapping column.

What if a compare pair fails the validation test?

A result of other than or indicates that the validation failed.

The status indicates that a validation was not performed. Check the Select column to make sure that you included the compare pair in the validation.

If you are manually mapping columns and a validation fails, click Reset to go back to the former mapping and then try the mapping and validation again. It is possible that you selected the wrong source and target columns as a pair.

If a second manual mapping fails validation, or if a system-generated mapping fails validation, use the color of the status icon as your guide to the cause and then check the underlying data for incompatibilities.

If a previous validation of the compare pair succeeded, but the current one failed, this probably means that the structure of the underlying objects changed.

Note:

A successful preliminary validation does not guarantee a successful runtime column validation. It is possible that changes to the structure of a source or target object can invalidate the original test and cause the job to fail at runtime. The validation process should only be used as a preliminary test for current incompatibilities.

Parent topic: Configuring Column Mappings

2.5.3 Using Delta Processing

This section provides answers to typical questions about the delta processing feature that is available for all Oracle GoldenGate Veridata supported databases.

What is Delta Processing?

Parent topic: Configuring Column Mappings

2.5.3.1 What is Delta Processing?

How does it work on NonStop Platforms?

Oracle GoldenGate Veridata finds a changed block by detecting a change in its Volume Sequence Number (VSN) since the time of the last comparison. The VSN is a disk-specific change number that increments sequentially with each database operation that is performed on the data. Each time that a row changes, there is a change in the VSN of the disk block where the row resides.

There is no relationship between a VSN in a file on one disk and a VSN on another. Oracle GoldenGate Veridata tracks VSNs on a per-partition basis on the source and target disks and maintains its own correlations to perform accurate delta comparisons. Once you enable delta processing, it is used for all subsequent runs until you disable it again.

Note:

The first run of a compare pair always compares all of the rows in the source and target objects to establish an initial VSN state from which to evaluate deltas in future runs.

How does it work on all other Platforms?

Oracle GoldenGate Veridata compares a source database table to the target database table. The source and target tables are configured using compare pairs, which are grouped and added to a job to run the comparison (see Configuring Compare Pairs). When all the rows in the table are compared, it is a Full Comparison Job.

During the subsequent runs of a comparison job, the comparison of the tables can be performed based on what has changed in the tables from previous job run; these jobs are Delta Processing Jobs. Delta processing is usually performed on tables that contain a large number of rows so it is probable that in these tables there will be columns eligible for delta processing. The delta comparison column must contain a value that is modified every time that the row is modified and this value must always be increasing. Any data type that meets this requirement is supported. By default, the columns of the table that are mapped to Numeric or Timestamp comparison formats are supported. For example, TIMESTAMP, TIMESTAMP_TZ, and NUMBER.

The delta base is the value of the Delta Column on the basis of which the delta comparison was performed. Every time a comparison is run, a delta base value is captured. Depending on the number of delta comparison jobs performed, there can be multiple delta base values so a list of delta base values for the compare pair is generated. For example, the first time a Full Comparison is run and the maximum value of the Delta Column is the delta base, DeltaBase-1. A second Delta Processing Job run based on DeltaBase-1 results in DeltaBase-2 being captured again as the maximum of Delta Column. In the third run, you can use either DeltaBase-1 or DeltaBase-2 for the comparison or run a Full Comparison Job.

When Should I use Delta Processing?

Delta processing is suitable for use with very large Enscribe files and SQL tables that, otherwise, would take a long time to process. It does consume additional overhead, so it is probably not practical for use with smaller sets of data. Try running a test comparison without delta processing first. If, in your opinion, the compare pair takes too long to process, try running it again with delta processing enabled. If the delta-enabled run is significantly shorter than the first test, continue to use it. If there is only marginal improvement, it might be better to disable delta processing to prevent the added overhead. The performance gains of delta processing are in the initial comparison step of the run. Delta processing can cause the confirmation step to be longer if the source and target rows end up on different data blocks.

What Process Performs the Delta Processing?

The delta processing is performed by the Oracle GoldenGate Veridata Agent.

For NonStop platforms, the VSN information is retrieved by a privileged process named vsnserv. During the installation of Oracle GoldenGate Veridata Agent on a NonStop platform, PROGID was used for the vsnserv program to run as SUPER.SUPER to be able to read the file labels for this purpose.

For all other platforms, the delta processing queries the compare pair to retrieve the delta base values for both the source and target tables, which creates the column mapping.

What sorting method can be used with delta processing?

To use delta processing, you must enable server-side sorting by setting the sorting method to Server within the profile that is associated with the compare pair or the one that is associated with the job when you run it.

Tip:

If you always will be using delta comparisons, consider setting the sorting method to Server within the default Oracle GoldenGate Veridata profile. That way, nobody will forget to select the correct profile when the jobs are run.

What other important things should I know when using delta processing?

The delta processing mechanism can fail to detect an out-of-sync delete, if that delete was the only source row that was modified in a block, and if that delete did not get propagated to the target. In such a case, the block on the target that contains the relevant row does not get modified, so it is skipped by the target Veridata Agent during delta processing.

The delta value field is pre-populated in the column mapping UI, based on the query for retrieving the delta value.

Support Considerations

The following table shows the supported delta column types. The possible list of delta columns for the delta configuration UI is identified by reviewing the compare formats for the corresponding column-pairs in the source and target tables.

Veridata Comparison Format	Can it be Delta Column?
`STRING`	No
`STRING_EN`	No
`TIMESTAMP`	Yes
`TIMESTAMP_TZ`	Yes*
`DATETIME`	Yes
`SMALLDATETIME`	No
`DATE`	No
`TIME`	No
`NUMBER`	Yes
`FLOAT`	No
`BINARY`	Yes*
`BLOB`	No
`CLOB`	No
`INTERVAL`	No
`DEC_FLOAT`	No
`BINARY_TIMESTAMP`	Yes*
`SYBTIME`	No
`CLOB_NFC`	No
`STRING_NFC`	No

Note:

* denotes that Hive doesn’t support the delta column. Supported data types for Hive are NUMBER, TIMESTAMP and DATETIME.

For Oracle Database, the ORA_ROWSCN pseudo delta column is supported, and selected by default when ROWDEPENDENCIS are enabled for that table.

For DB2 for i, z/OS, and LUW, columns having the GENERATED FOR EACH ROW ON UPDATE AS ROW CHANGE TIMESTAMP clause is selected by default.

For all supported databases, TIMESTAMP columns are shown in order first then followed by NUMERIC columns.

Only one column is supported for delta processing and is similar to:

SELECT column-names from table name where delta_column delta_condition supplied_delta_value

For example:

select * from TableA where startdate >= '12-01-2012 21:24:00'

How do I enable or disable delta processing?

You can enable or disable delta processing for database compare pair from the Delta Processing tab and the Existing Compare Pairs tab of the Compare Pair Configuration page. In addition, you can use the Delta Processing Enabled check box on the Run Configuration page to toggle this feature.

How do I configure delta processing?

On the Column Mapping Configuration page, click the Delta Processing tab.By default, the options are automatically populated and the Enable Delta Processing check box is selected. You can disable the feature by clearing the check box, which renders all other options inactive.

You can use the defaults or change any of the following:

the source or target columns for processing,
whether to use a source or target query
the source or target database query

After you have configured the delta processing for the compare pair, click Save to apply your changes.

How do I know that delta processing is being used?

When a compare pair is configured for delta processing, a delta symbol is displayed in the lists of the Edit Group and Compare Pair Configuration pages.

If delta processing is enabled for at least one compare pair in a job that is selected on the Run/Execute Job page, there will be a check mark in the Delta Processing Enabled box.

Selecting Delta-Base value

Veridata automatically selects the Delta column value of the latest in-sync comparison as delta base value.

If Veridata could not select the delta base value automatically the “Select …” link is shown in the run-jobs page. And On click of that all the delta base values are listed and users can choose a delta base value from it, it is always advisable to choose in-sync comparison for delta-base values.

Delta base time can also be selected by navigating to the finished compare details for the previous comparison and clicking the run button. Veridata will use the time from that comparison as the delta base time. If the out-of-sync data from the previous comparison has not been fixed, those out-of-sync rows may not be detected again until another complete comparison is run.

When Veridata automatically selects the Delta-Base value the time stamp of the comparison is shown as a link to the user in run-jobs page with tool-tip.

You can select that link to choose and alternate delta base value.

Can I override delta processing when I run a job?

What if I perform maintenance on the tables or files for which I am using delta processing?

When you perform maintenance on objects in a compare pair that has delta processing enabled, the best practice is to disable delta processing for the next run so that Oracle GoldenGate Veridata compares all of the rows. You can disable delta processing at the compare pair level or as a job override. Starting again with a full comparison allows a new delta base state to be established and will make subsequent delta comparisons faster. Otherwise, delta processing could actually take longer than a complete comparison of all rows.

For example, if a 'FUP RELOAD' is performed on the source, but not on the target, it could cause delta processing to return a much larger number of rows from the source than from the target, based on the last delta state. The source rows that are returned would be rows that actually did not change. This happens because FUP moves records around and combines blocks, but does not change the data. However, the VSNs for the affected blocks will change. Oracle GoldenGate Veridata cannot detect that the reload was done since the last delta state. Thus, the next time that the VSN for a block changes, all of the rows in that block will be returned.

Conversely, on the target, no VSNs are changed for the corresponding data (because a reload was not done there), so those rows are not returned for delta processing. This anomaly will be resolved by the confirmation step, but this slows the overall comparison process because that step is much slower than the initial comparison step.

Parent topic: Using Delta Processing

2.6 Configuring Profiles

A profile is a set of global processing parameters, each containing unique settings for a specific purpose. Oracle GoldenGate Veridata provides a default profile, but you probably will want to create your own profiles. too. You can create as many profiles as needed and associate them with any job or compare pair (to override the job profile; see Configuring Compare Pairs). You can override profile assignments at run time.

Profiles are managed from the Profile Configuration page. All profiles that exist within the Oracle GoldenGate Veridata repository are shown in the Existing Profiles list on this page.

To filter the Existing Profiles list:

Expand Filters and type a search string. The filter matches a search string character for character.
- Names Like: filters by profile name.
- Description Like: filters by a string in the job description.
Click Apply Filter to display the selected profiles.

To configure profiles

You must have the Administrator or Power User role to create, edit, or delete a profile.

To create a profile:

To create a profile, click New. The New Profile Assistant prompts for a name and description and then directs you to the Edit Profile page so that you can configure profile settings.
To edit a profile:

To edit a profile, click its name in the Existing Profiles list, or select it in the Select column and then click Edit. The Edit Profile page is displayed. See Editing Profile Settings.
To delete a profile:

To delete a profile, select it in the Select column of the Existing Profiles list, and then click the Delete button.
To change the default profile:

Oracle GoldenGate provides a default profile (see Using the default profile) that is used for all of the compare pairs and jobs that are not linked to a user-defined profile. This profile is used automatically unless another profile is selected when creating a compare pair or a job, or when running a job. You can change the settings of the default profile by editing it.

Other configuration tasks

Configuring Connections

Configuring Groups

Configuring Jobs

Parent topic: Configuring Workflow Objects

2.6.1 Editing Profile Settings

This section describes the parameters that you can edit for a profile.

The profile settings are categorized as follows:

General: Controls output options.
Sorting Method: Controls sorting method and memory management. Data is sorted to match keys (or a key specification) so that the correct source and target rows are compared.
Initial Compare: Controls parameters for the process that performs the initial compare step.
Confirm-Out-Of-Sync: Controls parameters for the process that performs the confirmation step.
Repair: Controls parameters for the repair process.

Specifying a Sorting Method

The Sorting Method settings of the Edit Profile page specify whether data sorting will be performed by the database or by Oracle GoldenGate Veridata Server. Specify this method in the Sort Data Using option on the Edit Profile page.

By default, Oracle GoldenGate Veridata uses the database to sort data for comparison. This default is due to historical conditions that are no longer valid. Server-side sorting is the current recommended sorting method. Database sorting should only be considered when the ordering produced by the database is identical to the ordering produced by Oracle GoldenGate Veridata Server. Following is a list of the types of conditions that will produce differing sorted ordering of the rows:

Character encoding conditions: Oracle GoldenGate Veridata compares character data as UTF-8 encoded bytes. To match server-side ordering, key columns that contain character data must contain only ASCII data or be encoded using UTF-8, and the database must use binary comparisons for character data (no comparisons that are case-insensitive or specific to a locale).
Some datetime data types, such as Teradata TIME, may sort differently in the database and in Oracle GoldenGate Veridata Server.
To make database ordering consistent with the ordering done by Oracle GoldenGate Veridata Server, the Oracle GoldenGate Veridata Agent may add ORDER BY clauses to the initial comparison SELECT statement that will make the database ignore indexes on the columns. An example is TIMESTAMP with TIMEZONE data, where Oracle GoldenGate Veridata orders the data by the string representation of the data rather than by the absolute time.

When Oracle GoldenGate Veridata Server performs the sort, the Oracle GoldenGate Veridata Agents return data in the natural order that is provided by each database, and then the data is sorted by two server sort processes, one to sort source rows and the other to sort target rows. Server-side sorting supports a maximum row length of 32768 bytes. This limit normally is not exceeded when the hash comparison method is used.

Specifying Temporary Storage Directory for Source and Target Data

Specifies a location on the source disk or target disk to use as temporary storage when there is not enough memory to process all of the data that is being sorted. If no locations are defined, the default is to use a directory under the Oracle GoldenGate Veridata Server home location.

Choosing locations on different physical drives might speed up comparisons in some circumstances. You can specify multiple locations for each process, separating each one with a semicolon (for example /tmp/sort1; /tmp/sort2). All locations specified must already exist. The drives used should have sufficient free disk space. To calculate the approximate amount of space needed, use this formula:

1.5 * (Trows * (Tkey + 20)) * nTables

where:

Trows = the number of rows in table

Tkey = the average size of the table key, in bytes

nTables = the number of tables that are being compared

Parent topic: Configuring Profiles

2.6.2 Using the default profile

The default Oracle GoldenGate Veridata profile is applied when a custom profile is not linked to a job or compare pair.

The name of the default profile is $default.

To view and edit the settings of this profile, use the Edit Profile page.

To edit the default profile, you must have the Oracle GoldenGate Veridata administrator role.

To restore the default profile to its original (installed) settings, click the Reset all values back to Oracle GoldenGate defaults button on the Edit Profile page.

Parent topic: Configuring Profiles

2.7 Configuring Jobs

To run comparisons, you must run a job. The job configuration determines which compare groups are processed.

Note:

Before creating a job, you must create at least one compare group (see Configuring Compare Pairs) to link to the job. To use customized runtime parameter settings, you must also create at least one profile (see Configuring Profiles). Otherwise, the job will use the default profile that is supplied by Oracle GoldenGate Veridata.

Jobs are managed from the Job Configuration page. To access this page, click Job Configuration in the navigation pane. All jobs that are defined within the Oracle GoldenGate Veridata repository are shown in the Existing Jobs list on this page.

To filter the Existing Jobs list

Expand Filters and type a search string. Strings are matched character for character.
- Names Like: filters by job name.
- Description Like: filters based on a string in the description.
Click Apply Filter to display the selected jobs.

To configure a job

You must have the Administrator or Power User role to create, edit, or delete a job.

To create a job

To create a job, click New. The New Job Assistant is displayed. The assistant will prompt for:
- a name and description.
- groups to be linked to the job.
- (optional) a profile to link to the job; otherwise the default profile will be used.
- (optional) connections to link to the job; connections specified will override those linked to each group.
To edit a job

To edit a job. click the name of the job in the Existing Jobs list, or select it in the Select column and then click Edit.

The Edit Job page is displayed.
To delete a job

To delete a job, select it in the Select column of the Existing Jobs list, and then click the Delete button.
To run a job

To run a job, select it in the Select column of the Existing Jobs list, and then click the Run button to go to the Run/Execute Job page.

Other configuration tasks

Configuring Connections

Configuring Groups

Configuring Profiles

Parent topic: Configuring Workflow Objects