4 Working with Data Replication
You can perform all data replication tasks from the Administration Server home page. You can add Extracts and Replicats after creating your deployments.
Before you begin creating Extract processes, you need to:
- Add trandata or schematrandata
- Enable supplemental logging for SQL Server CDC capture
- Create and enable heartbeat tables.
Before creating Replicats, you need to:
- Create the checkpoint table
You'll also create a user with the admin role from the Administration Server. The initial user created during deployment is a security admin role. The security admin user should not do other tasks. So, you need to create users with the admin role and this user is used to create Extract and Replicat processes.
Users in Service Manager deployment are different from Administration Sever deployment. Service Manager deployment. users are created from the Service Manager web interface, and normal deployment users are created from the Administration Server web interface. Users in Service Manager deployment have control of Service Manager functions like stopping, starting, enabling, and disabling services. Users created from the Administration Server can create Extract, Replicat, and other processes.
Topics:
Quick Tour of the Administration Server Home Page
When you click the Administrator Server link on the Service Manager home page, the login page for the Administration Server is displayed. After logging in, you can configure Extract and Replicat processes from this Web UI.
The Administration Server home page is used to add Extracts and Replicats. The table on the home page displays the severity of critical events. You can also use the left-navigation pane to access various configuration details, a list of severity issues with their diagnosis, and a list of administrators.
Now, that you have an overview of the Administration Server home page, let’s understand some of the key actions that you can perform from this page.
Action | Description |
---|---|
View the home page in tabular format |
Use the Table Layout swivel to turn the tabular format on and off. |
View Extracts and Replicats |
The statistical representation the home page displays current state of Extracts and Replicats (Starting, Running, Stopped, Abended, Killed) |
Add an Extract |
|
Create a Replicat |
|
Stop and start Extracts |
|
Stop and start Replicats |
|
View and search critical events |
Monitor severity of events using the Critical Events table and also search for specific events, if required. |
How to Add a Database Credential
To create and run Extract and Replicat processes, you need to set up database credentials.
-
Launch the Administration Server interface and log in.
-
Click Configuration from the Application Navigation pane.
-
Click the + sign next to Credentials, and set up your new credential alias, then click Submit.
-
Click the Login icon to verify that the new alias can correctly log in to the database.
If an error occurs, click the Alter Credential icon to correct the credential information, and then test the log in.
You can edit existing credentials to change the user name and password. Delete a credential by clicking the trash icon.
When you successfully log into your database, you can add and manage checkpoint tables, transaction information, and heartbeat tables. All of the tables can be searched using the various search fields. As you type, the table is filtered and you can use the search button with the search text.
Before Creating an Extract
As a prerequisite to create the primary Extract, you must perform the actions described in this topic.
ADD TRANDATA or ADD SCHEMATRANDATA
To enable supplemental logging at the table level, use the ADD
TRANDATA
command and for schema level, use ADD
SCHEMATRANDATA
. For details, see ADD TRANDATA
and ADD SCHEMATRANDATA
You can skip
ADD TRANDATA
in case of initial load without CDC.
Enable Supplemental Logging for SQL Server CDC
To capture tables in case of CDC capture, you need to enable supplemental logging for the tables. See Enabling Supplemental Logging old (CDC Extract).
Create Heartbeat Table for SQL Server CDC
-
From the Administration Server, select Configuration from the navigation pane.
-
Select the + sign next to the Heartbeat section of the Database tab. You'll need to enter the values for the heartbeat frequency, retention time, and purge frequency.
You can create the heartbeat table using the ADD HEARTBEATTABLE
command from the Admin Client or GGSCI. See ADD HEARTBEATTABLE.
How to Add Extracts
Set up database credentials to create and run Extracts using the steps in How to Add a Database Credential.
-
Log in to the Administration Server using the Oracle GoldenGate user credentials.
-
From the Overview page of the Administration Server, click the + sign next to Extract.
-
Choose the type of Extract to create and click Next. The types of Extract are:
-
Integrated Extract
-
Classic Extract
-
Initial Load Extract
Note:
An Initial Load Extract cannot be started from a secure deployment. You can only start it in a non-secure deployment. -
-
Enter and select the required information, which is designated with an asterisk (*). For all Extracts the Process Name, Credential Domain, and Credential Alias are required. A description is optional. The Create new credential option is common to all Extracts.
You can configure the following additional required and optional details based on the type of Extract you selected to create:Options Description Database Basic Information Process Name
Name of the Extract process. The name of the Extract process can be up to 8 characters.
All databases
Description
Description of the Extract process being created.
All databases
Intent
Describes the purpose of creating the Extract. The default option is Unidirectional. Other options are High Availability, Disaster Recovery, N-Way, which are informational only.
All databases
Begin
Used to set the beginning location in the redo or transaction log from which the Extract will start to capture data. Available options are Now, Custom Time, CSN or Position in Log, and EOF depending on the supported database.
All databases
Trail Name
A two character trail name.
All databases
Trail Subdirectory, Size, Sequence, and Offset
You can further configure the trail details.
All databases
Remote
Enable this option if the Extract trail is remote.
For Oracle databases, enable this option if the Extract trail is to be written directly to a remote Oracle GoldenGate Classic installation.
For MySQL, setting this option enables the
TRANLOGOPTIONS ALTLOGDEST REMOTE
parameter to support a remote Extract, and is not related to trails.Oracle, MySQL
Registration Information CSN
Commit Sequence Number (CSN) value
Oracle
Share
Choose the method to share the LogMiner data dictionary. Options are:-
Automatic: This option allows the system to choose the method for sharing the dictionary .
-
None: Choosing this option, will not allow the dictionary to be shared.
-
Extract: Choose this option to allow sharing the logminer dictionary for specific Extract.
Oracle
Optimized
Enable this option to optimize the Extract registration.
Oracle
Downstream Capture
Enable this option to set up a downstream Extract for log mining.
Oracle
Register Only
Use this option to just register the Extract and not add the Extract. The registration creates the replication slot when you register the Extract or use the Register Only option.
PostgreSQL
Source Database Credential Create new credential If you haven't set up your database login credentials, you can create and save the database login credentials from here. All Credential Domain Create a domain for the database. All Credential Alias Specifiy a credential for the database login. All User ID Specify a user name for logging into the database. All Password, Verify Password Enter the password used to login to the database and reenter the password to verify. All Credential Domain
Saves the credential user under the specified domain name. Enables the same alias to be used by multiple Oracle GoldenGate installations that use the same credential store. The default domain is Oracle GoldenGate. All databases
Credential Alias
Specifies an alias for the user name. Use this option if you do not want the user name to be in a parameter file or command. If
ALIAS
is not used, the alias defaults to the user name, which then must be used in parameter files and commands where a login is required. You can create multiple entries for a user, each with a different alias, by using theADD USER
option withALIAS
.All databases
Downstream Mining Mining Credential Domain Domain name of the downstream mining database. Oracle Mining Credential Alias Alias for the mining downstream database. Oracle No UserID Enable this option if there is no source database connection. Selecting this option enables the ADG fetch options. Oracle ADG Fetch Credential Domain Domain name for the ADG fetch database. Oracle ADG Fetch Credential Alias Domain alias for the ADG fetch database. Oracle You must enter the options for Managed Processes while creating all types of Extract processes. The following table provides these options:Option Description Profile Name
Provides the name of the autostart and autorestart profile. You can select the default or custom options.
If you have already created a profile, then you can select that profile also. If you select the Custom option, then you can set up a new profile from this section itself.
Critical to deployment health
(Oracle only) Enable this option if the profile is critical for the deployment health.Note:
This option only appears while creating the Extract or Replicat and not when you set up the managed processes in the Profiles page.Auto Start
Enables autostart for the process.
Startup Delay
Time to wait in seconds before starting the process
Auto Restart
Configures how to restart the process if it terminates
Max Retries
Specify the maximum number of retries to try to start the process
Retry Delay
Delay time in trying to start the process
Retries Window The duration interval to try to start the process Restart on Failure only If true the task is only restarted if it failes Disable Task After Retries Exhausted If true then the task is disabled after exhausting all attempts to restart the process. -
-
Click Next.
-
You can edit the parameter file in the text area to list the table details that you are interested in capturing. For example,
table source.table1;
You can select Register Extract in the background to register the Extract in the background asynchronously.
-
You can select Register Extract in the background to register the Extract in the background asynchronously.
-
Click Create and Run to create and start the Extract. If you select Create, the Extract is created but you need to start it using the Extract drop-down on the Overview page.
You are returned to the Overview page of the Administration Server. Select the Action list if you want to look at the Extract details such as process information, checkpoint, statistics, parameters, and report.
Using Extract Actions
Once you create an Extract, you can monitor various details associated with the Extract from the Administration Server home page.
You can change the status of the Extract process using the Action button to:
Action | Result |
---|---|
Details |
Displays the following tabs:
|
Start/Stop |
The Extract starts or stops immediately. |
Start/Stop (in the background) |
The Extract is started or stopped using a background process. |
Start with Options |
Allows you to change the Extract CSN options, then starts the Extract. |
Alter |
This option is available only when the Extract is stopped. Allows you to change when the Extract begins, the description, and the intent. It does not start the Extract. |
Delete |
This option displays only when the Extract is stopped. Deletes the Extract if you confirm the deletion. |
When you change the status, the list options change accordingly. As status are changing, the icons change to indicate the current and final status. The events are added to the Critical Events table. Additionally, progress pop-up notifications appear at the bottom of the page.
Before Creating Replicat
Before you start creating Replicat, create the checkpoint table.
-
From the Administration Server, go the Configuration page using the navigation pane.
-
Click the + sign next to the Checkpoint section on the Database tab.
-
Enter the checkpoint table name in the Checkpoint Table box. The table name must be a two-part or three-part value. For example, GGADMIN.CHKP1.
You can add the checkpoint table using the ADD CHECKPOINTTABLE
command
from the Admin Client or GGSCI. See ADD CHECKPOINTTABLE
How to Add a Replicat
You can add Replicats for the target deployment from the Administration Server.
Creating a Parallel Replicat
You can create a parallel Replicat from the user interface or the command line interface.
Before you start creating the parallel Replicat, make sure that you've select the checkpoint table.
Creating a Non-Integrated Parallel Replicat with the Administration Server
-
Log into the Administration Server.
-
Click Application Navigation on the top-left corner.
-
Select Configuration. Make sure that the database credentials are correct and the database user is connected. See How to Add a Database User
for details. -
Click the + sign to add a checkpoint table.
-
Enter the
schema.name
of the checkpoint table that you would like to create, and then click Submit. -
Validate that the table was created correctly by logging out of the Credential Alias using the log out database icon, and then log back in.
Once the log in is complete, your new checkpoint table is listed.
-
Click Overview to return to the main Administration Server page.
-
Click the + sign next to Replicats.
-
Select Nonintegrated Replicat then click Next.
-
Enter the required information making sure that you complete the Credential Domain and Credential Alias fields before completing the Checkpoint Table field, and then select your newly created Checkpoint Table from the list.
-
Click Next, and then click Create and Run to complete the Replicat creation.
Creating a Non-Integrated Parallel Replicat with the Admin Client
-
Go the
bin
directory of your Oracle GoldenGatehome directory.cd $OGG_HOME/bin
-
Start the Admin Client.
adminclient
The Admin Client command prompt is displayed.
OGG (not connected) 12>
-
Connect to the Service Manager deployment source:
connect https://localhost:9500 deployment Target1 as oggadmin password welcome1
You must use http or https in the connection string; this example is a non-SSL connection.
-
Add the Parallel Replicat, which may take a few minutes to complete:
add replicat R1, parallel, exttrail bb checkpointtable ggadmin.ggcheckpoint
You could use just the two character trail name as part of the
ADD REPLICAT
or you can use the full path, such as/u01/oggdeployments/target1/var/lib/data/bb
. -
Verify that the Replicat is running:
info replicat R1
Messages similar to the following are displayed:
REPLICAT R1 Initialized 2016-12-20 13:56 Status RUNNING NONINTEGRATED Parallel Checkpoint Lag 00:00:00 (updated 00:00:22 ago) Process ID 30007 Log Read Checkpoint File ./ra000000000First Record RBA 0
Basic Parameters for Parallel Replicat
The following table lists the basic parallel Replicat parameters and their description.
Parameter | Description |
---|---|
MAP_PARALLELISM |
Configures number of mappers. This controls the number of threads used to read the trail file. The minimum value is |
APPLY_PARALLELISM |
Configures number of appliers. This controls the number of connections in the target database used to apply the changes. The default value is |
MIN_APPLY_PARALLELISM
|
The Apply parallelism is auto-tuned. You can set a minimum and maximum value to define the ranges in which the Replicat automatically adjusts its parallelism. There are no defaults. Do not use with |
SPLIT_TRANS_REC |
Specifies that large transactions should be broken into pieces of specified size and applied in parallel. Dependencies between pieces are still honored. Disabled by default. |
COMMIT_SERIALIZATION |
Enables commit |
Advanced Parameters |
|
LOOK_AHEAD_TRANSACTIONS |
Controls how far ahead the Scheduler looks when batching transactions. The default value is 10000. |
CHUNK_SIZE |
Controls how large a transaction must be for parallel Replicat to consider it as large. When parallel Replicat encounters a transaction larger than this size, it will serialize it, resulting in decreased performance. However, increasing this value will also increase the amount of memory consumed by parallel Replicat. |
Example Parameter File
replicat repA
userid ggadmin, password ***
MAP_PARALLELISM 3
MIN_APPLY_PARALLELISM 2
MAX_APPLY_PARALLELISM 10
SPLIT_TRANS_RECS 1000
map *.*, target *.*;
Using Replicat Actions
Various Replicat actions can be performed from the Administration Server Overview page.
You can change the status of the Replicat process using the Actions button to:
Action | Result |
---|---|
Details |
Displays the Process Information page that has the following details:
|
Start/Stop |
The Replicat starts or stops immediately. |
Start/Stop (in the background) |
The Replicat is started or stopped using a background process. |
Start with Options |
Allows you to change the Replicat start point, CSN, filter duplicates, and threads options, then starts the Replicat. |
Force Stop |
The Replicat is immediately, forcibly stopped. |
Alter |
Allows you to change when the Replicat begins, the description, and the intent. It does not start the Replicat. |
Delete |
Deletes the Replicat if you confirm the deletion. |
When you change the status, the list options change accordingly. As status are changing, the icons change to indicate the current and final status. The events are added to the Critical Events table. Additionally, progress pop-up messages appear in the bottom of your browser.
How to Use the Master Keys and Encryption Keys
You can set the master keys and encryption keys using the Key Management tab in the Configuration page of the Administration Server.
Using Master Keys
If you want to encrypt your data, then create a Master Key by clicking the + sign in the Master Key section. The master key is generated automatically.
You can change the status of the key to Available or Unavailable, by clicking the edit icon in the Master Key table. You can also delete the Master Key from the table by clicking the delete icon.
For details on the Master Key concept, see Encrypting Data with the Master Key and Wallet Method. .
Using the Encryption Keys
To use this method of data encryption, you configure Oracle GoldenGate to
generate an encryption key and store the key in a
local ENCKEYS
file. The
ENCKEYS
file must be secured
through the normal method of assigning file
permissions in the operating system. This procedure
generates an AES encryption key and provides
instructions for storing it in the
ENCKEYS
file.
To generate the ENCKEYS files, click the + sign in the Encryption Keys section. The Encryption Keys is generated.
For details on the Encryption Keys concept, see the Encrypting the Data with the ENCKEYS Method.
How to Access the Parameter Files
The Global parameters, Extract, Replicat parameter files are available in the Parameter Files section of the Administration Server.
You use the Administration Server Configuration page and Parameter Files tab to work with your various parameter files.
You use the different parameter file options:
-
Select the Configuration option from the Administration Server left-navigation pane.
-
Select the Parameter Files tab.
A list of existing parameter files is displayed along with the GLOBALS parameter file.
-
If you select any of the parameter files, you are presented with the option to edit or delete the selected file. If you want to change the GLOBALS parameter file, you need to stop and restart all of the services.
-
Click + add parameter files.
-
Enter the file name and the required parameters. Make sure to enter the file name with the
.prm
extension. -
Click Submit. The new parameter file is displayed in the list of parameter files.
-
Identify the GoldenGate Deployment ETC Home:
-
Go to Service Manager Overview page.
-
Click the deployment from the Deployments section for which you need to find the parameters file.
-
Under the Deployment Detail window, navigate to the Oracle GoldenGate deployment
/etc
home directory. -
Go into the
/config/ogg
directory where the parameter file is located.
-
[oracle ~]$ cd /opt/app/oracle/gg_deployments/Atlanta/etc
[oracle etc]$ cd conf/ogg[oracle ogg]$ lsEXT_DEMO.prm GLOBALS REP_DEMO.prm
Setting Up Automated Tasks
The Administration Server performs the commands that were executed by the GGSCI utility in previous releases. However, the Administration Service provides enhanced capabilities to perform these tasks, while still being compatible with GGSCI.
Purging Trails
The Purge Trail page works the same way as the Manager PURGEOLDEXTRACTS
parameter in the Classic Architecture. It allows you to purge trail files when Oracle GoldenGate has finished processing them. Automating this task ensures that the trail files are periodically deleted to avoid excessive consumption of disk space.
From the Tasks tab, when you select the Purge Trail page, it allows you to configure the Administration Service purge trail process.
-
Add a Purge Trail task by clicking the
+
sign . -
Enter the Operation Name of the Administration Service task. The operation name is case sensitive. For example, you can create an operation with the name TASK1 and another operation named task1.
-
Enter the trail path or trail name in the Trail field.
-
Click the + sign to add the trail to the Selected Trails list.
-
If you don’t need to use Checkpoints, disable the option Use Checkpoints.
-
Set the Keep Rule value to specify the maximum number of hours, days, or number of files for which the Purge Trails task needs to be active.
-
Specify the number of hours or days when the purge trails task has to run, in the Purge Frequency field and click Submit.
-
Use the Purge Trails task table to edit or delete the task, as required.
Also see PURGE EXTTRAIL.
Purging Tasks
You can automatically purge processes associated with an Administration Service.
-
Enter the Operation Name that you need to set up for automatic purging.
-
Select the Extract or Replicat task (initial load process) Process Name for the operation. The list contains all processes so ensure that you select the correct task.
-
Select the Extract or Replicat task (initial load) Process Type for the operation.
-
If you enable Use Stop Status, the status of the task is used to perform the purge task.
-
Enter the hours or days after which you need to purge the process and click Submit.
-
Edit or delete the purge process task using the relevant icon from the Purge Tasks table.
Reporting Lag
-
From the Tasks tab, click Lag Report.
-
The Action column contains all the options to delete, alter, refresh, and view the lag report task details.
-
Select the required option.
-
If you select the Alter Task option, you are presented with options to edit the lag report. The options are:
-
Enabled: To keep processing the lag report task.
-
Check Every (in minutes): To set a time interval to check the lag report.
-
Report: To log report for the task.
-
If Exceeds: To specify a threshold after which a warning would be initiated.
-
Warning: To allow a warning to be generated incase the lag threshold exceeds the specified limit.
-
When Exceeds: The lag threshold after which the warning is triggered.
-
-
Click Submit.
Review Critical Events
You can review and search for critical events from the Administration Server home page, once you set up the distribution path.
Once you set up the Extracts and Replicats along with the Distribution path, you are able to see the critical events associated with them.
Search for Critical Events from the Review Critical Events Table
The Review Critical Events table displays the severity, error code, and error messages for critical events. You can view 20 error messages on a single page and you can also search for specific events.
Additionally, you can examine events in depth from the Performance Metrics Server. For details see Quick Tour of the Performance Metric Server home page.
How to Configure Managed Processes
Oracle GoldenGate Administration Server provides options to set up profiles for managed Extract and Replicat (ER) processes. These processes are assigned auto-start and auto-restart properties to control their life cycles.
-
Click Profile from the Administration Server navigation pane.
-
In the Managed Process Settings tab, you can click + sign to start creating a profile. There's also a default profile preset on this page.
-
Enter the details for the profile options including the Profile Name, Description, Auto Start and Auto Restart options. See the following table for Auto Start and Auto Restart options
Option Description Extract Type Intent
What you want the Extract to be used for, such as High Availability or the Unidirectional default.
Classic, Integrated, and Initial Load
Begin
How you want the Extract to start. At a custom time that you select, a database CSN, or the Now default.
Classic and Integrated
Trail Name
A two character trail name.
Classic and Integrated
Trail Subdirectory, Size, Sequence, and Offset
You can further configure the trail details.
Classic and Integrated
Remote
Set if the trail is not on the same server.
Classic and Integrated
Thread Number
Set to a specific redo log number. The default is 1.
Classic
Encryption Profile
Provide the name of the encryption profile for the Extract. If no encryption profile is created, then the default encryption profile is selected, by default
Classic, Integrated, and Initial Load.
Encryption Profile Type
Provide the type of Key Management Service being used. Oracle Key Vault is selected by default.
Classic, Integrated, and Initial Load.
Managed Options
X
X
Profile Name
Provides the name of the autostart and autorestart profile. You can select the default or custom options.
Classic, Integrated, and Initial Load.
Critical to deployment health
Enable this option if the profile is critical for the deployment health.
Classic, Integrated, and Initial Load.
Auto Start
Enables autostart for the process.
Enables autostart for the process.
Max Retries
Specify the maximum number of retries to try to start the process
Classic, Integrated, and Initial Load.
Retry Delay
Delay time in trying to start the process
Classic, Integrated, and Initial Load.
Retries Window The duration interval to try to start the process Classic, Integrated, and Initial Load.
Restart on Failure only If true the task is only restarted if it failes Classic, Integrated, and Initial Load.
Disable Task After Retries Exhausted If true then the task is disabled after exhausting all attempts to restart the process. Classic, Integrated, and Initial Load.
How to Access Extract and Replicat Log Information
The diagnosis of Extract and Replicat transactions provides information about the severity of a transaction along with the timestamp. This information is helpful in case you need to determine if and when a particular issue occurred including the cause of the issue.
The Extract and Replicat log information is available on the Diagnosis page of Administration Server. To access the Diagnosis page, click the left navigation page of the Administration Server and select Diagnosis.
Using the Table
An updated log of Extract and Replicat server messages is displayed. You can sort the list by date or severity by clicking on the adjacent arrow. Also, you can refresh this log and choose how many pages you want to view.
To search, you select Date, Severity, or Message, and then select the appropriate options to construct your search.
Notice the Notifications tab at the bottom of the page. It displays server messages, which are not updated in the log due to transaction errors. For example, failure to log in to the database using the database credentials.
How to Create Users from the Administration Server
Oracle GoldenGate Microservicesusers can be created from the Administration Server, once you log in using the credentials created at the time of configuring the deployment.
-
Click Administrator from the left navigation pane of the Administration Server.
-
Click + to add a user.
-
Enter the required credentials in the fields.
-
Make sure that you select a role from the Role drop-down list. The available roles are: Administrator, Security, User, and Operator.
-
Click Submit.
The new user is listed in the Users table including the role and information that you supplied.