5 Oracle GoldenGate Classic for Big Data

• Deploying Oracle GoldenGate Classic Architecture for Big Data and Cloud Platforms
  
• Finding Oracle GoldenGate for Big Data within the Oracle Cloud Marketplace
  
• Launching Oracle GoldenGate Classic for Big Data
  
• Getting Started with Oracle GoldenGate Classic for Big Data Compute Node
  
• Working with Oracle GoldenGate Classic for Big Data
  
• Connecting to Data Resources
  

5.1 Deploying Oracle GoldenGate Classic Architecture for Big Data and Cloud Platforms

Using Oracle GoldenGate Classic Architecture on Marketplace, you can deploy and manage your deployment from a single location.

Here are the prerequisites which are required before deployment:

• Oracle Cloud Account
• Access to compute node resources within Oracle Cloud Tenant

5.2 Finding Oracle GoldenGate for Big Data within the Oracle Cloud Marketplace

The following steps help you to deploy Oracle GoldenGate for Big Data from Oracle Cloud Marketplace:

1. Log in to Oracle Cloud Marketplace.
2. From the Oracle Cloud Marketplace home page, use the search box under Applications and search for the keyword GoldenGate.
3. From the Search Results, select Oracle GoldenGate for Big Data.

5.3 Launching Oracle GoldenGate Classic for Big Data

The following steps help you to launch Oracle GoldenGate Classic for Big Data on Oracle Cloud Marketplace:

1. From the application page, select Get App.
2. Select OCI Region or Log in using your OCI account login credentials.
   1. OCI Region – Select the OCI Region for which you want to install the compute node.
   2. Click Create Stack.
3. In the Sign In to the Cloud Tenant page, provide the OCI tenant details.
   You may be asked to sign in to your identity provider.
4. On the Oracle GoldenGate for Big Data page, find the selection box and provide the following information:
   1. Select Version – It provides a list of versions that are available in the listing. It is set to Oracle GoldenGate for Big Data by default.
   2. Select Compartment – Specifies the compartment where the compute node will be built. It is generally the location that you have access to build the compute node.
   3. Terms of Use – This check box is selected by default. Oracle recommends to review the licenses before proceeding with the instance creation.
   4. Launch Stack – It launches the stack in the OCI environment.
5. Fill in the required Stack Information, which includes name of the stack, description and tags. The details of the compartment in which you have to create the instance is pre-populated and it is not editable. After filling in the required details, click Next.
6. Fill in the required details to Configure Variables. This information is required to build the compute node for Oracle GoldenGate for Big Data:
   1. Name for New Resources -
      • Display Name - Display Name used to identify all new OCI resources.
      • Host DNS Name – Domain Name Service for the new compute node.
   2. Network Settings -
      • Create New Network – Select this check box, if you wish to create a new network resource.

        If you select this check box, the Create New Network wizard appears allowing you to add and edit the new network information such as:

        • Network Compartment
        • New VCN DNS Name
        • New VCN CIDR
        • New Subnet DNS Name
        • New Subnet CIDR

        If you do not select this check box, the Create New Network wizard does not appear and the compute node is created with the existing network options in the VCN.

      • Network Compartment (optional) – Compartment for new or existing network resources.
      • VCN (optional) – Existing VCN to use for the newly created instance if you are not creating a new network.
      • Subnet (optional) – Existing subnet to use for the newly created instance if you are not creating a new network. The subnet that you have selected must match the same Availability Domain set in the Instance Settings.
   3. Instance Settings -
      • Availability Domain – It specifies the availability domain for the newly created Oracle GoldenGate Instance. It must match the Subnet that you have selected in the Use Existing Network settings.
      • Compute Shape – Shape of new compute instance. Supported shapes are VM.Standard2.4, VM.Standard2.8, VM.Standard2.16, VM.Standard2.24
      • Assign Public IP – Indicates if the new VM should have a public IP address.
      • Custom Volume Sizes – Use this checkbox to customize the size of new block storage volumes.
   4. SSH Public Key - Public Key for allowing SSH access as the ‘opc’ user.
7. Click Next.
8. On the Review page, review the information you provided then click Create.
9. After clicking Create, you will be taken to the Stacks page. You can monitor the creation of the compute node using this page.
10. Upon completion, you can view the Oracle GoldenGate for Big Data compute node under Compute -> Instances.

5.4 Getting Started with Oracle GoldenGate Classic for Big Data Compute Node

After deploying Oracle GoldenGate Classic for Big Data, log in to the compute node and access Oracle GoldenGate Software Command Interface (GGSCI). For this, you need to identify the public IP address of the compute node where Oracle GoldenGate Classic for Big Data is running. The following steps help you to identify the public IP address:

• Log in to your Oracle Cloud Console.
• Navigate to Compute -> Instances.
• Select the hyperlink name of the compute node. The public IP address is listed under Primary VNIC Information.
• To access the compute node where Oracle GoldenGate is running, connect as the opc account using SSH. For more information on how to access a node using the opc account refer to Connecting to an Instance.

A public SSH key is specified as a part of the deployment process and you can use the private key when you are connecting to the Oracle GoldenGate instance. The following example illustrates how you connect to the Oracle GoldenGate compute node:

ssh -i <private-key-filename> opc@<public-id-address>

• Installing Third Party Drivers or Libraries
  
• Running GGSCI
  

5.4.1 Installing Third Party Drivers or Libraries

Oracle GoldenGate Classic for Big Data compute node has Oracle Java run time already installed and pre-configured. However, the Oracle GoldenGate Classic for Big Data compute node does not have the required third party drivers or libraries. You have to install these drivers or libraries before using Oracle GoldenGate for Big Data. You can identify the third party libraries based on the sources or targets that you wish to connect.

For example, before creating a kafka connection, you need to download all the dependent libraries from Kafka Maven repository depending on the Kafka version. The list of dependent libraries for each version of Kafka is listed in the Kafka section of Using Oracle GoldenGate for Big data guide.

For more information on dependent libraries, refer to Using Oracle GoldenGate for Big Data guide.

5.4.2 Running GGSCI

Oracle GoldenGate Classic for Big Data allows you to quickly access the GoldenGate Service Command Interface (GGSCI). After logging in to the compute node, you can find GGCSI from home -> bigdata directory (/home/opc/bigdata). From here, you can execute GGSCI.

To start GGSCI, excute the following commands:

$ cd bigdata
$ export LD_LIBRARY_PATH=/home/opc/java/jre/lib/amd64/server
$ ./ggsci

5.5 Working with Oracle GoldenGate Classic for Big Data

After deploying Oracle GoldenGate Classic for Big Data on Oracle Cloud Marketplace, you can access the latest release of Oracle GoldenGate Classic for Big Data.

Before you can start using Oracle GoldenGate Classic for Big Data, there are a few tasks that you must perform to ensure that your environment is complete and ready to replicate your data. Before you begin data replication, you must perform the following tasks:

Topics:

• Configuring Source and Target Data System for Replication
• Establishing Connectivity
• Configuring Capture Support
• Configuring Apply Support

• Configuring Source and Target Data System for Replication
  
• Establishing Connectivity
  
• Configuring Capture Support
  
• Configuring Apply Support
  

5.5.1 Configuring Source and Target Data System for Replication

Before you can begin replicating data, you should prepare the source or target database to support Oracle GoldenGate. To prepare your database, follow the steps listed in Installing Oracle GoldenGate for Big Data in Installing and Upgrading Oracle GoldenGate for Big Data guide. See Using the Generic Build of Oracle GoldenGate to capture and/or apply data to Big Data and Cloud systems.

5.5.2 Establishing Connectivity

The Oracle GoldenGate Classic for Big Data on Oracle Cloud Marketplace compute node acts as a hub where you can manage your connections to source and/or target Big Data systems. To do this, establish a network connection between OCI compute node and your source and/or target Big Data systems. For this, you have to install the required third party connection libraries on the OCI compute node.

For more details on this, refer to Installing Oracle GoldenGate for Big Data chapter of Installing and Upgrading Oracle GoldenGate for Big Data guide.

5.5.3 Configuring Capture Support

Before you can begin replication, you have to setup the capture process. The capture process is also known as Extract.

Before installing Remote Extract for Cassandra source, refer to Setting Up Cassandra Change Data Capture section in Using Oracle GoldenGate for Big Data Guide, to know about the steps that you need to perform on the Cassandra platform for the replication to happen. For setting up JMS remote capture, refer to Configuring Message Capture section in Administering Oracle GoldenGate for Big Data guide.

To build the remote extract, perform the following steps within Oracle GoldenGate for Non-Oracle:

1. Log in to the Oracle GoldenGate Compute Node.

   $ ssh -i <private key> opc@<public_ip_address>

2. Start GGSCI.

   $ cd bigdata
   $ ggsci

3. Connect to the source database using the credential store.

   ggsci > alter credentialstore add user <user_id>@<tnsnames_string> password ******** alias <alias_name> domain OracleGoldenGate

4. Add an Extract.
   • Remote Extract

     ggsci > add EXTRACT <extract_name>, VAM, begin now
     ggsci > add EXTTRAIL ./dirdat/id, EXTRACT <extract_name>, megabytes 100

Note:

The extract_name parameter can be replaced with any process name that can be a maximum of 8 characters in length. The trail identifier (id) can be a maximum of two characters.

5.5.4 Configuring Apply Support

The apply process for replication, also known as Replicat, is very easy and simple to configure. To build a Replicat, perform the following steps with Oracle GoldenGate for Big Data.

1. Log in to the Oracle GoldenGate Compute Node.

   $ ssh -i <private key> opc@<public_ip_address>

2. Start GGSCI.

   $ cd bigdata
   $ export LD_LIBRARY_PATH=/home/opc/java/jre/lib/amd64/server
   $ ./ggsci

3. Add a Replicat.

   ggsci > add REPLICAT <replicat_name>, EXTTRAIL ./dirdat/aa

Note:

• You can replace replicat_name and trail name characters specified above with any valid name.
• Process names can be a maximum of 8 characters and trail names can be two characters in length.

5.6 Connecting to Data Resources

• Connecting to Big Data Systems
  

5.6.1 Connecting to Big Data Systems

You can use Oracle GoldenGate Classic for Big Data on Oracle Cloud Marketplace to remotely capture and apply data to Big Data Database on-premises or across Oracle Cloud platforms. This allows you to enable replication between cloud services in a scalable fashion and centrally manage the replication processes between cloud services. To know more about the supported Big Data systems and their versions, refer to GoldenGate for Big Data 19.1 Certification Matrix.

Use Cases for Replication

Use Oracle GoldenGate Classic for Big Data to replicate data between data resources in the following use cases:

• Data Migrations
• Data Distribution
• Real-Time Data Warehousing
• Operational Reporting
• Building Data Lakes
• Streaming Analytics

Prerequisites

Ensure that the following are set up before you begin replication:

• Oracle GoldenGate Classic for Big Data
• Any Sources and Targets that are supported by GoldenGate for BigData

To move data from on-premises to the cloud or from on-premises to on-premises, perform the tasks listed in the following sections:

• Configuring Big Data Systems for Replication
• Configuring Oracle GoldenGate Classic for Big Data Compute Node

• Configuring Big Data Systems for Replication
  
• Configuring Oracle GoldenGate Classic for Big Data Compute Node
  

5.6.1.1 Configuring Big Data Systems for Replication

To prepare your Big Data systems for replication, perform the tasks that are outlined in Installing Oracle GoldenGate for Big Data chapter of Installing and Upgrading Oracle GoldenGate for Big Data guide.

If your sources or targets are other data sources and to know about the various data sources supported by Oracle Golden Gate, refer to Using Oracle GoldenGate for Oracle Database and Using Oracle GoldenGate for Heterogeneous Databases guides.

5.6.1.2 Configuring Oracle GoldenGate Classic for Big Data Compute Node

To establish connection from Oracle GoldenGate Classic for Big Data compute node to your on-premises data systems or third party cloud systems, you must establish connectivity between OCI and on-premises data systems. To do this, you can use the following options:

• IPSec VPN
• FastConnect

For establishing an IPSec VPN with OCI, refer to IPSec VPN documentation. For establishing a FastConnect connection, refer to the FastConnect documentation.