1. Using GoldenGate Stream Analytics on Cloud Marketplace
  2. Customizing Configurations

2 Customizing Configurations

This chapter describes how to configure an external metadata store or a spark cluster.

For a list of other supported system configurations, see the latest certification matrix.

2.1 Configuring an external Metadata Store

You can use an Oracle database, an autonomous database, or MySQL as the external metadata store.

2.1.1 Customer Managed Oracle Database

A regular Oracle database can be used as a metadata store by editing the file jetty-osa-datasource.xml, in /u01/app/osa/osa-base/etc folder.

Note:

After configuring the OSA to a new metadata store, execute the below sql to the same user/schema.

UPDATE osa_system_property SET value="true" where mkey="osa.oci.instance".

Follow the steps below to use Oracle database as a metadata store:

  1. Stop GGSA by running the command sudo systemctl stop osa.
  2. Comment the MySQL section in jetty-osa-datasource.xml file as shown below.
    <?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<!--
 <New id="osads" class="org.eclipse.jetty.plus.jndi.Resource">
        <Arg>
            <Ref refid="wac"/>
        </Arg>
        <Arg>jdbc/OSADataSource</Arg>
        <Arg>
            <New class="com.mysql.cj.jdbc.MysqlConnectionPoolDataSource">
                <Set name="URL">jdbc:mysql://localhost:3306/OSADB</Set>
                <Set name="User">osa</Set>
                <Set name="Password">
                    <Call class="org.eclipse.jetty.util.security.Password" name="deobfuscate">
                        <Arg>OBF:{OBFUSCATED_PASSWORD}</Arg>
                    </Call>
                </Set>
                <Set name="maxAllowedPacket">209715200</Set>
                <Set name="allowPublicKeyRetrieval">true</Set>
            </New>
        </Arg>
    </New>
-->
</Configure>
  3. Add the following section for Oracle database right below the commented section. 
    <New id="osads" class="org.eclipse.jetty.plus.jndi.Resource">
         <Arg>
             <Ref refid="wac"/>
         </Arg>
         <Arg>jdbc/OSADataSource</Arg>
         <Arg>
             <New class="oracle.jdbc.pool.OracleDataSource">
                 <Set name="URL">jdbc:oracle:thin:@myhost.example.com:1521:XE</Set>
                 <Set name="User">osa</Set>
                 <Set name="Password">
                    <Call class="org.eclipse.jetty.util.security.Password" name="deobfuscate">
                        <Arg>OBF:{OBFUSCATED_PASSWORD}}</Arg>
                    </Call>
                </Set>
                <Set name="connectionCachingEnabled">true</Set>
                <Set name="connectionCacheProperties">
                    <New class="java.util.Properties">
                        <Call name="setProperty"><Arg>MinLimit</Arg><Arg>1</Arg></Call>
                        <Call name="setProperty"><Arg>MaxLimit</Arg><Arg>15</Arg></Call>
                        <Call name="setProperty"><Arg>InitialLimit</Arg><Arg>1</Arg></Call>
                    </New>
                </Set>
             </New>
         </Arg>
</New>
  4. Decide on an OSA schema username and a plain-text password. For example, osa as schema user name and alphago as password.

    Change directory to /u01/app/osa/lib and execute the following command:

    java -cp ./jetty-util-9.4.17.v20190418.jar org.eclipse.jetty.util.security.Password osa <your password>

    For example, java -cp ./jetty-util-9.4.17.v20190418.jar org.eclipse.jetty.util.security.Password osa alphago

    You should see results like below on console:

    2019-06-18 14:14:45.114:INFO::main: Logging initialized @1168ms to org.eclipse.jetty.util.log.StdErrLogalphago

    OBF:obfuscated password

    MD5:34d0a556209df571d311b3f41c8200f3

    CRYPT:osX/8jafUvLwA

    Note down the obfuscated password string that is displayed (shown in bold), by copying it to clipboard or notepad.

  5. Change database host, port, SID, osa schema user name and osa schema password fields marked in bold in the code in Step 3.
  6. The next step is to initialize the metadata store and set the password for user osaadmin.

    Note:

    You will need database administrator credentials with sysdba privileges to perform this step.
    1. Change directory to /u01/app/osa/osa-base/bin
    2. Execute ./start-osa.sh dbroot=<db sys user> dbroot_password=<db sys password> osaadmin_password=${OSA_ADMIN_PASSWORD}
    3. Execute ./stop-osa.sh
    4. Execute sudo systemctl start osa, to start GGSA as a system service

2.1.2 Autonomous Database

If you are using an Autonomous Database as a Metadata store, the Oracle Stream Analytics platform reads and writes all its meta-data to an Autonomous Database that is fully managed by Oracle.

  1. Stop GGSA by running the command sudo systemctl stop osa
  2. Comment the MySQL section in jetty-osa-datasource.xml file as shown below. 
    <?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<!--
 <New id="osads" class="org.eclipse.jetty.plus.jndi.Resource">
        <Arg>
            <Ref refid="wac"/>
        </Arg>
        <Arg>jdbc/OSADataSource</Arg>
        <Arg>
            <New class="com.mysql.cj.jdbc.MysqlConnectionPoolDataSource">
                <Set name="URL">jdbc:mysql://localhost:3306/OSADB</Set>
                <Set name="User">osa</Set>
                <Set name="Password">
                    <Call class="org.eclipse.jetty.util.security.Password" name="deobfuscate">
                        <Arg>OBF:{OBFUSCATED_PASSWORD}</Arg>
                    </Call>
                </Set>
                <Set name="maxAllowedPacket">209715200</Set>
                <Set name="allowPublicKeyRetrieval">true</Set>
            </New>
        </Arg>
    </New>
-->
</Configure>
  3. Add the following section for Oracle database right below the commented section. 
    <New id="osads" class="org.eclipse.jetty.plus.jndi.Resource">
         <Arg>
             <Ref refid="wac"/>
         </Arg>
         <Arg>jdbc/OSADataSource</Arg>
         <Arg>
             <New class="oracle.jdbc.pool.OracleDataSource" type="adw">
                 <Set name="URL">jdbc:oracle:thin:@{service_name}?TNS_ADMIN={wallet_absolute_path}</Set>
                 <Set name="User">osa</Set>
                 <Set name="Password">
                    <Call class="org.eclipse.jetty.util.security.Password" name="deobfuscate">
                        <Arg>OBF:{obfuscated_password}</Arg>
                    </Call>
                </Set>
                <Set name="connectionCachingEnabled">true</Set>
                <Set name="connectionCacheProperties">
                    <New class="java.util.Properties">
                        <Call name="setProperty"><Arg>MinLimit</Arg><Arg>1</Arg></Call>
                        <Call name="setProperty"><Arg>MaxLimit</Arg><Arg>15</Arg></Call>
                        <Call name="setProperty"><Arg>InitialLimit</Arg><Arg>1</Arg></Call>
                    </New>
                </Set>
             </New>
         </Arg>
     </New>
  4. Replace the following variables in above XML with actual values:
    • service_name – one of the service names listed in tnsnames.ora file inside the wallet
    • wallet_absolute_path – the absolute path of wallet folder on the machine where osa is installed
    • obfuscated_password – the obfuscated password for user ‘osa’. See the previous section on how to generate an obfuscated password
  5. The next step is to initialize the metadata store and set the password for user osaadmin.

    Note:

    You will need database administrator credentials with sysdba privileges to perform this step.
    1. Change directory to /u01/app/osa/osa-base/bin
    2. Execute ./start-osa.sh dbroot=<db sys user> dbroot_password=<db sys password> osaadmin_password=${OSA_ADMIN_PASSWORD}
    3. Execute ./stop-osa.sh
    4. Execute sudo systemctl start osa, to start GGSA as a system service

2.2 Configuring an external Spark cluster

Ensure network connectivity to hosts when configuring an external Spark cluster.

Spark on Oracle Big Data Service

You can configure GoldenGate Stream Analytics to use an external Spark cluster running in Oracle Big Data Service.

For configuration steps refer to Setting up Runtime for GoldenGate Stream Analytics Server in GoldenGate Stream Analytics Install Guide for Hadoop 2.7 and Higher.

2.3 Configuring an External OCI Streaming Service

Ensure network connectivity to hosts when configuring an external OCI Streaming Service.

You can configure GoldenGate Stream Analytics to use an external OCI Streaming Service.

For configuration steps refer to Configuring OCI Streaming Service in Using Oracle Stream Analytics.