1. User's Guide
  2. Appendices

Appendices

Using mtactl to Manage the MTA extproc

The multithreaded agent control utility (mtactl) enables Oracle Big Data SQL users to start, stop, and configure the MTA (Multi-Threaded Agent) extproc in both Oracle Clusterware Ready Service (CRS) and non-CRS Oracle Database environments.

Note:

In non-CRS environments, customers must run mtactl in order to start the MTA extproc.

Usage

In this usage description, mta_sid is the SID that a given multithreaded extproc agent services. 

mtactl {start|restart|stop|status|delete|show|bdsql} <mta_sid>
mtactl unset <parameter> <mta_sid>
mtactl set <parameter> <parameter_value> <mta_sid>
mtactl -help
mtactl <command> -help

Multithreaded Agent Control Utility Commands

Table -7 mtactl Commands

Command Full Syntax Description
start mtactl start <mta_sid> Start an MTA extproc for this SID, with the existing init parameter values stored in the repository. Use the default values if the repository does not exist.

Note:

If you used Oracle Big Data SQL 3.1, be aware that the behavior of restart and start are now reversed from what they were in 3.1 – start now uses init values from the repository if available. restart always uses the default values.
restart mtactl start <mta_sid> Clean up the repository and restart the MTA extproc agent for the SID, with the default values.
stop mtactl stop <mta_sid> Stop the MTA extproc agent that services the given SID.
status mtactl status <mta_sid> Display status for the MTA extproc that services the given SID.
delete mtactl delete <mta_sid> Clean up the repository for the given SID.
show mtactl show <mta_sid> Display the init parameters for the MTA extproc that services the given SID.
bdsql mtactl bdsql <mta_sid> Display additional operations. These are for setting up the MTA extproc for use with Oracle Big Data SQL.
set mtactl set <init parameter> <value> <mta_sid> Set the init parameters for the MTA extproc that services the given SID. Supported parameters are:
max_dispatchers
tcp_dispatchers
max_task_threads
max_sessions
listener_address
unset mtactl unset <init parameter> <mta_sid> Unset the init parameters in the repository for the MTA extproc that services the given SID.

Examples

$ mtactl start BDSQL_hadoop_cl_1 //note: using existing or default init parameter values

$ mtactl delete BDSQL_hadoop_cl_1
$ mtactl set max_sessions 200 BDSQL_hadoop_cl_1
$ mtactl set max_dispatchers 5 BDSQL_hadoop_cl_1
$ mtactl set max_task_threads 5 BDSQL_hadoop_cl_1
$ mtactl set listener_address "(ADDRESS=(PROTOCOL=ipc)(KEY=crs))" BDSQL_hadoop_cl_1
$ mtactl start BDSQL_hadoop_cl_1 (note: use customized parameter values)

$ mtactl restart BDSQL_hadoop_cl_1 //note: using default init parameter values

Change History for Previous Releases

The following are changes in previous versions of the product.

Changes in Oracle Big Data SQL 3.1

The following are changes in Oracle Big Data SQL Release 3.1.

Expanded Deployments

Release 3.1 broadens support for Oracle Big Data SQL connectivity between Oracle Engineered Systems and commodity servers.

In earlier Oracle Big Data SQL releases, the following Oracle Database/Hadoop connections are possible:

  • Oracle Exadata Database Machine to Oracle Big Data Appliance.

  • Oracle Database on commodity servers to commodity Hadoop systems.

As of Release 3.1, Oracle Big Data SQL supports all of the following Oracle Database/Hadoop system connections:

  • Oracle Database on commodity servers to Oracle Big Data Appliance.

  • Oracle Database on commodity servers to commodity Hadoop systems.

  • Oracle Exadata Database Machine to Oracle Big Data Appliance.

  • Oracle Exadata Database Machine to commodity Hadoop systems.

The phrase “Oracle Database on commodity servers” refers to non-Exadata Linux systems that are officially-supported as Oracle Database platforms. “Commodity Hadoop systems” refers to Hortonworks HDP systems or to Cloudera CDH-based systems other than Oracle Big Data Appliance. In all cases, Oracle Database servers and Hadoop systems must meet the prerequisites identified in the Oracle Big Data SQL Master Compatibility Matrix (Doc ID 2119369.1 in My Oracle Support).

Oracle SPARC SuperCluster Support

Release 3.1 provides support for Oracle SPARC SuperCluster, with certain limitations:

  • Ethernet connections between BDA (and commodity Hadoop systems) and SPARC SuperCluster are not supported.

  • Oracle Database Tablepaces in HDFS (with Smart Scan technology) is not supported on this platform.

Ethernet Option for Connections to the Exadata Database Machine

The preferred method of connecting Oracle Big Data Appliance and Oracle Exadata Database for any purpose is through InfiniBand. Previous releases of Oracle Big Data SQL have required InfiniBand for these connections. In Release 3.1, Ethernet networking between the Exadata Database Machine and Oracle Big Data Appliance is now supported. This enables you to use Oracle Big Data SQL with these two Engineered Systems in environments where InfiniBand is not feasible, such as when the two systems are geographically distant from each other.

Release 3.1 also enables Ethernet connections between commodity Hadoop systems and the Oracle Exadata Database Machine.

Oracle Big Data SQL connectivity between commodity Hadoop systems and commodity Oracle Database servers has been Ethernet-based throughout previous releases.

Note that Ethernet connections between Oracle Big Data Appliance (or commodity Hadoop systems) and Oracle SPARC SuperCluster are not supported at this time.

Simplified Deployment on Oracle Database Servers – Oracle Grid Infrastructure is now Optional

In previous releases of Oracle Big Data SQL, Oracle Grid Infrastructure is a prerequisite of the installation for all Oracle Database servers, including standalone servers that are not part of an Oracle RAC system. In Release 3.1, you have the option to install Oracle Big Data SQL on servers where Oracle Grid Infrastructure is not present. Note that in these cases, the installer makes some configuration file changes that require a restart of Oracle Database.

Unified Platform Support in the Oracle Big Data SQL Installer

Previous Oracle Big Data SQL releases included two separate installation procedures – one for Oracle Engineered Systems and another for commodity servers. In Release 3.1, you use the same installation process for both Oracle and non-Oracle platforms. This is also true for maintenance. For all of the supported Hadoop/Oracle Database combinations there is a uniform set of steps to update the Oracle Big Data SQL configuration when there are changes to the Hadoop cluster or Oracle Database server.

New Features to Simplify ILM – Oracle Database Tablespaces in HDFS (With Smart Scan Technology)

Oracle Database ILM (Information Lifecycle Management) can now be extended to use Hadoop to store read-only Oracle Database tablespaces. When you move tablespaces from Oracle Database to HDFS, the tables, partitions, and data retain their original Oracle Database internal format, remain accessible to queries, and support the full range of Oracle Database performance optimizations and security features, including the following:

  • Smart Scan for HDFS, which enables off-load of query processing to Oracle Big Data SQL on the Hadoop cluster. Smart Scan also provides filtering of query results in Hadoop prior to the return of the data to Oracle Database. In most circumstances, this can be a significant performance optimization. Indexing, Hybrid Columnar Compression, Partition Pruning, and Oracle Database In-Memory are also supported.

  • Oracle Advanced Security Option (ASO) Transparent Encryption and Data Redaction.

Tablespaces stored in HDFS are read-only, therefore this storage is best suited to data archiving.

See Section 3.2 in the Oracle Big Data SQL User’s Guide for details.

Enhancements in Oracle Shell for Hadoop Loaders 1.2

Oracle Shell for Hadoop Loaders (OHSH) is an intuitive command line tool for data migration. You can set up resources to connect to Hive, HDFS, or Oracle Database, and then access each of these data sources through the uniform OHSH interface. OHSH is one of the ways to use Copy to Hadoop. Copy to Hadoop users can download OHSH from OTN.

OHSH 1.2 includes the following changes:

  • On-disk logging of load operations in the $HOME/.ohsh shadow directory.

  • The ability to minimize output when doing load commands. (See the help command for set outputlevel.)

  • Loading Hive tables from Oracle tables not living in the oracle user's schema.

  • Wallet and TNS usage by OHSH relies on the setting of environmental variables WALLET_LOCATION and TNS_ADMIN. The set tnsadmin and set walletlocation commands are no longer supported.

In addition, you no longer set HIVE0_URL to the fully-qualified URL of remote HiveServer2 in order to create a %hive0 resource. In OHSH 1.2, set the environmental variable HS2_HOST_PORT in bin/ohsh, which is the <hostname>:<port> pair of HiveServer2.

Enhancements to Copy To Hadoop

The new method directcopy is added to Copy to Hadoop.

This is a direct, single-step method of copying data from Oracle Database to HDFS. See Using Copy to Hadoop to do Direct Copies in the Oracle Big Data SQL User’s Guide for more information.

Granting Access – Users Now Require the BDSQL_USER Role

In prior Oracle Big Data SQL releases, all users are granted Big Data SQL access implicitly. Release 3.1 introduces the BDSQL_USER role. Users requiring Oracle Big Data SQL access must be granted this role explicitly.

You must also now grant read privileges on the BigDataSQL configuration directory object.

For example, to grant access to user1: 

SQL> grant BDSQL_USER to user1; 
SQL> grant read on directory ORACLE_BIGDATA_CONFIG to user1;

Installation Instructions Moved to Oracle Big Data SQL Installation Guide

The Oracle Big Data SQL Installation Guide provides instructions how to install and uninstall the software. In releases prior to Oracle Big Data Appliance 3.1, installation instructions are in the user’s guide.

Changes in Oracle Big Data SQL 3.0.1

The following are changes in Oracle Big Data SQL Release 3.0.1.

In addition to continued support for Oracle Big Data Appliance and the Exadata Database Machine, Release 3.0.1 includes further improvements to Big Data SQL support for platforms other than Oracle Engineered Systems. These currently include Hortonworks Data Platform (HDP) and Cloudera’s Distribution Including Apache Hadoop (CDH ) running on non-Big Data Appliance commodity hardware where these Hadoop systems are used in conjunction with Oracle Database running on non-Exadata servers.

Support for Newer CDH and HDP Platforms

Oracle Big Data SQL 3.0.1 runs on Hortonworks Hadoop Data Platform (HDP) 2.3 – 2.4.x and Cloudera’s Distribution Including Apache Hadoop (CDH) 5.4.7 – 5.7. CDH 5.4.7 support includes both commodity CDH systems and Oracle Big Data Appliance 4.3.

Improved Installation Automation

The installation on HDP and CDH now automatically saves the YARN resource management configuration for later restoration.

The oracle user and oinstall group setup are now managed by the installation program.

Uniform Uninstall Procedure for HDP and CDH

The uninstall procedure is now the same for both HDP and CDH systems.

Other Changes for HDP and CDH

  • The default JDK is upgraded to jdk-8u66-linux-x64

  • bds-database-create-bundle.sh includes a new --jdk-tar-path parameter for specifying a JDK other than the default.

  • bds-database-install.sh includes two new parameters:

    • --cell-config-dir

    • --root-script-only

  • The installation process now creates and populates cellinit.ora if the file does not exist or is empty.

Oracle Shell for Hadoop Loaders

Oracle Shell for Hadoop Loaders is a helper shell that provides a simple-to-use command line interface to Copy to Hadoop, Oracle Loader for Hadoop, and Oracle SQL Connector for HDFS.

Support for Oracle Big Data Appliance 4.5

Oracle Big Data SQL Release 3.0.1 is a mammoth installation option in Oracle Big Data Appliance 4.5. You can also enable Oracle Big Data SQL Release 3.0.1 at any time after the mammoth installation by using the bdacli utililty.

Note:

In Release 3.0.1, Oracle Big Data SQL does not yet support connections between Oracle Big Data Appliance and non-Exadata Oracle Database servers, or between non-Big Data Appliance Hadoop systems and the Exadata Database Machine.