Preface

Changes in Oracle SQL Connectors for HDFS

• The property oracle.hadoop.exttab.dataCompressionCodec is now deprecated.

  OSCH now processes datasets containing both compressed and uncompressed files. OSCH automatically discovers the compression codec of the dataset at runtime.

• -createTable for delimited text source now supports the NULLIF clause.

  You can use the configuration property nullIfSpecifier to set NULLIF at -createTable time. For example:

  oracle.hadoop.exttab.nullIfSpecifier=<NULLIF-value> 
  

  or:

  oracle.hadoop.exttab.colMap.<columnName>.nullIfSpecifier=<NULLIF-value>
  

  Note that the column-level nullIfSpecifier overrides the external table level nullIfSpecifier.

New and Enhanced Features

• Oracle R Advanced Analytics for Hadoop (ORAAH) 2.7

  ORAAH 2.7 provides the following new features:

  • New ORAAH Spark-based LM algorithm with summary statistics.

  • Enhanced ORAAH Spark-based GLM full formula support and summary functions for the Spark-based GLM.

  • Enhanced ORAAH Spark-based Deep Neural Networks now supporting full formula parsing, and Modeling plus Scoring in Spark, with computations up to 30% faster.

  • New Oracle R API for the Spark MLlib Gaussian Mixture Models clustering algorithm.

  • General improvements to HIVE integration, especially for BDA secure clusters with enabled SSL connection and Kerberos authentication.

  • Automated Hive JDBC driver lookup for known installations, such RPM or parcel installations.

• Oracle Shell for Hadoop Loaders (OHSH) 1.2

  New features and changes in Release 1.2 include:

  • On-disk logging of load operations in the $HOME/.ohsh shadow directory.

  • The ability to minimize output when doing load commands. (See the help command for set outputlevel.)

  • Loading Hive tables from Oracle tables not living in the oracle user's schema.

  • Wallet and TNS usage by OHSH relies on the setting of environmental variables WALLET_LOCATION and TNS_ADMIN. The set tnsadmin and set walletlocation commands are no longer supported.

  In addition, you no longer set HIVE0_URL to the fully-qualified URL of remote HiveServer2 in order to create a %hive0 resource. In OHSH 1.2, set the environmental variable HS2_HOST_PORT in bin/ohsh, which is the <hostname>:<port> pair of HiveServer2.

New and Enhanced Features

• Oracle Datasource for Apache Hadoop (formerly Oracle Table Access for Apache Hadoop)

  Oracle Datasource for Apache Hadoop (OD4H) is now part of the Oracle Big Data Connectors suite and is licensed for use at no additional cost.

  OD4H turns Oracle Database tables into Hadoop data sources (i.e., external tables), enabling direct, and consistent HiveQL/SparkSQL queries, as well as direct Hadoop API access.

  OD4H optimizes queries execution plans using predicate and projection pushdown as well as partition pruning. Oracle Database table access is performed in parallel using smart and secure connections (Kerberos, SSL,  Oracle Wallet), regulated by both Hadoop (i.e., maximum concurrent tasks) and Oracle DBAs (i.e. , max pool size).

• Oracle Shell for Hadoop Loaders 1.1

  Oracle Shell for Hadoop Loaders (OHSH) was introduced recently in Oracle Big Data Data Connectors 4.5. OHSH is an intuitive command line tool for data migration. You can set up resources to connect to Hive, HDFS or Oracle Database and access each of these data sources through OHSH’s uniform interface. Copy to Hadoop users can download OHSH from OTN .

  Changes in this release:

  • Interactive command history is now persistent across OHSH sessions.

  • Support for spooling of OHSH output to a text file. By default the spool file is ohshspool.txt in the directory where OHSH is invoked. Spooling can be turned on, off, or directed to a user-specified file as follows:

    ohsh> spool on
    ohsh> spool off
    ohsh> set spool <filename>
    

  • New Hive CLI. Beeline is now the CLI for Hive resources. The syntax to create a Hive resource is now as follows.

    ohsh> create hive resource <resource_id> connectionurl=<DQString>
    

    In this case, if the user has specified the HIVE0_URL variable in bin/ohsh, the command creates a hive0 resource.

New and Enhanced Features

• Oracle Shell for Hadoop Loaders

  Oracle Shell for Hadoop Loaders is a new user interface for Big Data Connectors. Is it not itself a Big Data Connector. Oracle Shell for Hadoop Loaders is a shell and command line that provides the user with a single environment for interacting with Big Data Connectors – Oracle Loader for Hadoop, Oracle SQL Connector for HDFS, and Copy to Hadoop. In addition to providing a single point of access, Oracle Shell for Hadoop Loaders can reduce some of the overhead involved in using the Connectors, because otherwise these products must be configured, managed, and run separately.

• Oracle R Advanced Analytics for Hadoop (ORAAH) 2.6 Improvements

  ORAAH 2.6 includes expanded support for predictive modeling algorithms, including integration of many Spark MLlib capabilities, as well as enhancements for existing custom Spark algorithms.

• Oracle XQuery for Hadoop 4.5.0 Improvements

  Adds support for W3C XQuery 3.0 including the try/catch expression, the switch expression, and standard functions and operators.

New Features

New Features