Changes in Oracle Big Data Appliance Release 5.1

Release 5.1 includes the following new features, software revisions, and other changes.

Cloudera 6.2.1 Support

Oracle Big Data Appliance 5.1 is based on Cloudera Enterprise 6.2.1. See the Cloudera web site for details about release 6.2.1.

Supported Upgrades

You can upgraded to Oracle Big Data Appliance Release 5.1 directly from Releases 4.12, 14.13, 4.14. There are important prerequisites to fulfill before you can do the upgrade. See Upgrading the Software on Oracle Big Data Appliance in the Oracle Big Data Appliance Owner's Guide for the prerequisites, the upgrade procedure, and some known issues.

Upgrade is supported for existing Oracle Linux 6 clusters to Big Data Appliance 5.1, but there is no option to create new Oracle Linux 6 clusters.

Support for Migration from Oracle Linux 6 to Oracle Linux 7

Oracle Big Data Appliance 5.1 includes support for migrating cluster nodes from Oracle Linux 6 to Oracle Linux 7.

See:

Migrating from Oracle Linux 6 to Oracle Linux 7 in the Oracle Big Data Appliance Owner's Guide.

New Mammoth Options for Upgrade and Cluster Extension

Mammoth is the utility that you run in order to perform full installations of the Big Data Appliance software on a new rack as well as upgrades, cluster extensions, and patch installation. The -s (single step) and -r (range of steps) Mammoth options that were previously available only for full installation are now available for both upgrades and cluster extensions.

This example uses -s to run only Step 1 of the upgrade. Step 1 is a set of pre-checks and pre-configuration operations.
```
# ./mammoth -s 1 -p
```
This example uses -r to run a range of the cluster extension steps. It runs all steps up to Step 4 (PrepareBaseImage) on each of the new nodes, but does not proceed further.
```
# ./mammoth -r 1-4 -e  node13 node14 node15
```

Uniform Service Layout for Multirack Clusters

In earlier releases, there are difference in the distribution of services among the nodes of a multirack cluster, depending upon whether the cluster was originally configured as multirack or was a single-rack cluster extended to additional racks. In both cases, prior to Release 5.1, the services layout in the first rack is different from that of a single rack cluster.

In Oracle Big Data Appliace 5.1 the distribution of services in a single-rack cluster and in the first rack of a multirack cluster are the same. The services on the second and subsequent racks are now consistent regardless of whether the cluster started out as a single-rack or multirack cluster.

X8-2L Servers

Big Data Appliance 5.1 supports X8-2L servers as well as earlier server models supported by previous releases. Important differences between X8-2L and X7-2L in Oracle Big Data Appliance are in processing power and storage:

X8-2L – 2.4 GHz Intel Xeon 8260 CPUs, 14 TB HDDs, and 240 GB Intel M.2 SSDs
X7-2L - 2.1 GHz Xeon 8160 CPUs, 10 TB HDDs, 150 GB Intel M.2 SSDs

The same 32 GB DDR4 - 2666 MHz Memory is used in both Big Data Appliance X8-2L and X7-2L.

The extra 4 TB storage in X8-2L HDDs is not used by Big Data Appliance 5.1 and is available for customer use in this release.

You can integrate X8-2L servers into existing clusters consisting of X7-2L, X6-2L, and X5-2L servers. To do so, first ensure that the existing nodes of the cluster are running Big Data Appliance release 4.10 or higher.

New X8 racks are shipped with X8-2L servers with the base image pre-installed. The OS level is Oracle Linux 7.

It is possible to reimage X8-2L servers to Oracle Linux 6 in order to use them to extend an Oracle Linux 6 cluster. However, if you intend to migrate the cluster to Oracle Linux 7 in the near term, bear in mind that at this time there is no support for migrating X8-2L servers from Oracle Linux 6 to Oracle Linux 7.

X8 racks can be integrated into multirack configurations with existing X7, X6, or X5 racks.

There are no cabling changes for X8 racks.

CPU Core Capping

Core capping lets you disable or enable physical cores within the two CPUs on the appliance. One of the potential uses for this capability is to bring servers into licensing compliance. Cores can be enabled or disabled via the bdacli utility.

See Capping CPU Cores on Servers in the Oracle Big Data Appliance Owner's Guide.

Automated Configuration of Replacements for Failed Operating System Disks and Data Disks

The bdaconfiguredisk utility now enables you to configure replacement disks or reconfigure disks with no manual intervention after executing the script. The script supports both OS and data disks. See bdaconfiguredisk in the Oracle Big Data Appliance Owner's Guide.

About Big Data SQL 4.0

Oracle Big Data SQL supports queries against non-relational data stored in multiple big data sources, including Apache Hive, HDFS, Oracle NoSQL Database, Apache Kafka, Apache HBase, and other NoSQL databases.

In addition to ORACLE_HIVE and ORACLE_HDFS, Release 4.0 also includes the new ORACLE_BIGDATA driver. This driver enables you to create external tables over data within object stores in the cloud. Currently Oracle Cloud Infrastructure and Amazon S3 are supported.

Another feature introduced in Release 4.0 is Query Server. This is a lightweight, zero-maintenance Oracle Database that runs locally on an edge node within Big Data Appliance clusters. It gives you an easy way to query data in Hadoop without the need for a full Oracle Database installation. Query Server provides no persistent storage except for certain categories of metadata that are useful to retain across sessions.

Big Data SQL 4.0 is primarily intended for use with Oracle Database 18c or later. Oracle Database 12.1 and 12.2 are also fully supported (even though you can't leverage the new 4.0 capabilities with these database versions).

Big Data SQL components must be installed on both Hadoop and Oracle Database. The installation guide provides instructions for both parts of the installation, including generic instructions for installing the product on several supported Hadoop frameworks. However on Oracle Big Data Appliance, most of the Hadoop-side installation is integrated with Mammoth, the Big Data Appliance installer. Installing Oracle Big Data SQL in the Oracle Big Data Appliance Owner's Guide explains what is different about the Hadoop-side installation of Big Data SQL on Big Data Appliance.

See the Oracle Big Data SQL User's Guide for usage and reference information.

Software Versions in This Release

Cloudera Enterprise 6.2.1, including CDH, Cloudera Manager, and Key Trustee, Sentry, Impala, Cloudera Search, Apache HBase 2.0, Apache Hive 2.0, Apache Spark 2.2, Apache Kafka 2.10.)

The Cloudera parcels for Kudu, Kafka, and Key Trustee Server are included for your convenience, but are not deployed or configured by default.

Note that in Oracle Big Data Appliance 4.13, the Mammoth installer deployed Apache Spark 2 automatically. In the upgrade to Oracle Big Data Appliance 5.1, this package is removed.
Oracle Big Data SQL 4.0
Oracle Big Data Connectors 5.0
Oracle NoSQL Database Enterprise Edition 19.3.12
Oracle NoSQL Community Edition 18.1.19
MySQL 5.7.27
Oracle R Advanced Analytics for Hadoop (ORAAH) 2.8.1
Oracle's R Distribution (ORD) 3.3.0
Oracle Big Data Spatial & Graph 2.5.3

The 2.5.3 release includes several new APIs. Two new algorithms have been added for the in-memory analyst (PGX), along with several new features for PGQL support.
Java JDK 8u221
Oracle Linux 7 with UEK4 for new clusters. Oracle Linux 6 with UEK4 for Oracle Linux 6 cluster upgrades.

Note:

New Oracle Big Data Appliance racks are delivered with Oracle Linux 7.

All servers in existing clusters (not in newly delivered racks) where you are installing Oracle Big Datas Appliance 5.1 must first be updated to at least Oracle Big Data Appliance 4.10.0 before any X8-2L or X7–2L servers can be added as nodes in the cluster.

See Oracle Big Data Appliance Patch Set Master Note (Doc ID 1485745.1) in My Oracle Support for the base image download and instructions.

Software and Features Not Supported in This Release

Because of the transition to Cloudera Enterprise 6.x, the following software cannot currently be supported in Oracle Big Data Appliance Release 5.1. In some of these cases, you are required to uninstall or disable the software prior to upgrading to Release 5.1.

Oracle Big Data Discovery
If Oracle Big Discovery is installed, uninstall it prior to the upgrade. This product is not compatible with Cloudera 6.
New Kafka clusters and cluster upgrades
Creation of new Kafka clusters is temporarily de-supported in Oracle Big Data Appliance 5.1.

You do not need to uninstall existing Kafka clusters. They remain functional, but cannot be upgraded.
Rolling Upgrades
Rolling upgrades (where nodes are upgraded one-after-the-other and downtime is avoided) are generally an option in Oracle Big Data Appliance releases. However, this option is not currently available for upgrades from Oracle Big Data Applaince release 4.x to 5.1. An upgrade from 4.x to 5.1 will require some cluster downtime.
The ODI (Orace Data Integrator) Agent
There is no version of the ODI Agent that supports Oracle Big Data Appliance 5.1 at this time. The agent is not compatible with Cloudera 6. Clusters cannot be upgraded to release 5.1 if the agent is enabled. Before you upgrade, check to ensure that the existing ODI (Oracle Data Integrator) Agent is disabled:
```
# bdacli getinfo cluster_odi_enabled
```
The Oracle Big Data Appliance Configuration Utility will not allow you to select the ODI Agent for installation. If you import a pre-existing master.xml file into the Configuration Utility ensure that the ODI Agent installation is de-selected before generating new configuration files. Likewise, if you use the bdacli utility to enable Oracle Big Data Connectors, the bdacli enable bdc command will not enable the ODI Agent.