This guide describes Oracle Big Data Appliance, which is used for acquiring, organizing, and analyzing very large data sets. It includes information about hardware operations, site planning and configuration, and physical, electrical, and environmental specifications.
This preface contains the following topics:
This guide is intended for Oracle Big Data Appliance customers and those responsible for data center site planning, installation, configuration, and maintenance of Oracle Big Data Appliance.
For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.
Access to Oracle Support
Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.
Oracle Big Data Appliance Documentation Library
In the Big Data portal on the Oracle Help Center, you can find a link to the complete Oracle Big Data Appliance library for your release of the product. The library includes the following documentation for Oracle Big Data Appliance as well as documentation for other Oracle big data products:
Oracle Big Data Appliance Owner’s Guide (this guide)
Note:The Oracle Big Data Appliance Licensing Information User Manual is the consolidated reference for licensing information for Oracle and third-party software included in the Oracle Big Data Appliance product. Refer to this manual or contact Oracle Support if you have questions about licensing.
Help for Oracle Big Data Appliance in My Oracle Support
My Oracle Support (support.oracle.com) provides detailed “how to” instructions in the form of MOS notes (My Oracle Support notes) that are constantly updated. If you do not find information that you need in Oracle Big Data Appliance documentation library, check My Oracle Support. Use your Oracle account credentials to log in.
Your Oracle support representative may sometimes refer you to a MOS note for assistance with an issue.
In the search field, you can enter a specific MOS note number or any set of keywords.
Description of the illustration bigog_mos_search_field.png
The Oracle Big Data Appliance Information Center
The Oracle Big Data Appliance Information Center (Doc ID 1445762.2) is an aggregation of MOS notes on many topics, such as
The latest base image and how to use it for new installs and reprovisions.
Certified CDH upgrades.
Tips on health checks as well as diagnostic and monitoring tools.
Documentation for Affiliated Hardware Products
The following Oracle libraries contain hardware information for Oracle Big Data Appliance.
Oracle Server X7-2L Documentation Library at https://docs.oracle.com/cd/E72463_01/index.html
Oracle Server X6-2L Documentation Library at https://docs.oracle.com/cd/E62172_01/index.html
Oracle Server X5-2L Documentation Library at https://docs.oracle.com/cd/E41033_01/index.html
Sun Rack II 1042 and 1242 library at
Sun Network QDR InfiniBand Gateway Switch Firmware Version 2.0 Documentation at
Sun Datacenter InfiniBand Switch 36 Firmware Version 2.0 Documentation at
Oracle Integrated Lights Out Manager 3.1 Documentation Library Documentation Library at
The following text conventions are used in this document:
Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary.
Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values.
Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter.
The pound (
Backus-Naur Form Syntax
The syntax in this reference is presented in a simple variation of Backus-Naur Form (BNF) that uses the following symbols and conventions:
|Symbol or Convention
Brackets enclose optional items.
Braces enclose a choice of items, only one of which is required.
A vertical bar separates alternatives within brackets or braces.
Ellipses indicate that the preceding syntactic element can be repeated.
Delimiters other than brackets, braces, and vertical bars must be entered as shown.
Words appearing in boldface are keywords. They must be typed as shown. (Keywords are case-sensitive in some, but not all, operating systems.) Words that are not in boldface are placeholders for which you must substitute a name or value.
Changes in Oracle Big Data Appliance Release 5.1
Release 5.1 includes the following new features, software revisions, and other changes.
Cloudera 6.2.1 Support
Oracle Big Data Appliance 5.1 is based on Cloudera Enterprise 6.2.1. See the Cloudera web site for details about release 6.2.1.
You can upgrade to Oracle Big Data Appliance Release 5.1 directly from Releases 4.12, 14.13, and 4.14.
There are important prerequisites to fulfill before you can do the upgrade. See Upgrading the Software on Oracle Big Data Appliance in this guide for the prerequisites, the upgrade procedure, and some known issues.
Upgrade is supported for existing Oracle Linux 6 clusters to Big Data Appliance 5.1, but there is no option to create new Oracle Linux 6 clusters.
Support for Migration from Oracle Linux 6 to Oracle Linux 7
Oracle Big Data Appliance 5.1 includes support for migrating cluster nodes from Oracle Linux 6 to Oracle Linux 7.
See:Migrating from Oracle Linux 6 to Oracle Linux 7 in this guide.
Big Data Appliance 5.1 supports X8-2L servers as well as earlier server models supported by previous releases. Important differences between X8-2L and X7-2L in Oracle Big Data Appliance are in processing power and storage:
- X8-2L – 2.4 GHz Intel Xeon 8260 CPUs, 14 TB HDDs, and 240 GB Intel M.2 SSDs
- X7-2L - 2.1 GHz Xeon 8160 CPUs, 10 TB HDDs, 150 GB Intel M.2 SSDs
The same 32 GB DDR4 - 2666 MHz Memory is used in both Big Data Appliance X8-2L and X7-2L.
The extra 4 TB storage in X8-2L HDDs is not used by Big Data Appliance 5.1 and is available for customer use in this release.
You can integrate X8-2L servers into existing clusters consisting of X7-2L, X6-2L, and X5-2L servers. To do so, first ensure that the existing nodes of the cluster are running Big Data Appliance release 4.10 or higher.
New X8 racks are shipped with X8-2L servers with the base image pre-installed. The OS level is Oracle Linux 7.
It is possible to reimage X8-2L servers to Oracle Linux 6 in order to use them to extend an Oracle Linux 6 cluster. However, if you intend to migrate the cluster to Oracle Linux 7 in the near term, bear in mind that at this time there is no support for migrating X8-2L servers from Oracle Linux 6 to Oracle Linux 7.
X8 racks can be integrated into multirack configurations with existing X7, X6, or X5 racks.
There are no cabling changes for X8 racks.
New Mammoth Options for Upgrade and Cluster Extension
Mammoth is the utility that you run in order to perform full installations of the Big Data Appliance software on a new rack as well as upgrades, cluster extensions, and patch installation. The
-s (single step) and
-r (range of steps) Mammoth options that were previously available only for full installation are now available for both upgrades and cluster extensions.
- This example uses
-sto run only Step 1 of the upgrade. Step 1 is a set of pre-checks and pre-configuration operations.
# ./mammoth -s 1 -p
- This example uses -r to run a range of the cluster extension steps. It runs all steps up to Step 4 (PrepareBaseImage) on each of the new nodes, but does not proceed further.
# ./mammoth -r 1-4 -e node13 node14 node15
Uniform Service Layout for Multirack Clusters
In earlier releases, there are difference in the distribution of services among the nodes of a multirack cluster, depending upon whether the cluster was originally configured as multirack or was a single-rack cluster extended to additional racks. In both cases, prior to Release 5.1, the services layout in the first rack is different from that of a single rack cluster.
In Oracle Big Data Appliace 5.1 the distribution of services in a single-rack cluster and in the first rack of a multirack cluster are the same. The services on the second and subsequent racks are now consistent regardless of whether the cluster started out as a single-rack or multirack cluster.
CPU Core CappingCore capping lets you disable or enable physical cores within the two CPUs on the appliance. One of the potential uses for this capability is to bring servers into licensing compliance. Cores can be enabled or disabled via the bdacli utility.
Automated Configuration of Replacements for Failed Operating System Disks and Data Disks
The bdaconfiguredisk utility now enables you to configure replacement disks or reconfigure disks with no manual intervention after executing the script. The script supports both OS and data disks.
Several changes to default Spark configuration parameters have been made. These changes speed up the execution of most Spark jobs.
Improvements have been made to further enhance the security of Big Data Appliance clusters. These changes should reduce the impact of future vulnerabilities.
About Big Data SQL 4.0
Oracle Big Data SQL supports queries against non-relational data stored in multiple big data sources, including Apache Hive, HDFS, Oracle NoSQL Database, Apache Kafka, Apache HBase, and other NoSQL databases, as well as the Amazon S3 and Oracle Object Store.
In addition to ORACLE_HIVE and ORACLE_HDFS, Release 4.0 also includes the new ORACLE_BIGDATA driver. This is the driver that enables you to create external tables over data within object stores in the cloud.
Another feature introduced in Big Data SQL 4.0 is Query Server, a lightweight, zero-maintenance Oracle Database that runs locally on an edge node within Big Data Appliance clusters. It gives you an easy way to query data in Hadoop without the need for a full Oracle Database installation. Query Server provides no persistent storage except for certain categories of metadata that are useful to retain across sessions.
Big Data SQL 4.0 is primarily intended for use with Oracle Database 18c or later. Oracle Database 12.1 and 12.2 are also fully supported (although you cannot leverage the new Big Data SQL 4.0 capabilities with these database versions).
Big Data SQL components must be installed on both Hadoop and Oracle Database. The Oracle Big Data SQL Installation Guide provides instructions for both parts of the installation, including generic instructions for installing the product on several supported Hadoop frameworks. However, on Oracle Big Data Appliance, most of the Hadoop-side installation is integrated with Mammoth, the Big Data Appliance installer. Installing Oracle Big Data SQL in this guide explains what is different about the Hadoop-side installation of Big Data SQL on Big Data Appliance.
See Also:Oracle Big Data SQL 4.0 Documentation
Software Versions in This Release
Cloudera Enterprise 6.2.1, including CDH, Cloudera Manager, and Key Trustee, Sentry, Impala, Cloudera Search, Apache HBase 2.0, Apache Hive 2.0, Apache Spark 2.2, Apache Kafka 2.10.)
The Cloudera parcels for Kudu, Kafka, and Key Trustee Server are included for your convenience, but are not deployed or configured by default.
Note that in Oracle Big Data Appliance 4.13, the Mammoth installer deployed Apache Spark 2 automatically. In the upgrade to Oracle Big Data Appliance 5.1, this package is removed.
- Oracle Big Data SQL 4.0
Oracle Big Data Connectors 5.0
Oracle NoSQL Database Enterprise Edition 19.3.12
Oracle NoSQL Community Edition 18.1.19
Oracle R Advanced Analytics for Hadoop (ORAAH) 2.8.1
Oracle's R Distribution (ORD) 3.3.0
Oracle Big Data Spatial & Graph 2.5.3
The 2.5.3 release includes several new APIs. Two new algorithms have been added for the in-memory analyst (PGX), along with several new features for PGQL support.
Java JDK 8u231
Oracle Linux 7 with UEK4 for new clusters. Oracle Linux 6 with UEK4 for Oracle Linux 6 cluster upgrades.
New Oracle Big Data Appliance racks are delivered with Oracle Linux 7.
All servers in existing clusters (not in newly delivered racks) where you are installing Oracle Big Datas Appliance 5.1 must first be updated to at least Oracle Big Data Appliance 4.10.0 before any X8-2L (or X7–2L) servers can be added as nodes in the cluster.
See Oracle Big Data Appliance Patch Set Master Note (Doc ID 1485745.1) in My Oracle Support for the base image download and instructions.
Software and Features Not Supported in This Release
Because of the transition to Cloudera Enterprise 6.x, the following software and functionality cannot currently be supported in Oracle Big Data Appliance Release 5.1.
- Oracle Big Data Discovery
If Oracle Big Discovery is installed, uninstall it prior to the upgrade to Big Data Appliance 5.1. Any previously installed versions of this product are not compatible with Cloudera 6.
- The CDH 5-Compatible ODI (Orace Data Integrator) AgentThe Oracle Big Data Appliance 5.1 installation does not include a compatible version of the ODI Agent at this time. The agent carried in previous releases is not compatible with Cloudera 6. Therefore, clusters cannot be upgraded to release 5.1 if the agent is enabled. Before you upgrade, check to ensure that the existing ODI Agent is disabled:
# bdacli getinfo cluster_odi_enabled
To remove the existing ODI Agent, use bdacli to disable Oracle Big Data Connectors. Then re-enable Big Data Connectors without including the ODI Agent --
bdacli disable bdcand then
bdacli enable bdc(
The Oracle Big Data Appliance Configuration Utility will not allow you to select the ODI Agent for installation. If you import a pre-existing master.xml file into the Configuration Utility ensure that the ODI Agent installation is de-selected before generating new configuration files. Likewise, if you use the bdacli utility to enable Oracle Big Data Connectors, the
bdacli enable bdccommand will not enable the ODI Agent.
After the Mammoth installation, you can separately install the standalone ODI Agent 18.104.22.168 using the instructions in the ODI product documentation. This version of the agent is certified as compatible with Cloudera 6.
- Rolling Upgrades
Rolling upgrades (where nodes are upgraded one-after-the-other and downtime is avoided) are generally an option in Oracle Big Data Appliance releases. However, this option is not currently available for upgrades from Oracle Big Data Applaince release 4.x to 5.1. An upgrade from 4.x to 5.1 will require some cluster downtime.
- New Kafka Clusters and Cluster upgrades
Creation of new Kafka clusters is temporarily de-supported in Oracle Big Data Appliance 5.1.
You do not need to uninstall existing Kafka clusters. They remain functional, but cannot be upgraded.
Recent Change History Prior to Oracle Big Data Appliance 5.1
The Recent Change History appendix to this guide describes changes in Releases 4.14 and 4.13. If you are upgrading from an older version of Oracle Big Data Appliance, then it may be informative to read this change summary.