Creating and Modifying Clusters
Planning a Cluster
Before you create a cluster, understand your options.
Understand Instance Types and Shapes
Big Data Service cluster nodes run in Oracle Cloud Infrastructure compute instances (servers).
When you create a cluster, you choose an instance type, which determines whether the instance will run directly on the "bare metal" of the hardware or in a virtualized environment. You also choose a shape, which configures the resources assigned to the instance.
There are two types of Oracle Cloud Infrastructure compute instances:
-
Bare Metal: A bare metal compute instance uses a dedicated physical server for the node, for highest performance and strongest isolation.
-
Virtual Machine (VM): Through virtualization, a VM compute instance can host multiple, isolated nodes that run on a single physical bare metal machine. VM instances are less expensive than bare metal instances and are useful for building less demanding clusters that don't require the performance and resources (CPU, memory, network bandwidth, storage) of an entire physical machine for each node.
VM instances run on the same hardware as bare metal instances, with the same firmware, software stack, and networking infrastructure.
For more information about compute instances, see Overview of Compute Service.
The shape determines the number of CPUs, amount of memory, and other resources allocated to the compute instance hosting the cluster node. See Supported Node Shapes in the Oracle Cloud Infrastructure documentation for the available shapes.
The shapes of the Big Data Service master nodes and the worker nodes don't have to match. But the shapes of all of the master nodes must match each other and the shapes of all the worker nodes must match each other.
Plan Your Cluster Layout, Shape, and Storage
Before you start the process to create a cluster, you must plan the layout of the cluster, the node shapes, and storage.
Nodes and services are organized differently on clusters, based on whether the cluster is highly available (HA) and secure, or not.
About Using HA Clusters
Use HA clusters for production environments. They're required for resiliency and to minimize downtime.
In this release, a cluster must be both HA and secure, or neither.
Types of Nodes
The types of nodes are as follows:
- Master or utility nodes include the services required for the operation and management of the cluster. These nodes don’t store or process data.
- Worker nodes store and process data. The loss of a worker node doesn't affect the operation of the cluster, although it can affect performance.
- Compute only worker nodes process data. The loss of a compute only worker node doesn't affect the operation of the cluster, although it can affect performance.Note
Compute only worker nodes aren’t supported for CDH clusters. - Edge nodes are extended nodes to the cluster that have only clients installed. You can install additional packages and run additional applications in this node instead of worker/compute/master nodes to avoid classpath conflicts and resources issues with cluster services.
High Availability (HA) Cluster Layout
A high availability cluster has two master nodes, two utility nodes, three or more worker nodes, and zero or more compute only worker nodes.
| Type of node | Services on ODH | Services on CDH |
|---|---|---|
| First master node |
|
|
| Second master node |
|
|
| First utility node |
|
|
| Second utility node |
|
|
| Worker nodes (3 minimum) |
|
|
| Compute only worker nodes |
|
NA |
| Edge nodes |
|
NA |
Minimal (non-HA) Cluster Layout
A non-high availability cluster has one master node, one utility node, three or more worker nodes, and zero or more compute only worker nodes.
| Type of node | Services on ODH | Services on CDH |
|---|---|---|
| Master node |
|
|
| Utility node |
|
|
| Worker nodes |
|
|
| Compute only worker nodes |
|
NA |
| Edge nodes |
|
NA |
The shape describes the compute resources allocated to the node.
The shapes used for master/utility nodes and worker nodes can be different. But all master/utility nodes must be of the same shape and all worker nodes must be of the same shape.
The following table shows what shapes can be used for the different node types. For a list of the resources provided by each shape, see Compute Shapes.
| Node Type | Available Shapes | Required Number of Virtual Network Interface Cards (VNICs) |
|---|---|---|
| Master or utility |
VM.Standard2.4 VM. Standard2.8 VM. Standard2.16 VM.Standard2.24 VM.Standard.E4.Flex * VM.Standard3.Flex* VM.Optimized3.Flex* VM.DenseIO.E4.Flex* VM.DenseIO2.8 VM.DenseIO2.16 VM.DenseIO2.24 BM.Standard2.52 BM.DenseIO2.52 BM.HPC2.36 BM.Standard3.64* BM.Optimized3.36* BM.DenseIO.E4.128* BM.Standard.E4.128* |
3 minimum Used for the cluster subnet, the DP access subnet, and the customer's subnet *You must specify a minimum of 3 OCPU and 32GB memory. Note
The following shapes are not supported for CDH clusters. They are only supported for ODH clusters. VM.Standard.E4.Flex VM.Standard3.Flex VM.Optimized3.Flex VM.DenseIO.E4.Flex BM.Standard3.64 BM.Optimized3.36 BM.DenseIO.E4.128 BM.Standard.E4.128 |
| Worker |
VM.Standard2.1* VM.Standard2.2* VM. Standard2.4 VM. Standard2.8 VM. Standard2.16 VM.Standard2.24 VM.Standard.E4.Flex * VM.Standard3.Flex* VM.Optimized3.Flex* VM.DenseIO.E4.Flex* VM.DenseIO2.8 VM.DenseIO2.16 VM.DenseIO2.24 BM.Standard2.52 BM.DenseI2.52 BM.HPC2.36 BM.Standard3.64* BM.Optimized3.36* BM.DenseIO.E4.128* BM.Standard.E4.128* |
2 minimum Used for the cluster subnet and the customer's subnet * Be aware that VM.Standard2.1 and VM.Standard2.2 are small shapes and won't support running large workloads. For VM.Standard.E4.Flex, VM.Standard3.Flex, and VM.Optimized3.Flex you must specify minimum of 1 OCPU and 16GB memory. Note
The following shapes are not supported for CDH clusters. They are only supported for ODH clusters. VM.Standard.E4.Flex VM.Standard3.Flex VM.Optimized3.Flex VM.DenseIO.E4.Flex BM.Standard3.64 BM.Optimized3.36 BM.DenseIO.E4.128 BM.Standard.E4.128 |
| Compute only worker |
VM.Standard2.1* VM.Standard2.2* VM. Standard2.4 VM. Standard2.8 VM. Standard2.16 VM.Standard2.24 VM.Standard.E4.Flex * VM.Standard3.Flex* VM.Optimized3.Flex* VM.DenseIO.E4.Flex* VM.DenseIO2.8 VM.DenseIO2.16 VM.DenseIO2.24 BM.Standard2.52 BM.DenseI2.52 BM.HPC2.36 BM.Standard3.64* BM.Optimized3.36* BM.DenseIO.E4.128* BM.Standard.E4.128* |
2 minimum Used for the cluster subnet and the customer's subnet * Be aware that VM.Standard2.1 and VM.Standard2.2 are small shapes and won't support running large workloads. For VM.Standard.E4.Flex, VM.Standard3.Flex, and VM.Optimized3.Flex you must specify minimum of 1 OCPU and 16GB memory. Compute only worker nodes are not supported for CDH clusters. Note
The following shapes are not supported for CDH clusters. They are only supported for ODH clusters. VM.Standard.E4.Flex VM.Standard3.Flex VM.Optimized3.Flex VM.DenseIO.E4.Flex BM.Standard3.64 BM.Optimized3.36 BM.DenseIO.E4.128 BM.Standard.E4.128 |
| Edge |
VM.Standard2.1* VM.Standard2.2* VM. Standard2.4 VM. Standard2.8 VM. Standard2.16 VM.Standard2.24 VM.Standard.E4.Flex * VM.Standard3.Flex* VM.Optimized3.Flex* VM.DenseIO.E4.Flex* VM.DenseIO2.8 VM.DenseIO2.16 VM.DenseIO2.24 BM.Standard2.52 BM.DenseI2.52 BM.HPC2.36 BM.Standard3.64* BM.Optimized3.36* BM.DenseIO.E4.128* BM.Standard.E4.128* |
2 minimum Used for the cluster subnet and the customer's subnet * Be aware that VM.Standard2.1 and VM.Standard2.2 are small shapes and won't support running large workloads. For VM.Standard.E4.Flex, VM.Standard3.Flex, and VM.Optimized3.Flex you must specify minimum of 1 OCPU and 16GB memory. Note
Since the Edge node is specific to client application usecases, choose shape as per application requirement. Edge nodes are not supported for CDH clusters. |
|
Cloud SQL query server |
VM. Standard2.4 VM. Standard2.8 VM. Standard2.16 VM.Standard2.24 VM.Standard.E4.Flex * VM.Standard3.Flex* VM.Optimized3.Flex* VM.DenseIO.E4.Flex* VM.DenseIO2.8 VM.DenseIO2.16 VM.DenseIO2.24 BM.HPC2.36 BM.Standard2.52 BM.Optimized3.36* |
-- |
Not all shapes are available by default. To see what shapes are available by default through the Cloud console, see Find the Limits for Your Tenancy . To submit a request to increase your service limits, see Request an Increase for Big Data Service Nodes.
Nodes based on standard VM shapes use network-attached block storage.
Block storage isn't supported for nodes based on DenseIO and HPC shapes.
All nodes have a boot volume of 150 GB.
| Option | Limits/Guidelines |
|---|---|
| Minimum initial block storage | 150 GB |
| Default initial block storage * | 150 GB |
| Minimum additional block storage | 150 GB |
| Default additional block storage * | 1 TB |
| Incremental step for (initial and additional) block storage | 50 GB |
| Maximum block storage for a single node |
48 TB The 48 TB total results from 12 volumes of 4 TB each. If you add block storage multiple times, the maximum remains 48 TB, but it may be spread across more than 12 volumes. |
| Maximum block volume size |
4 TB If you specify the maximum 48 TB, 12 drives of 4 TB each are created. If you specify a lower number, enough 4 TB devices for that amount are created, and more devices are created as you add more storage. |
The figures below show initial sizes only. because you can't add additional block storage to master or utility nodes.
| Option | Limits and Guidelines |
|---|---|
| Minimum initial block storage | 150 GB |
| Default initial block storage | 1 TB |
| Minimum additional block storage | 150 GB |
| Default additional block storage | 1 TB |
| Incremental step for (initial and additional) block storage | 50 GB |
| Maximum block storage for a single node | 32TB |
| Maximum block volume size | 32 TB |
| MySQL placement | For utility nodes move /var/lib/mysql to /u01 and create a symbolic link. This prevents filling up the boot volume. |
| Option | Guidelines |
|---|---|
| Default initial block storage | 2 TB |
| Minimum initial block storage | 150 GB |
Query server storage is used for temporary table space to perform heavy JOIN and GROUP BY operations. 2 TB is recommended for typical processing. For small environments, for example development, this number can be adjusted down.
For best performance, consider these factors:
- I/O throughput
- Networking between the compute device and block storage device.
See Block Volume Performance in the Oracle Cloud Infrastructure documentation.
The table below describes how Big Data Service allocates block volume storage for nodes of different sizes.
| What | Amount |
|---|---|
| Initial volume allocation for master nodes and utility nodes | 1 large volume |
| Volume allocation for additional block storage for master nodes and utility nodes | 1 large volume |
| Initial volume allocation for worker nodes. |
|
| Volume allocation for additional block storage for worker nodes |
The minimum number of volumes that can accommodate the storage size, with a maximum volume size of 4 TB per volume. (The last volume may be smaller than 4 TB.) |
It's recommended that you use edge nodes for staging.
Understand cluster profiles
Cluster profiles enable you to create optimal clusters for a specific workload or technology. After creating a cluster with a specific cluster profile, additional Hadoop services can be added to the cluster.
Oracle Big Data Service allows creating cluster for numerous cluster profile types.
| Cluster profile | Components (Secure and Highly Available) | Components |
|---|---|---|
| HADOOP_EXTENDED1 | Hive, Spark, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Ranger, Hue, Oozie, Tez | Hive, Spark, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Hue, Oozie, Tez |
| HADOOP | HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Ranger, Hue | HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Hue |
| HIVE | Hive, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Ranger, Hue, Tez | Hive, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Hue, Tez |
| SPARK | Spark, Hive, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Ranger, Hue 2 | Spark, Hive, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Hue 2 |
| HBASE | HBase, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Ranger, Hue | HBase, HDFS, Yarn, ZooKeeper, MapReduce2, Ambari Metrics, Hue |
| TRINO | Trino, Hive, HDFS, ZooKeeper, Ambari Metrics, Ranger, Hue 2 | Trino, Hive, HDFS, ZooKeeper, Ambari Metrics, Hue 2 |
1 HADOOP_EXTENDED consists of components that you created clusters before cluster profiles were available.
2 Only Hive metastore component from Hive service is available.
The table below lists the Hadoop component versions included in cluster profiles corresponding to ODH version.
ODH 1.x
| Cluster profile | Version |
|---|---|
| HADOOP_EXTENDED | HDFS 3.1, Hive 3.1, Spark 3.0.2 |
| HADOOP | HDFS 3.1 |
| HIVE | Hive 3.1 |
| SPARK | Spark 3.0.2 |
| HBASE | HBase 2.2 |
| TRINO | Trino 360 |
ODH 2.x
| Cluster profile | Version |
|---|---|
| HADOOP_EXTENDED | HDFS 3.3, Hive 3.1, Spark 3.2 |
| HADOOP | HDFS 3.3 |
| HIVE | Hive 3.1 |
| SPARK | Spark 3.2 |
| HBASE | HBase 2.2 |
| TRINO | Trino 389 |
Creating a Cluster
Create an Big Data Service cluster from the Oracle Cloud Console.
Before you can create a cluster, you must have:
-
An Oracle Cloud account or trial, with your sign-in credentials. See Request and Manage Free Oracle Cloud Promotions.
-
A Virtual Cloud Network (VCN), with subnets. See Creating Network Resources.
- Permission to create a cluster. See Creating Policies.
- A Secure Shell (SSH) key pair. See Creating a Key Pair.
The Cluster creation wizard asks you to provide information about your network and to make choices based on your network. To prepare for those questions, have the name of your network, its compartment, and its regional subnet name ready.
Creating a Secure and Highly Available Cluster
You make a cluster secure and highly available by setting an option when you create it. A secure cluster has the full Hadoop security stack, including HDFS Transparent Encryption, plus Kerberos and Apache Sentry.
Make a new cluster secure and highly available by selecting a Secure and Highly Available (HA) under General Settings in the Create Cluster wizard. You can't make an existing cluster secure and highly available if it wasn't created with those features, and you can't remove those features from an existing cluster. See Creating a Cluster for instructions.
Define Security Rules
An administrator must configure security rules to control network traffic to and from Big Data Service resources.
In Oracle Cloud Infrastructure, two kinds of virtual firewalls are available for controlling traffic to and from your cloud resources. Security lists include security rules that apply to an entire subnet. Network security groups include security rules that apply to a defined set of resources that are organized into groups. Network security groups allow finer-grained control, while security lists are easier to set up and maintain.
Security lists and network security groups both include security rules. A security rule allows a particular type of traffic in or out of a Virtual Network Interface Card (VNIC).
A VNIC is a networking component that enables a networked resource such as an instance (a node in Big Data Service) to connect to a virtual cloud network (VCN). The VNIC determines how the instance connects with endpoints inside and outside the VCN. Each VNIC resides in a subnet in a VCN. A security list defines a set of security rules that apply to all the VNICs in a subnet. A network security group defines a set of security rules that apply to a group of VNICs that you define.
It's important to understand the role of VNICs in your network architecture, but for the purposes of this documentation, it's usually sufficient to refer how security rules work in VCNs and subnets.
For more information see Security Rules.
Typically, Big Data Service uses security lists. That means that you create security rules for a subnet, and any cluster in that subnet is subject to those rules. The following instructions tell how to create security rules in a security list defined for the subnet used by your cluster.
A security list can define both ingress rules (for incoming traffic) and egress rules (for outgoing traffic).
- Direction (ingress or egress)
- Stateful or stateless
- Source type and source (ingress rules only)
For complete documentation about security rules, see Parts of a Security Rule.
The following sections contain specific details about creating ingress and egress rules for Big Data Service clusters.
You must open certain ports on Big Data Service clusters to allow access to services like Apache Ambari, Cloudera Manager, and Hue. Configure these ports in the security ingress rules that apply to a cluster.
When creating a cluster, you have the option to use a NAT gateway or not. Whether or not you choose that option affects how you can control outbound traffic.
-
If you choose the NAT gateway option when creating a cluster, all nodes will have full outbound access to the public internet. You won't be able to limit that access in any way (for example by restricting egress to only a few IP ranges).
-
If you choose not to create a NAT gateway when creating a cluster, you can create a NAT gateway on the VCN you're using to access the cluster. You can also edit policies on this NAT gateway to limit egress to specified IP ranges.
-
If you map the VM IPs onto public IPs then a NAT gateway isn't needed.
Establishing Connections to Nodes with Private IP Addresses
By default, cluster nodes are assigned private IP addresses and are therefore not publicly available on the internet. You can make them available in any of the ways described in the following topics:
Big Data Service nodes are by default assigned private IP addresses, which aren't accessible from the public internet. One way to make a node accessible from the internet is to map a node's private IP address to a public IP address.
The instructions below use the Oracle Cloud Infrastructure Cloud Shell, which is a web browser-based terminal accessible from the Oracle Cloud Console. You'll gather some information about your network and your cluster nodes, and then you'll pass that information to commands in the shell. To perform this task, you must have a cluster running in a VCN in your tenancy, and that cluster must have a regional, public subnet.
Required IAM Privileges for Mapping Private to Public IP Address
You must have appropriate Oracle Infrastructure Identity and Access Management (IAM privileges) to map private to public IP addresses.
The tenancy administrator or a delegated administrator with the appropriate privileges must create a policy according to the following guidelines.
GroupThe policy can assign privileges to any Big Data Service group, to give members of that group the rights to map IP addresses.
Verbsvnic_readprivate_ip_readpublic_ip_readpublic_ip_deletepublic_ip_createpublic_ip_updateprivate_ip_assign_public_ipprivate_ip_unassign_public_ippublic_ip_assign_private_ippublic_ip_unassign_private_ip
The policy must specify the tenancy or
the <compartment_name> of the compartment containing the
subnet used for the IP addresses.
allow group bds_net_admins to vnic_read in tenancy
allow group bds_net_admins to private_ip_read in tenancy
allow group bds_net_admins to public_ip_read in tenancy
allow group bds_net_admins to public_ip_delete in tenancy
allow group bds_net_admins to public_ip_create in tenancy
allow group bds_net_admins to public_ip_update in tenancy
allow group bds_net_admins to private_ip_assign_public_ip in tenancy
allow group bds_net_admins to private_ip_unassign_public_ip in tenancy
allow group bds_net_admins to public_ip_assign_private_ip in tenancy
allow group bds_net_admins to public_ip_unassign_private_ip in tenancy- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- On the Clusters page, click the name of your cluster, for example mycluster.
- On the Cluster Information tab of the Cluster Details page, under Customer Network Information, click the Copy link next to Subnet OCID. Then paste that OCID to an editor or a file, so you can retrieve it later in this process.
- On the same page, under List of Cluster Nodes, find the
node you want to map. Node names are constructed like
clusterndpwhere- cluster is the first 7 letters of the cluster name.
- nd is the type of node:
mn= master node,un= utility node, andwn= worker node. - p is the position in the list of the type of node
identified by nd, where
0is the first,1is the second, etc.
For example, the name
myclustun0refers to the first (0) utility node (un) in a cluster namedmycluster. - In the IP Address column, find the private IP address for
that node, for example,
192.0.2.1. Save the address so you can retrieve it later.
-
In the Cloud Console, click the
Cloud Shell icon at the top of the page. It may take
a few moments to connect and authenticate you. -
At the prompt, enter the following.
export DISPLAY_NAME=<display-name>export SUBNET_OCID=<subnet-ocid>export PRIVATE_IP=<ip-address>oci network public-ip create --display-name $DISPLAY_NAME --compartment-id `oci network private-ip list --subnet-id $SUBNET_OCID --ip-address $PRIVATE_IP | jq -r '.data[] | ."compartment-id"'` --lifetime "RESERVED" --private-ip-id `oci network private-ip list --subnet-id $SUBNET_OCID --ip-address $PRIVATE_IP | jq -r '.data[] | ."id"'`The three
exportstatements above set variables that are used in theoci networkcommand that follows. The variables are:-
<display-name> (optional) is a "friendly name" that will be attached to the reserved public IP address. This name is not pre-existing. It's created when running this command.
For convenience, you might want to use the name of the node whose private IP address you're mapping, for example
myclusun0, which is the name of the first utility node in a cluster namedmycluster. -
<subnet-ocid> is the OCID of the customer public subnet used by the cluster for example,
ocid1.subnet.oc1.iad.... -
<ip-address> is the private IP address assigned to the node you want to map, for example,
192.0.2.1.
Enter the command beginning with
oci network public-ip create --compartment-id...exactly as it's shown above, with no breaks.For example:
$ export DISPLAY_NAME="myclustun0"$ export SUBNET_OCID="ocid1.subnet.oc1.…"$ export PRIVATE_IP="192.0.2.1"$ oci network public-ip create --display-name $DISPLAY_NAME --compartment-id `oci network private-ip list --subnet-id $SUBNET_OCID --ip-address $PRIVATE_IP | jq -r '.data[] | ."compartment-id"'` --lifetime "RESERVED" - private-ip-id `oci network private-ip list --subnet-id $SUBNET_OCID --ip-address $PRIVATE_IP | jq -r '.data[] | ."id"'`
The output returned is:{ "data": {"assigned-entity-id": "ocid1.privateip.oc1...","assigned-entity-type": "PRIVATE_IP","availability-domain": null,"compartment-id": "ocid1.compartment.oc1...","defined-tags": {},"display-name": "publicip...","freeform-tags": {},"id": "ocid1.publicip.oc1....","ip-address": "203.0.113.1","lifecycle-state": "ASSIGNED","lifetime": "RESERVED","private-ip-id": "ocid1.privateip....","scope": "REGION","time-created": "2020-04-13..."},"etag": "1234abcd"}
-
-
In the output returned, find the value for
ip-address. In the above example, it's203.0.113.1. That is the new reserved public IP address that is mapped to the private IP address for the node.To see the reserved public IP address in the console, click the
navigation menu. Under Networking, click
Virtual Cloud Networks. Then, in the navigation
list on the left, under Networking, click
IP Management. The new reserved public IP address
appears in the Reserved Public IP Addresses list. If
you supplied a display name in the command you ran, above, that name will
appear in the Name column. Otherwise, a name like
publicipnnnnnnnnnis generated.
Delete a Public IP Address
To delete a public IP, you can use the following:
oci network public-ip delete --public-ip-id
ocid1.publicip.oc1....
The value for --public-ip-id is shown in output returned by the
previous command, as shown above: "id":
"ocid1.publicip.oc1....",.
Alternatively, you can go to the Networking Reserved Public IP Addresses page in the Cloud Console and delete reserved public IPs there.
Open Ports to Make Services Available
Making the node publicly available isn't enough to make a service like Apache Ambari or Cloudera Manager available from the internet. You must also open the port for the service by adding an ingress rule to a security list. See Define Security Rules.
You can use a bastion host to provide access to the a cluster's private network from the public internet.
A bastion host is a compute instance that serves as the public entry point for accessing a private network from external networks like the internet. Traffic must flow through the bastion host to access the private network, and you can set up security mechanisms on the bastion to handle that traffic.
Site-to-Site VPN provides a site-to-site IPSec VPN between your on-premises network and your virtual cloud network (VCN). The IPSec protocol suite encrypts IP traffic before the packets are transferred from the source to the destination and decrypts the traffic when it arrives.
For details for connecting to Big Data Service with VPN see Site-to-Site VPN.
Use FastConnect to access public services in Oracle Cloud Infrastructure without using the internet, for example, access to Object Storage, or the Oracle Cloud Console and APIs. Without FastConnect, the traffic destined for public IP addresses would be routed over the internet. With FastConnect, that traffic goes over your private physical connection.
For details for connecting Big Data Service with Oracle Cloud Infrastructure FastConnect see FastConnect Overview.
Adding Worker and Edge Nodes to a Cluster
When you add worker, compute only worker, or edge nodes to a cluster, you expand both compute and storage. The new nodes use the same instance shape and amount of block storage as the existing worker nodes in the cluster.
Adding Block Storage to Worker Nodes
Block storage is a network-attached storage volume that you can use like a regular hard drive. You can attach extra block storage to the worker nodes of a cluster.
Nodes in a cluster can have remote, network-attached, block storage or local, direct-attached, Non-Volatile Memory Express (NVMe) storage. Remote block storage is flexible and economical, while local NVMe storage provides the highest performance. The default storage type is determined when the cluster is created, based on the shape chosen for the cluster. The high-performance bare metal nodes and dense I/O virtual machine nodes are created with NVMe storage. Other kinds of virtual machine nodes are created with block storage.
You can attach extra storage to any cluster. You can't remove storage.
To add a block volume to the cluster:
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, find the cluster to which you want to add block storage.
- Click the actions menu, and select Add Block Storage.
-
In the Add Block Storage dialog box, enter information,
as follows:
- Additional Block Storage per Node (in GB) - Enter a number to indicate how many gigabytes of block storage to add, between 150GB and 32TB, in increments of 50GB.
- Cluster Admin Password - Enter the administration password for the cluster.
- Click Add.
Adding Cloud SQL to a Cluster
You can add Oracle Cloud SQL to a cluster so you can use SQL to query your big data sources.
Cloud SQL is not included with Big Data Service. You must pay an extra fee for use of Cloud SQL.
When you add Cloud SQL support to a cluster, a query server node is added and big data cell servers are created on all worker nodes.
For clusters with Oracle Distribution including Apache Hadoop, Cloud SQL is only supported for non-HA clusters.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, find the cluster to which you want to add Cloud SQL.
- Click the actions menu, and select Add Cloud SQL.
-
Enter details in the Add Cloud SQL dialog box, as
follows:
- Query Server Node Shape: Select a shape to be used for the query server node. For information about the available shapes, see Understand Instance Types and Shapes.
- Query Server Node Block Storage (in GB): Enter a number to indicate how many gigabytes of block storage to add, between 1TB and 48TB, in increments of 50GB.
- Cluster Admin Password: Enter your cluster administration password.
- Click Add.
Configuring an External Metastore
Data Catalog provides a highly available and scalable metastore for Hive implementations.
You can configure this Data Catalog metastore as an external metastore for your Big Data Service cluster.
You can create one external metastore for your Big Data Service cluster.
To create an external metastore configuration for Big Data Service, you must have the permissions and policies to access and manage a valid Data Catalog metastore.
When you create an external metastore for the Big Data Service cluster, the local metastore for the cluster is made inactive and the new external Data Catalog metastore is activated.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Metastore Configurations.
- Click Create external metastore configuration.
- In the Create external metastore configuration panel, review the Metastore name of the external Data Catalog metastore that is available in the specified compartment for you to configure.
- Select the API key alias you specified when you created the Object Storage API key.
- Enter the API key passphrase. You specified this passphrase when you created the API key.
- Enter the Cluster admin password for the cluster. You specified this password when you created the cluster.
- Click Create.
After successful creation of the external metastore for the Big Data Service cluster, the local metastore for the cluster is made inactive and the new external Data Catalog metastore is activated.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Metastore Configurations.
- The default local metastore is listed. An external metastore is also listed if you have created it. Click the local or external metastore name to view additional details.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Metastore Configurations.
- Click the actions menu for the metastore configuration you want to test and select Test configuration.
- In the Test configuration dialog, enter the Cluster admin password for your cluster and click Test configuration.
- Review the connection status and when done, click Close.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Metastore Configurations.
- Click the actions menu for the external metastore configuration and select Update API key.
- In the Update API key panel, select the new API key.
- Enter the API key passphrase for the new API key.
- Enter the Cluster admin password for the cluster.
- Click Update.
When you activate the local metastore, the external metastore becomes inactive.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Metastore Configurations.
- Click the actions menu for the local metastore configuration and select Activate.
- In the Activate local metastore dialog, enter the Cluster admin password for the cluster.
- Click Activate.
You can delete an inactive external metastore. The external metastore becomes inactive when you activate the local metastore.
You can not delete the default local metastore for your cluster.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Metastore Configurations.
- Click the actions menu for the inactive external metastore configuration and select Delete. You can not delete an active external metastore. To make the external metastore inactive, activate the local metastore.
Install Ext JS to Support Oozie UI
Oozie UI depends on the Ext JS JavaScript framework, and you must manually install Ext JS on your cluster.
These steps apply for clusters using Cloudera Distribution including Apache Hadoop.
Ext JS is a JavaScript framework for building data-intensive, cross-platform web and
mobile applications. Big Data Service requires Ext
JS 2.2 placed in the Oozie webapp directory to make Oozie UI work.
Ext JS is licensed under GPLv2.
To install Ext JS 2.2:
-
Download
ext2.2.zipfromhttp://archive.cloudera.com/gplextras/misc/ext-2.2.zip. - Copy the zip file to a cluster node where Oozie server is installed, and unzip it there.
-
Place the unzipped directory, with name
ext-2.2, in/usr/lib/oozie/embedded-oozie-server/webapp/and verify that ownership of the directory is withoozie:hadoop.
Modifying a Cluster
Stopping and Starting Clusters
Beginning with BDS 3.0.18, you can stop and start clusters.
Stopping a cluster decreases the service cost of the cluster to 25% of the current service cost for the cluster. When workload requests increase, you can restart the cluster.
Stopping a Cluster
You can stop clusters when your workload requests decrease. This reduces the service cost of the cluster to 25% of the current service cost.
Starting a Cluster
You can start clusters when your workload requests increase. The service cost for the cluster returns to normal.
Renaming a Cluster
You can change the name of any cluster.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- Click the action menu for cluster you want to rename, and select Rename Cluster.
- In the Rename Cluster dialog box, enter a new name for the cluster and click Rename.
Changing the Shape of Cluster Nodes
You can change the shapes of the nodes of a cluster after the cluster is created, with the following restrictions:
- All the master nodes and utility nodes in a cluster must use the same shape, and all worker nodes must use the same shape. However, the master nodes and utility nodes can use a different shape than the worker nodes. Therefore, when you change the shapes of nodes, you can change the shape of all the master and utility nodes together, and you can change the shape of all the worker nodes together.
- You can change the shapes only of nodes using standard shapes, and you can only change them to other standard shapes. For information about standard shapes, see Compute Shapes. For information about shapes supported in Big Data Service, see Plan Your Cluster Layout, Shape, and Storage.
To change the shape of a cluster:
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- Click Change Shape.
-
In the Change Shape panel, do the following:
- Choose node type - The available options are as follows:
- Master/utility: Select this option to change the shape of all the master and utility nodes
- Worker: Select this option to change the shape of all the worker nodes
- Compute only worker: Select this option to change the shape of compute only worker nodes.
- Edge: Select this option to change the shape of edge nodes.
- Cloud SQL: Select this option to change the shape of all the Cloud SQL nodes, if installed
- Existing Shape - This read-only field shows the current shape of the nodes of the type you selected for Choose your node type, above.
- New Shape - Select a new shape for the nodes of the selected type.
- Cluster Admin Password - Enter the admin password for the cluster. (The password was assigned when the cluster was created.)
- Choose node type - The available options are as follows:
- Click Change Shape.
Autoscaling a Cluster
You can create an autoscale configuration for a cluster so that the compute shapes and numbers of the worker nodes are automatically increased or decreased, based on the CPU utilization thresholds.
Autoscaling allows you to maintain optimum performance of your cluster, while keeping costs as low as possible. Autoscaling monitors your CPU utilization and automatically adjusts the CPU capacity, based on the configuration parameters you set.
When a cluster is autoscaled, the new details should be reflected in Apache Ambari or Cloudera Manager. To register that change with Apache Ambari or Cloudera Manager, a new cluster admin password is created when you create an autoscale configuration. The password is deleted when the autoscale configuration is deleted.
The autoscale feature collects data about the CPU utilization of worker or compute only worker nodes in a cluster. Two autoscale trigger types are available:
- Metrics: This configuration includes the parameters for scaling up (switching to the next larger compute shape) and scaling down (switching to the next smaller compute shape) or scaling out (adding more nodes to the cluster) and scaling in (removing nodes from the cluster). A scale-up or scale-out configuration specifies a duration and a percentage, so that when the average CPU usage exceeds the specified percentage for the specified duration, the node is scaled up or out. A scale-down or scale-in configuration specifies a duration and a percentage, so that when the average CPU usage falls below the specified percentage for the specified duration, the node is scaled down or in.
The average usage is based on the entire duration specified in the configuration. That is, the autoscale action is triggered at the end of the specified duration. For example, if the scale-up configuration is set to 60% for 6 hours, then the average CPU usage for the entire six hours must exceed 60% for the six-hour duration. The usage might fall below or rise above 60% for brief periods in that six-hour window, but the scale up action is triggered only after the data for the entire six hours is evaluated and averaged, and that average exceeds the percentage specified in the configuration.
If you want the cluster to autoscale more frequently, based on more frequent fluctuations in CPU activity, use shorter duration values. Legal values for scaling durations are 5–60 minutes or 1–24 hours. Enter hours as units of 60 minutes. For example,
60,120,180,240, etc., to1440minutes.Autoscale durations are mapped to Oracle Cloud Infrastructure Monitoring Query Language (MQL) interval values, where the ranges of values allowed for interval are
1m-60m,1h-24h, and1d. (Notice that while the minimum MQL interval is one minute, the minimum Big Data Service interval is five minutes.) See the "Interval Query Component" section in Monitoring Query Language (MQL) Reference.Autoscale takes advantage of Oracle Cloud Infrastructure alarms, and the autoscale duration value is also used as the notification interval for the autoscale alarm. (See Managing Alarms.) If the conditions for an autoscale action are still in effect after another interval, then the alarm will trigger another autoscale.
-
Schedule: Oracle supports two types of schedule-based policies, schedule-based vertical scaling and schedule-based horizontal scaling.
As part of schedule-based vertical scaling, you specify the target shape and the shape configuration (OCPU count and memory size).
As part of schedule-based horizontal scaling, you specify the target node count.
Note
All scheduled-based triggers/conditions associated with a cluster must be more than four hours apart.
Note
Scheduled-based conditions are snoozed for 15 minutes (up to a maximum of two hours), if the cluster's lifecycle state is not active when the trigger fires.
If you use a flex shape with either type of autoscale configuration, you can add or remove the exact number of OCPUs and control the memory usage during each autoscale operation. With a flex shape, you can also set minimum and maximum boundaries. See Supported Node Shapes in the Oracle Cloud Infrastructure documentation for the available shapes.
Oracle recommends that you constantly tune your autoscale values to meet your needs. See the recommendations for tuning alarms in the "Routinely Tune Your Alarms" section in Best Practices for Your Alarms.
Quota
Your tenancy must have a quota that allows you to scale up or scale out worker or compute only worker nodes. If not, the autoscaling operation will fail. See Viewing Your Service Limits, Quotas, and Usage.
Network
When your cluster was created, one of the following options was selected:
-
Deploy Oracle-managed Service gateway and NAT gateway (Quick Start)
If the cluster was created with this option selected, you can configure and use autoscale.
-
Use the gateways in your selected Customer VCN (Customizable)
If the cluster was created with this option selected:
-
If the subnet used for your cluster is private, you must have a service gateway and a NAT gateway. See Overview of Service Gateway.
-
If the subnet is public, you must assign public IP addresses to the nodes. See Map a Private IP Address to a Public IP Address.
-
You can autoscale a cluster horizontally or vertically when specified metric thresholds are exceeded.
Horizontal scaling adds or removes nodes to your cluster. Vertical scaling changes the shape of a node in your cluster.
For example in case of vertical scaling, when thresholds are met the shapes of all the worker nodes in the cluster are automatically scaled up to the next higher VM.Standard shape, scaled down to the next lower VM.Standard shape, or for Flex shapes to the configured OCPU and memory values.
In case of horizontal scaling, when thresholds are met the number worker nodes in the cluster are automatically scaled out or scaled in depending on the configured rules.
ODH clusters support both vertical and horizontal scaling. Horizontal scaling applies only to compute only worker nodes.
CDH clusters support only vertical scaling.
You can have one autoscale configuration per supported node type. Therefore, on a cluster with both worker and compute only worker nodes, you can have up to two autoscale policies.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Autoscale Configurations.
- In the list of autoscale configurations, click the name of your autoscale configuration.
- Click the actions menu and select Delete.
- To confirm, in the Delete Autoscale Configuration dialog box, enter the autoscale configuration name, the cluster admin password, and then click Delete.
Using Object Storage API keys
Big Data Service uses the OCI API signing key mechanism to connect to Object Storage. You can access Object Storage buckets encrypted with user managed keys from the Big Data Service cluster nodes. For details on data encryption, see securing Object Storage.
To use the Object Storage API keys, you must create a Big Data Service cluster with version 3.0.4 or later. The Big Data Service Version is displayed on the Cluster Information tab of the cluster details page. For earlier versions, use OCI HDFS connector for Object Storage.
allow any-user to {USER_INSPECT, USER_READ, USER_UPDATE, USER_APIKEY_ADD, USER_APIKEY_REMOVE} in tenancy where request.principal.id = target.user.id- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- In the left panel, under Resources click Object Storage API keys.
- Click Create key.
- In the Create API key panel, enter a key alias to uniquely identify this key in the cluster.
- Enter the OCID of the user who can use this API key. To retreive the user OCID, from the Console navigate to Identity & Security → Users. From the actons menu for the user, click Copy OCID.
- Enter and confirm a passphrase. This passphrase is used to encrypt the API key and cannot be changed later.
- Select a default region that is used to establish the Object Storage endpoint name.
- Click Create.
You can view and copy the public key of the Object Storage API key from its configuraton file.
- Access the cluster details page of the cluster that has the API key you want to view.
- In the left panel, under Resources click Object Storage API keys.
- From the actions menu of the API key you want to view, click View configuration file.
- The public key details of the API key are displayed in the View configuration file dialog.
- View the configuration file details or copy the public key.
- Access the cluster details page of the cluster that has the API key you want to test.
- In the left panel, under Resources click Object Storage API keys.
- From the actions menu of the API key you want to test, click Test connection.
-
Enter the Object Storage URI for the bucket you want to connect to in the URI format
oci://MyBucket@MyNamespace/. - Enter the passphrase of the API key. You specified this passphrase when you created the API key.
- Click Test connection. The status of the test connection is displayed.
When an Object Storage API key is deleted, all user access to run Objet Storage jobs on the Big Data Service clusters is revoked.
- Access the cluster details page of the cluster that has the API key you want to delete.
- In the left panel, under Resources click Object Storage API keys.
- From the actions menu of the API key you want to delete, click Delete.
- To confirm the deletion, enter the key alias of the key you want to delete.
- Click Delete.
Applying Tags to a Cluster
You can use Oracle Cloud Infrastructure tags to help organize your resources.
Tags are named key/values pairs that you can associate with resources, which you can use to organize and list the resources.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of your cluster.
- Click More Actions, and then Add Tags.
- In the Add Tags dialog box, enter information, as described in Resource Tags.
- Click Add Tags.
Restarting a Cluster Node
You can restart a node in a running cluster.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of the cluster with the node you want to restart.
- On the Cluster Details page, under List of cluster nodes, click the action menu for the node you want to restart. Select Restart Node from the menu.
- In the Restart Node dialog box, enter the name of the node to restart, and click Restart.
Deleting a Cluster Node
You can delete a node in a running cluster.
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- In the list of clusters, click the name of the cluster with the node you want to delete.
- On the Cluster Details page, under List of cluster nodes, click the action menu for the node you want to delete. Select Delete Node from the menu.
- In the Delete Node dialog box, enter the name of the node to delete, and click Delete.
- Select the Force delete even when decommissioning fails checkbox if you want to delete the node even when decommissioning of the node fails.
Removing Cloud SQL from a Cluster
Oracle Cloud SQL can be added to a Big Data Service cluster, for an extra fee. If Cloud SQL has been added to a cluster, you can remove it, and you'll no longer be charged for Cloud SQL on the cluster.
Removing Cloud SQL from a cluster terminates the query server node and deletes any files on that node. This is an irreversible action.
Removing Cloud SQL from the cluster:
- Removes Cloud SQL cells from the cluster worker nodes
- Terminates the query server node and deletes any files or work that you have on that host. (The VM is terminated.)
- Has no impact on Hive metadata or the sources that Cloud SQL accesses.
- Ends the billing for Cloud SQL. You no longer pay for Cloud SQL once it is removed.
To remove Cloud SQL from a cluster:
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- Click action menu for the cluster you want to modify and select Remove Cloud SQL.
- In the Remove Cloud SQL dialog box, enter the cluster admin password and click Remove.
Updating bootstrap script URL
You can run a bootstrap script on all the cluster nodes, after a cluster is created, when the shape of a cluster changes, or when you add or remove nodes from a cluster. You can use this script to install, configure, and manage custom components in a cluster.
The custom bootstrap script is run as an
opc user. If you want to run a command that needs root access, add sudo.$getClusterName: Displays the cluster name.$getAllNodesIps: Displays a space-separated list of IPs of all the nodes.$getUtilityNodesIps: Displays a space-separated list of IPs of the utility nodes.$getWorkerNodesIps: Displays a space-separated list of IPs of the worker nodes.$getMasterNodesIps: Displays a space-separated list of IPs of the master nodes.$getQueryServerNodesIps: Displays a space-separated list of IPs of the query server nodes.$eventType: Displays the event type that initiated the bootstrap script.
$getClusterName$getAllNodesIps$getUtilityNodesIps$getWorkerNodesIps$getMasterNodesIps$getQueryServerNodesIps
#!/bin/bash
#Helper Method Sample
#Use Following Helper function to get list of IPs of all nodes and you can use this to run a command for all nodes
for node in $getAllNodesIps
do
# setup common things for all nodes
echo "Node name: $node"
done
#Cluster Name
echo "Cluster Name"+$getClusterName
#Master Nodes
for node in $getMasterNodesIps
do
# setup common things for master nodes
echo "Master Node name: $node"
done
#Worker Nodes
for node in $getWorkerNodesIps
do
# setup common things for worker nodes
echo "Worker Node name: $node"
done
#Utility Nodes
for node in $getUtilityNodesIps
do
# setup common things for Utility nodes
echo "Utility Node name: $node"
done
#QueryServer Nodes
for node in $getQueryServerNodesIps
do
# setup common things for Query Server nodes
echo "QueryServer Node name : $node"
done
echo "bootstrap script execution complete!!!"$eventType helper variable as shown in the following example:#!/bin/bash
#Take action on certain Event Types
echo "Customer Bootstrap Script Execution for EventType :$eventType"
if [ $eventType == "CreateBdsInstance" ]
then
echo "setup things post Cluster Creation"
fi
if [ $eventType == "AddWorkerNodes" ]
then
echo "setup things post ADD Node "
fi
if [ $eventType == "ChangeShape" ]
then
echo "setup things post Change Shape"
fiTo update the bootstrap script URL, follow these steps:
Finding Available Updates
To find the available updates for a cluster, follow these steps:
Installing Available Updates
To install the available updates for a cluster, follow these steps:
Terminating a Cluster
You can terminate any cluster.
Terminating a cluster deletes the cluster and removes all the data contained in local storage or block storage. This is an irreversible action.
To terminate the cluster:
- Open the navigation menu and click Analytics and AI. Under Data Lake, click Big Data Service.
- In the left panel, under Compartment, select the compartment that hosts your cluster.
- Click the action menu for the cluster you want to terminate and select Terminate Big Data Cluster.
- In the Terminate Big Data Cluster dialog box, enter the name of the cluster and click Terminate.