Manage VM Clusters
Learn how to manage your VM clusters on Exadata Cloud Infrastructure.
- Introduction to Scale Up or Scale Down Operations
With the Multiple VMs per Exadata system (MultiVM) feature release, you can scale up or scale down your VM cluster resources. - Overview of VM Cluster Node Subsetting
VM Cluster Node Subsetting enables you to allocate a subset of database servers to new VM clusters to enable maximum flexibility in the allocation of compute (CPU, memory, local storage) resources. - About Application VIP
Oracle Exadata Database Service on Dedicated Infrastructure fully supports creating additional Virtual IP Addresses on an Exadata VM Cluster. - Using the Console to Manage VM Clusters on Exadata Cloud Infrastructure
Learn how to use the console to create, edit, and manage your VM Clusters on Oracle Exadata Database Service on Dedicated Infrastructure. - Overview of Automatic Diagnostic Collection
By enabling diagnostics collection and notifications, Oracle Cloud Operations and you will be able to identify, investigate, track, and resolve guest VM issues quickly and effectively. Subscribe to Events to get notified about resource state changes. - Incident Logs and Trace Files
This section lists all of the files that can be collected by Oracle Support if you opt-in for incident logs and trace collection. - Health Metrics
Review the list of database and non-database health metrics collected by Oracle Trace File Analyzer. - Using the API to Manage Exadata Cloud Infrastructure Instance
- Troubleshooting Virtual Machines Using Console Connections
You can troubleshoot malfunctioning virtual machines using console connections. For example, a previously working Guest VM stops responding.
Parent topic: How-to Guides
Introduction to Scale Up or Scale Down Operations
With the Multiple VMs per Exadata system (MultiVM) feature release, you can scale up or scale down your VM cluster resources.
- Scale VM Resources in Multi VM Enabled Infrastructure
Increase or decrease the OCPUs, memory, storage or local disk size (/u02
) storage available to a VM cluster - Resizing Memory and Large Pages
You can scale the database server memory up and down in a VM Cluster. Scaling memory requires a rolling restart of the database servers to take effect. - Calculating the ASM Storage
- Estimating How Much Local Storage You Can Provision to Your VMs
- Scaling Local Storage
Parent topic: Manage VM Clusters
Scale VM Resources in Multi VM Enabled Infrastructure
Increase or decrease the OCPUs, memory, storage or local disk size
(/u02
) storage available to a VM cluster
Oracle doesn't stop billing when a VM or VM Cluster is stopped. To stop billing for a VM Cluster, lower the OCPU count to zero.
Scaling up or down of these resources requires thorough auditing of existing usage and capacity management by the customer DB administrator. Review the existing usage to avoid failures during or after a scale down operation. While scaling up, consider how much of these resources are left for the next VM cluster you are planning to create. Oracle Exadata Database Service on Dedicated Infrastructure tooling calculates the current usage of memory, local disk, and ASM storage in the VM cluster, adds headroom to it, and arrives at a "minimum" value below which you cannot scale down, and expects that you specify the value below this minimum value.
- When creating or scaling a VM Cluster, setting the number of OCPUs to zero will shut down the VM Cluster and eliminate any billing for that VM Cluster, but the hypervisor will still reserve the minimum 2 OCPUs for each VM. These reserved OCPUs cannot be allocated to any other VMs, even though the VM to which they are allocated is shut down. The Control Plane does not account for reserved OCPUs when showing maximum available OCPU, so you should account for these reserved OCPU when performing any subsequent scaling operations to ensure the operation can acquire enough OCPUs to successfully complete the operation.
- For memory and
/u02
scale up or scale down operations, if the difference between the current value and the new value is less than 2%, then no change will be made to that VM. This is because memory change involves rebooting the VM, and/u02
change involves bringing down the Oracle Grid Infrastructure stack and un-mounting/u02
. Production customers will not resize for such a small increase or decrease, and hence such requests are a no-op. - You can scale down the following resources in any combinations:
- OCPU
- Memory
- Local storage
- Exadata storage
Each scaling operation can take several minutes to complete. The time for each operation will vary based on activity in the system, but as a general rule, most operations should complete within 15 minutes for a quarter rack, 20 minutes for a half rack, and 30 minutes for a full or larger rack. Performing multiple OCPU scaling operations over a short period of time can lengthen the time for completion. Although online, OCPU scaling is not implemented on all VMs in parallel so as to detect and protect from any anomalies before they affect the entire system. Memory and Local Storage scaling require a VM reboot, and are performed one VM at a time in a rolling manner.
If you run multiple scale-down operations, then each operation is performed serially. For example, if you scale memory and local storage from the Console, then the system will first scale memory, and when that operation completes, it will scale storage. The time to complete all operations will be the sum of the time to complete individual operations.
- Storage servers added to the infrastructure but yet to run the 'Add Capacity' step will not have any disk groups created on them.
Parent topic: Introduction to Scale Up or Scale Down Operations
Resizing Memory and Large Pages
You can scale the database server memory up and down in a VM Cluster. Scaling memory requires a rolling restart of the database servers to take effect.
Changing the memory in a VM Cluster will affect the large pages (HugePages) settings for the VMs in that cluster. When a VM is initially created, each VM's operating system is configured with 50% of the memory allocated to the VM for large pages, and databases are configured to use that memory for their SGA. Oracle recommends that you not modify the large pages configuration unless you understand the implication of any changes you make. Improper configurations can prevent all databases from starting, and even prevent the VM from starting up.
Although not recommended, you are allowed to modify the large pages configuration. Any changes you make may be modified by automation should you subsequently resize the memory available to the VM. After a memory resize operation, the cloud automation will attempt to maintain the same amount of large pages memory as a percentage of the total memory, with a cap of 60%. If you configure large pages to be greater than 60% of total memory, then the cloud automation will resize it to 60% of total memory. This automatic resize is to ensure sufficient conventional memory for the virtual machine to start. The automation will perform a precheck to determine the actual large pages memory in use by the running database instances, and ensure after the resize that there is enough large pages memory available to support those same databases. If there will not be sufficient memory available after the resize, then the precheck will fail and the resize will not continue.
Parent topic: Introduction to Scale Up or Scale Down Operations
Calculating the ASM Storage
Use the following formula to calculate the minimum required ASM storage:
- For each disk group, for example,
DATA
,RECO
, note the total size and free size by running theasmcmd lsdg
command on any Guest VM of the VM cluster. - Calculate the used size as (Total size - Free size) / 3 for each disk group. The /3 is used because the disk groups are triple mirrored.
-
DATA:RECO ratio is:
80:20 if Local Backups option was NOT selected in the user interface.
40:60 if Local Backups option was selected in the user interface.
- Ensure that the new total size as given in the user interface passes
the following conditions:
Used size for DATA * 1.15 <= (New Total size * DATA % )
Used size for RECO * 1.15 <= (New Total size * RECO % )
Example 5-2 Calculating the ASM Storage
- Run the
asmcmd lsdg
command in the Guest VM:- Without
SPARSE:
/u01/app/19.0.0.0/grid/bin/asmcmd lsdg ASMCMD> State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED HIGH N 512 512 4096 4194304 12591936 10426224 1399104 3009040 0 Y DATAC5/ MOUNTED HIGH N 512 512 4096 4194304 3135456 3036336 348384 895984 0 N RECOC5/ ASMCMD>
- With
SPARSE:
/u01/app/19.0.0.0/grid/bin/asmcmd lsdg ASMCMD> State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED HIGH N 512 512 4096 4194304 12591936 10426224 1399104 3009040 0 Y DATAC5/ MOUNTED HIGH N 512 512 4096 4194304 3135456 3036336 348384 895984 0 N RECOC5/ MOUNTED HIGH N 512 512 4096 4194304 31354560 31354500 3483840 8959840 0 N SPRC5/ ASMCMD>
Note
The listed values of all attributes for SPARSE diskgroup (SPRC5) present the virtual size. In Exadata DB Systems and Exadata Cloud Infrastructure, we use the ratio of 1:10 for
physicalSize
:virtualSize
. Hence, for all purposes of our calculation we must use 1/10th of the values displayed above in case of SPARSE for those attributes. - Without
SPARSE:
- Used size for a disk group = (Total_MB - Free_MB) /3
- Without SPARSE:
Used size for DATAC5 = (12591936 - 10426224 ) / 3 = 704.98 GB
Used size for RECO5 = (3135456 - 3036336 ) / 3 = 32.26 GB
- With SPARSE:
Used size for DATAC5 = (12591936 - 10426224 ) / 3 ~= 704.98 GB
Used size for RECO5 = (3135456 - 3036336 ) /3 ~= 32.26 GB
Used size for SPC5 = (1/10 * (31354560 - 31354500)) / 3 ~= 0 GB
- Without SPARSE:
- Storage distribution among diskgroups
- Without SPARSE:
DATA:RECO ratio is 80:20 in this example.
- With SPARSE:
DATA RECO: SPARSE ratio is 60:20:20 in this example.
- Without SPARSE:
- New requested size should pass the following conditions:
- Without SPARSE: (For example, 5 TB in user interface.)
5 TB = 5120 GB ; 5120 *.8 = 4096 GB; 5120 *.2 = 1024 GB
For DATA: (704.98 * 1.15 ) <= 4096 GB
For RECO: (32.36 * 1.15) <= 1024 GB
- With SPARSE: (For example, 8 TB in the user interface.)
8 TB = 8192 GB; 8192 *.6 = 4915 GB; 8192 *.2 = 1638 GB; 8192 *.2 = 1638 GB
For DATA: (704.98 * 1.15 ) <= 4915 GB
For RECO: (32.36 * 1.15) <= 1638 GB
For SPR: (0 * 1.15) <= 1638 GB
- Without SPARSE: (For example, 5 TB in user interface.)
Above resize will go through. If above conditions are not met by the new size, then resize will fail the precheck.
Parent topic: Introduction to Scale Up or Scale Down Operations
Estimating How Much Local Storage You Can Provision to Your VMs
The following does not apply to X6, X7, X8, and Base Systems as they do not support multiple VMs. The Base System has 200 GB available for
/u02
.
VM Images include the files necessary to boot and run the VM and its operating system, as well as space for Oracle Homes stored in /u02
. To estimate how much additional local storage space beyond the minimum can be allocated to any file system associated with a VM, subtract the size of the VM images for all VMs on a server from the total available space. If you have not modified the default VM Image size by expanding any file systems, use the VM Image size (default and minimum) below. If you have or plan to modify your VM Image size, you must use the OCI console and "Scale VM Cluster" action to check the allocated and available for an existing VM Cluster as expanding some non-/u02 file systems will consume more incremental storage than was added to the file system. This information is also available in the "Configure VM Cluster" action while creating a new VM Cluster.
X8M-2 Systems
- Total space available for VM images (X8M): 2243 GB
- VM Image size (default and minimum) including
/u02
: 244 GB - Default (minimum)
/u02
: 60 GB
X9M-2 Systems
- Total Available for VM Images: 2243 GB
- VM Image size (default and minimum) including
/u02
: 244 GB - Default (minimum)
/u02
: 60 GB
Example: If you have an X9M Elastic System with 2 VMs per physical server, and have not made any changes to any of the file systems, you will have 2243 GB available for all VMs, and each will consume 244 GB (488 total), leaving 1755 GB to expand any VM Images. The default VM image will include 60GB of /u02
per VM to store Oracle homes. The 1755 GB of available space can be used to expand /u02
, or can be used to expand other file systems in the VM Image. Every GB used to expand /u02
will consume a GB of available space. Every GB used to expand other file systems in the VM image may consume more than a GB of space. Refer to the information in the console when expanding non-/u02 file systems to see the actual available space impact of expanding these file systems.
Parent topic: Introduction to Scale Up or Scale Down Operations
Scaling Local Storage
Scale Local Space Operation Guidelines
You can scale local storage by modifying the size of many of the individual file systems in a VM. By default, the file systems are created at their minimum size. You can increase the size of the file systems as required. However, note that you can only shrink /u02
. Other file systems can only be increased in size. The maximum supported size of any file system is 900 GB.
The storage consumed by all file systems is greater than the sum of the file system sizes. Refer to the calculations displayed in the OCI console to see the effects on free local storage when resizing a file system.
/u02
/
/u01
/tmp
/var
/var/log
/var/log/audit
/home
/crashfiles
/boot
/acfs01
/u01/app/19.0.0.0/grid
- With the exception of
/u02
, you can only expand the file systems and cannot reduce their size once they have been expanded. - A rolling restart of each VM is required for the resizing to take effect.
- Each file system can only be expanded to a maximum of 900 GB
- Ability to increase the size of additional local file systems is only supported on X8M and later systems.
For more information about resizing these file systems, see Estimating How Much Local Storage You Can Provision to Your VMs.
Resource Limit Based On Current Utilization
- Any scale-down operation must leave 15% buffer on top of highest local space utilization across all nodes in the cluster.
- The lowest local space per node allowed is higher of the above two limits.
- Run the
df –kh
command on each node to find out the node with the highest local storage. - You can also use the utility like
cssh
to issue the same command from all hosts in a cluster by typing it just once. - Lowest value of local storage each node can be scaled down to would be = 1.15x (highest value of local space used among all nodes).
ACFS File Systems
If requested by support, you can also resize the /acfs01
file system. This file system is used by the system to stage software. It uses Exadata storage and is not subject to the limits described above for /u02
. It is a shared file system visible from all nodes in the cluster, and can be online resized from the command line of any VM.
- Default size: The default size of
/acfs01
is 100 GB. - Scaling /acfs01: You can scale
acfs01
as usergrid
from any VM via the/sbin/acfsutil
command. No reboot is required. The resize operation will not affect the availability of the database service running in the VM cluster. The following command issued by thegrid
user will increase the size of/acfs01
by 100 GB:/sbin/acfsutil size +100 GB /acfs01
. - You can create additional ACFS file systems if required. These will also consume storage from the Exadata Storage diskgroups and may be shared across all VMs in the cluster. Refer to the ACFS documentation for more information.
Parent topic: Introduction to Scale Up or Scale Down Operations
Overview of VM Cluster Node Subsetting
VM Cluster Node Subsetting enables you to allocate a subset of database servers to new VM clusters to enable maximum flexibility in the allocation of compute (CPU, memory, local storage) resources.
- Create a smaller VM cluster to host databases that have low resource and scalability requirements or to host a smaller number of databases that require isolation from the rest of the workload.
- Expand or shrink an existing VM cluster by adding and removing nodes to ensure optimal utilization of available resources.
- VM Cluster Node Subsetting capability is available for new VM clusters in Oracle Exadata Database Service on Dedicated Infrastructure service.
- All VMs across a VM cluster will have the same resource allocation per VM irrespective of whether the VM was created during cluster provisioning or added later by extending an existing VM cluster.
- Any VM cluster must have a minimum of 2 DB servers.
- You can host a maximum of 8 VMs per DB Server on X8M and above generations.
- Exadata Infrastructures with X8M and above generation of DB Servers can support a maximum of 8 VM clusters across all DB Servers.
- Maximum number of clusters across the infrastructure depends on the resources available per DB server and is subject to the per DB Server maximum VM limit.
With the release of Multi-VM, the add and remove virtual machine api for cloud vm clusters will not be supported via terraform.
You can perform these operations through UI, SDK, OCI CLI, OCI Ansible or similar tools. Terraform states should be managed similar to other operations which happen outside of terraform but need to be managed in terraform.
For more information, see Detecting and Managing Drift with Terraform.
- Add a VM to a VM Cluster
Add a Virtual Machine to a VM Cluster - Terminate a VM from a VM Cluster
To remove a virtual machine from a provisioned cluster, use this procedure.
Related Topics
Parent topic: Manage VM Clusters
Add a VM to a VM Cluster
Add a Virtual Machine to a VM Cluster
Once the VM cluster is upgraded to Exadata Database Service Guest VM OS 23.1, you will be able to add a new VM or a new database server to this VM cluster if Exadata Cloud Infrastructure is running an Exadata System Software version 22.1.16 and later.
Upgrade to Exadata System Software 23.1 for Exadata Cloud Infrastructure will be available with February 2023 update cycle.
- This operation is only available with Multi-VM enabled Infrastructure.
- To add a VM to a VM Cluster requires that all TCP ports to be open for the client subnet CIDR for ingress and egress.
- Open the navigation menu. Under Oracle Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Choose the Region and Compartment that contains the VM cluster for which you want to scale the CPU resources.
- Click VM Clusters.
- Click the name of the VM cluster to which you want to add a virtual machine.
- Under Resources, select Virtual Machines, and click the Add Virtual Machines button.
- In the Add Virual Machines window, select the DB server where you want the new VM to
reside.
Note
The VM that is added will have the same resources as the other VMs in the cluster. - Click Add Virtual Machine.
Add a VM to a VM Cluster is NOT supported using Terraform.
Related Topics
Parent topic: Overview of VM Cluster Node Subsetting
Terminate a VM from a VM Cluster
To remove a virtual machine from a provisioned cluster, use this procedure.
- Open the navigation menu. Under Oracle Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Choose the Region and Compartment that contains the VM cluster for which you want to scale the CPU resources.
- Click VM Clusters.
- Click the name of the VM cluster for which you want to remove a virtual machine.
- On the Exadata VM Cluster Details page, in the Virtual Machines section, select the Virtual Machine that will be removed, click the more commands symbol (three dots) and click Terminate
Remove a VM from a VM Cluster is NOT supported using Terraform at this time.
Parent topic: Overview of VM Cluster Node Subsetting
About Application VIP
Oracle Exadata Database Service on Dedicated Infrastructure fully supports creating additional Virtual IP Addresses on an Exadata VM Cluster.
These application VIPs are required to protect additional applications such as Oracle GoldenGate installed on an Oracle Exadata Database Service on Dedicated Infrastructure system or other services such as XA-Agents, and to provide High Availability to these additional applications. For more information, see Oracle Grid Infrastructure Standalone Agents for Oracle Clusterware and Making Applications Highly Available Using Oracle Clusterware.
Within the Oracle Cloud Infrastructure, adding Virtual IP Addresses on the cluster stack alone is not sufficient as these additional (secondary) IP addresses also have to be added to the VCN layer as “Floating IP” addresses so that the VCN layer knows where these IP addresses are running, and in case of failover by the Clusterware to change the VNIC the floating IP address is attached to. For more information, see Creating an Application VIP Managed by Oracle Clusterware and Overview of IP Addresses.
Adding an Application VIP to an Exadata VM Cluster consists of the following steps:
- Add the Virtual IP address to the Clusterware layer within the Exadata
DomU, by following the standard Oracle Clusterware
documentation, or the guide provided by the
application, for example, by using
.appvipcfg create -network=1 -ip=10.10.10.10 -vipname=applicationvip
-
Attach the Application Floating IP address object on the Exadata VM Cluster to add the knowledge of the floating IP to the VCN layer. Ensure that you choose the same subnet as you created the backend application VIP, which normally is the client subnet.
The private IP address needs to be the same as the one specified in the
appvipcfg
command above. The Virtual IP Address Hostname is the name under which the IP address is reachable via DNS and does not have to be the same as thevipname
.If you have already started the VIP in the backend, ensure that the Virtual Machine Name reflects the host on which the VIP was started in the backend.
-
Test the relocation of the VIP. The VIP should stay available (test this via. ping), and the user interface should display after a short while that the floating IP also has moved to another host.
If you did choose the wrong host while creating the VCN attachment, then simply relocate the VIP within the cluster. The VCN layer will detect the change and the user interface should get updated after a short while.
A single Virtual VM Cluster has a limitation of 8 additional Application VIPs. The limitation exists because a single VNIC can only have 31 additional secondary IP addresses. For more information, see Overview of IP Addresses. If all VIPs are started on the same node, then the application VIPs cannot be reached.
If more application VIPs are required, then raise an SR to have this limit increased. However, there are a few additional steps required then, to ensure that under no scenario more than 31 secondary IP addresses are attached to a single Exadata VM Cluster node. One way to accomplish this would be to ensure application VIPs are bound by the Clusterware to certain nodes so that this scenario is prevented.
A setup with 32 additional application VIPs would look as follows:
Floating IP | Node 1 | Node 2 | Node 3 | Node 4 |
---|---|---|---|---|
Private Hostname | 1 | 1 | 1 | 1 |
VIP Hostname | 4 | 4 | 4 | 4 |
SCAN | 3 | 3 | 3 | 3 |
Appvip 1-8 | 8 | 8 | - | - |
Appvip 9-16 | - | 8 | 8 | - |
Appvip 17-24 | - | - | 8 | 8 |
Appvip 25-32 | 8 | - | - | 8 |
Max Possible VIP if all Floating IPs failover | 24 | 24 | 24 | 24 |
Using the Console to Manage VM Clusters on Exadata Cloud Infrastructure
Learn how to use the console to create, edit, and manage your VM Clusters on Oracle Exadata Database Service on Dedicated Infrastructure.
- To create a cloud VM cluster resource
Create a VM cluster in an Exadata Cloud Infrastructure instance. - To add database server or storage server capacity to a cloud VM cluster
This topic describes how to use the Oracle Cloud Infrastructure (OCI) Console to add the new capacity to your cloud VM cluster. - Using the Console to Enable, Partially Enable, or Disable Diagnostics Collection
You can enable, partially enable, or disable diagnostics collection for your Guest VMs after provisioning the VM cluster. Enabling diagnostics collection at the VM cluster level applies the configuration to all the resources such as DB home, Database, and so on under the VM cluster. - Using the Console to Update the License Type on a VM Cluster
To modify licensing, be prepared to provide values for the fields required for modifying the licensing information. - To add SSH keys to a VM cluster
The VM cluster exists, and you wish to add a another user which requires another SSH key. - Using the Console to Add SSH Keys After Creating a VM Cluster
- Using the Console to Stop, Start, or Reboot a VM Cluster Virtual Machine
Use the console to stop, start, or reboot a virtual machine. - Using the Console to Check the Status of a VM Cluster Virtual Machine
Review the health status of a VM cluster virtual machine. - Using the Console to Move a VM Cluster to Another Compartment
To change the compartment that contains your VM cluster on Exadata Cloud Infrastructure, use this procedure. - To change the VM cluster display name
- Using the Console to Terminate a VM Cluster
Before you can terminate a VM cluster, you must first terminate the databases that it contains. - To view details about private DNS configuration
- To Attach a Virtual IP Address
Attach a Virtual IP address from a VM cluster using this procedure. - To Detach a Virtual IP Address
Attach a Virtual IP address from a VM cluster using this procedure.
Parent topic: Manage VM Clusters
To create a cloud VM cluster resource
Create a VM cluster in an Exadata Cloud Infrastructure instance.
To create a cloud VM cluster in an Exadata Cloud Infrastructure instance, you must have first created a Cloud Exadata infrastructure resource.
Multi-VM enabled Infrastructure will support creating multiple VM Clusters. Infrastructures created before the feature Create and Manage Multiple Virtual Machines per Exadata System (MultiVM) and VM Cluster Node Subsetting was released only support creating a single cloud VM cluster.
- Open the navigation menu. Click Oracle Database, then click Oracle Exadata Database Service on Dedicated Infrastructure
- Under Oracle Exadata Database
Service on Dedicated Infrastructure, click Exadata VM
Clusters.
Note
Multiple VM clusters may be created only in a Multi-VM enabled Infrastructure. - Click Create Exadata VM Cluster.
The Create Exadata VM Cluster page is displayed. Provide the required information to configure the VM cluster.
- Compartment: Select a compartment for the VM cluster resource.
- Display name: Enter a user-friendly display name for the VM cluster. The name doesn't need to be unique. An Oracle Cloud Identifier (OCID) will uniquely identify the DB system. Avoid entering confidential information.
- Select Exadata infrastructure: Select the infrastructure
resource that will contain the VM cluster. You must choose an infrastructure
resource that has enough resources to create a new VM cluster. Click
Change Compartment and pick a different compartment from
the one you are working in to view infrastructure resources in other
compartments.
Note
Multiple VM clusters may be created only in a Multi-VM enabled Infrastructure - Choose the Oracle Grid Infrastructure version: From the list, choose the Oracle Grid Infrastructure release (19c and 23ai) that you want to install on the VM cluster.
The Oracle Grid Infrastructure release determines the Oracle Database releases that can be supported on the VM cluster. You cannot run an Oracle Database release that is later than the Oracle Grid Infrastructure software release.
Note
Minimum requirements for provisioning a VM Cluster with Grid Infrastructure 23ai:- Exadata Guest VM running Exadata System Software 23.1.8
- Exadata Infrastructure running Exadata System Software 23.1.x
- Choose an Exadata image version:
- Exadata infrastructure with Oracle Linux 7 and Exadata image version 22.1.10.0.0.230422:
- The Change image button is not enabled.
- The Oracle Grid Infrastructure version defaults to 19.0.0.0.0.
- The Exadata guest version will be the same as that of the host OS.
- Exadata infrastructure with Oracle Linux 8 and Exadata image version 23.1.3.0.0.230613:
- The Exadata guest version defaults to the latest (23.1.3.0).
- The Oracle Grid Infrastructure version defaults to 19.0.0.0.0
- The Change image button is enabled.
- Click Change image.
The resulting Change image panel displays the list of available major versions of Exadata image (23.1.3.0 and 22.1.3.0).
The most recent release for each major version is indicated by "(latest)".
- Slide Display all available versions.
Six past versions including the latest versions of Exadata images 23.1.3.0 and 22.1.3.0 are displayed.
- Choose a version.
- Click Save Changes.
- Exadata infrastructure with Oracle Linux 7 and Exadata image version 22.1.10.0.0.230422:
- Configure the VM cluster: Specify the DB servers to used for
new VM cluster (by default all DB Servers are selected). Click Change DB
Servers to select from the available DB servers. In the
Resource allocation per VM pane:
- Specify the number of OCPU cores you want to allocate to each of the VM cluster's virtual machine compute nodes. Minimum is 2 OCPU per VM . The read-only Requested OCPU count for the Exadata VM cluster field displays the total number of OCPU cores you are allocating.
- Specify the Memory per VM to allocate to each VM. The minimum per VM is 30 GB.
- Specify the Local Storage per VM to allocate local storage to each VM. The minimum per VM is 60 GB.
Each time when you create a new VM cluster, the space remaining out of the total available space is utilized for the new VM cluster.
In addition to
/u02
, you can specify the size of additional local file systems.For more information and instructions to specify the size for each individual VM, see Introduction to Scale Up or Scale Down Operations.
- Click Show additional local file systems configuration options.
- Specify the size of
/
,/u01
,/tmp
,/var
,/var/log
,/var/log/audit
, and/home
file systems as needed.Note
- You can only expand these file systems and cannot reduce the size once expanded.
- Due to backup partitions and mirroring, the
/
and/var
file systems will consume twice the space they were allocated, which is indicated in the read-only Total allocated storage for / (GB) due to mirroring and Total allocated storage for /tmp (GB) due to mirroring fields.
- After creating the VM Cluster, check the Exadata Resources section on the Exadata Infrastructure Details page to check the file size allocated to the local storage (
/u02
) and local storage (additional file systems).
-
Configure Exadata storage: Specify the following:
- Specify the usable Exadata storage TB. Specify the storage in multiples of 1 TB. Minimum: 2 TB
- Allocate storage for Exadata sparse snapshots:
Select this configuration option if you intend to use
snapshot functionality within your VM cluster. If you select this
option, the SPARSE disk group is created, which enables you to use VM
cluster snapshot functionality for PDB sparse cloning. If you do not
select this option, the SPARSE disk group is not created and snapshot
functionality will not be available on any database deployments that are
created in the environment.
Note
The storage configuration option for sparse snapshots cannot be changed after VM cluster creation. -
Allocate storage for local backups: Select this
option if you intend to perform database backups to the local Exadata
storage within your Exadata Cloud Infrastructure instance. If you select this option, more space
is allocated to the RECO disk group, which is used to store backups on
Exadata storage. If you do not select this option, more space is
allocated to the DATA disk group, which enables you to store more
information in your databases.
Note
The storage configuration option for local backups cannot be changed after VM cluster creation.
- Add SSH key: Add the public key portion of each key pair you want to use for SSH access to the DB system:
- Generate SSH key pair (Default option) Select this radio button to generate an SSH keypair. Then in the dialog below click Save private key to download the key, and optionally click Save public key to download the key.
- Upload SSH key files: Select this radio button to browse or drag and drop .pub files.
- Paste SSH keys: Select this radio button to paste in individual public keys. To paste multiple keys, click + Another SSH Key, and supply a single key for each entry.
-
Configure the network settings: Specify the following:
Note
IP addresses (100.64.0.0/10) are used for Exadata Cloud Infrastructure X8M interconnect.- Virtual cloud network: The VCN in which you want to create the VM cluster. Click Change Compartment to select a VCN in a different compartment.
- Client subnet: The subnet to which the VM cluster should attach. Click Change Compartment to select a subnet in a different compartment.
Do not use a subnet that overlaps with 192.168.16.16/28, which is used by the Oracle Clusterware private interconnect on the database instance. Specifying an overlapping subnet causes the private interconnect to malfunction.
- Backup subnet: The subnet to use for the backup network, which is typically used to transport backup information to and from the Backup Destination, and for Data Guard replication. Click Change Compartment to select a subnet in a different compartment, if applicable.
Do not use a subnet that overlaps with 192.168.128.0/20. This restriction applies to both the client subnet and backup subnet.
If you plan to back up databases to Object Storage or Autonomous Recovery service, see the network prerequisites in Managing Exadata Database Backups.
Note
In case Autonomous Recovery Service is used, a new dedicated subnet is highly recommended. Review the network requirements and configurations required to backup your Oracle Cloud databases to Recovery Service. See, Configuring Network Resources for Recovery Service. -
Network Security Groups: Optionally, you can specify one or more network security groups (NSGs) for both the client and backup networks. NSGs function as virtual firewalls, allowing you to apply a set of ingress and egress security rules to your Exadata Cloud Infrastructure VM cluster. A maximum of five NSGs can be specified. For more information, see Network Security Groups and Network Setup for Exadata Cloud Infrastructure Instances.
Note that if you choose a subnet with a security list, the security rules for the VM cluster will be a union of the rules in the security list and the NSGs.
To use network security groups:
- Check the Use network security groups to control traffic check box. This box appears under both the selector for the client subnet and the backup subnet. You can apply NSGs to either the client or the backup network, or to both networks. Note that you must have a virtual cloud network selected to be able to assign NSGs to a network.
- Specify the NSG to use with the network. You might need to use more than one NSG. If you're not sure, contact your network administrator.
- To use additional NSGs with the network, click +;Another Network Security Group.
- To use private DNS
ServiceNote
A Private DNS must be configured before it can be selected. See Configure Private DNS- Check the Use private DNS Service check box.
- Select a private view. Click Change Compartment to select a private view in a different compartment.
- Select a private zone. Click Change Compartment to select a private zone in a different compartment.
- Hostname prefix: Your choice of hostname for the Exadata DB system. The host name must begin with an alphabetic character and can contain only alphanumeric characters and hyphens (-). The maximum number of characters allowed for an Exadata DB system is 12.
Caution:
The hostname must be unique within the subnet. If it is not unique, the VM cluster will fail to provision. - Host domain name: The domain name for the VM cluster. If the selected subnet uses the Oracle-provided Internet and VCN Resolver for DNS name resolution, this field displays the domain name for the subnet and it can't be changed. Otherwise, you can provide your choice of the domain name. Hyphens (-) are not permitted.
If you plan to store database backups in Object Storage or Autonomous Recovery service, Oracle recommends that you use a VCN Resolver for DNS name resolution for the client subnet because it automatically resolves the Swift endpoints used for backups.
- Host and domain URL: This read-only field combines the host and domain names to display the fully qualified domain name (FQDN) for the database. The maximum length is 63 characters.
- Choose a license type: The type of license you want to use for the VM cluster. Your choice affects metering for billing.
- License Included means the cost of the cloud service includes a license for the Database service.
- Bring Your Own License (BYOL) means you are an Oracle Database customer with an Unlimited License Agreement or Non-Unlimited License Agreement and want to use your license with Oracle Cloud Infrastructure. This removes the need for separate on-premises licenses and cloud licenses.
- Diagnostics Collection: By enabling diagnostics
collection and notifications, Oracle Cloud Operations and you will be able to
identify, investigate, track, and resolve guest VM issues quickly and effectively.
Subscribe to Events to get notified about resource state changes.
Note
You are opting in with the understanding that the above list of events (or metrics, log files) can change in the future. You can opt out of this feature at any time.- Enable Diagnostic Events: Allow Oracle to collect and publish critical, warning, error, and information events to me.
- Enable Health Monitoring: Allow Oracle to collect health metrics/events such as Oracle Database up/down, disk space usage, and so on, and share them with Oracle Cloud operations. You will also receive notification of some events.
- Enable Incident Logs and Trace Collection: Allow Oracle to collect incident logs and traces to enable fault diagnosis and issue resolution.
Note
You are opting in with the understanding that the above list of events (or metrics, log files) can change in the future. You can opt-out of this feature at any time.All three checkboxes are selected by default. You can leave the default settings as is or clear the checkboxes as needed. You can view the Diagnostic Collection settings on the VM Cluster Details page under General Information >> Diagnostics Collection.- Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files (all three options).
- Disabled: When you choose not to collect diagnostics, health metrics, incident logs, and trace files (all three options).
- Partially Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files ( one or two options).
- Click Show Advanced Options to specify advanced options for the VM cluster:
-
Time zone: This option is located in the Management tab. The default time zone for the DB system is UTC, but you can specify a different time zone. The time zone options are those supported in both the Java.util.TimeZone class and the Oracle Linux operating system. For more information, see DB System Time Zone .
Note
If you want to set a time zone other than UTC or the browser-detected time zone, and if you do not see the time zone you want, try selecting the Select another time zone, option, then selecting "Miscellaneous" in the Region or country list and searching the additional Time zone selections.
- SCAN Listener Port: This option is located in the Network tab. You can assign a SCAN listener port (TCP/IP) in the range between 1024 and 8999. The default is 1521
Note.
Manually changing the SCAN listener port of a VM cluster after provisioning using the backend software is not supported. This change can cause Data Guard provisioning to fail. - Tags: If you have permissions to create a resource, then you also have permissions to apply free-form tags to that resource. To apply a defined tag, you must have permissions to use the tag namespace. For more information about tagging, see Resource Tags. If you are not sure whether to apply tags, skip this option (you can apply tags later) or ask your administrator.
-
- Click Create Exadata VM Cluster.
WHAT NEXT?
After your VM cluster is successfully created and in the Available state, you can view the VM Cluster Details page by clicking the name of the VM cluster in the list of clusters. From the VM Cluster Details page, you can create your first database in the cluster by clicking Create Database.
Related Topics
- Network Security Groups
- Network Setup for Exadata Cloud Infrastructure Instances
- Security Lists
- Configure Private DNS
- DB System Time Zone
- Resource Tags
- To create a database in an existing Exadata Cloud Infrastructure instance
- Getting Started with Events
- Overview of Database Service Events
- Overview of Automatic Diagnostic Collection
To add database server or storage server capacity to a cloud VM cluster
This topic describes how to use the Oracle Cloud Infrastructure (OCI) Console to add the new capacity to your cloud VM cluster.
This procedure does not apply to Multi-VM enabled Infrastructure
If you have used the task To add compute and storage resources to a flexible cloud Exadata infrastructure resource by adding additional database (compute) or storage servers to the service instance, you must add the additional capacity to the cloud VM cluster to utilize the additional resources.
- Open the navigation menu. Under Oracle Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Under Oracle Exadata Database Service on Dedicated Infrastructure, click Exadata VM Clusters.
- In the list of cloud VM clusters, click the name of the cluster to which you want to add capacity.
- On the VM Cluster Details page, click Scale VM Cluster.
- If you have additional capacity available as a result of scaling the cloud Exadata infrastructure resource, a banner at the top of the Scale VM Cluster panel provides a message telling you the type and amount of additional capacity available to the VM cluster. Check the Add Capacity box.
- Select either the Add Database Server or the Add Storage radio button, depending on which type of capacity you want to add to the cloud VM cluster.
- Click Update. The cloud VM cluster goes into the Updating state. When the capacity has been successfully added, the cluster returns to the Available state.
If you have added additional database servers to the cluster, you can allocate the new CPU cores once the cluster is in the Available state by clicking the Scale VM Cluster button again. See To scale CPU cores in an Exadata Cloud Service cloud VM cluster or DB system for more information on adding CPU cores to your cloud VM cluster.
Using the Console to Enable, Partially Enable, or Disable Diagnostics Collection
You can enable, partially enable, or disable diagnostics collection for your Guest VMs after provisioning the VM cluster. Enabling diagnostics collection at the VM cluster level applies the configuration to all the resources such as DB home, Database, and so on under the VM cluster.
- You are opting in with the understanding that the list of events, metrics, and log files collected can change in the future. You can opt-out of this feature at any time.
- Oracle may add more metrics in the future, but if you have already chosen to collect metrics, you need not update your opt-in value. It will remain enabled/disabled based on your current preference.
- If you have previously opted in for incident log and trace file collection and decide to opt out when Oracle Cloud operations run a log collection job, then the job will run its course and will not cancel. Future log collections won't happen until you opt-in again to the incident logs and trace file collection option.
Related Topics
Using the Console to Update the License Type on a VM Cluster
To modify licensing, be prepared to provide values for the fields required for modifying the licensing information.
To add SSH keys to a VM cluster
The VM cluster exists, and you wish to add a another user which requires another SSH key.
- Open the navigation menu. Click Oracle Database, then click Oracle Exadata Database Service on Dedicated Infrastructure
- Choose your Compartment.
- Click Exadata VM Clusters.
- In the list of VM clusters, find the cluster you want to manage and click its highlighted name.
- Click Add SSH Keys.
- Select one of the following options:
-
Generate SSH key pair: Use this option to create a new SSH key pair. Click both Save Private Key and Save Public Key when using this option. The private key is downloaded to your local machine, and should be stored in a safe location. You cannot download another copy of the private key generated during this operation after completing the operation.
-
Upload SSH key files: Select this option to browse or drag and drop .pub files.
-
Paste SSH keys: Select this option to paste in individual public keys. To paste multiple keys, click + Another SSH Key, and supply a single key for each entry.
-
- Click Save Changes.
Using the Console to Stop, Start, or Reboot a VM Cluster Virtual Machine
Use the console to stop, start, or reboot a virtual machine.
Using the Console to Check the Status of a VM Cluster Virtual Machine
Review the health status of a VM cluster virtual machine.
Using the Console to Move a VM Cluster to Another Compartment
To change the compartment that contains your VM cluster on Exadata Cloud Infrastructure, use this procedure.
When you move a VM cluster, the compartment change is also applied to the virtual machines and databases that are associated with the VM cluster. However, the compartment change does not affect any other associated resources, such as the Exadata infrastructure, which remains in its current compartment.
To change the VM cluster display name
This topic only applies to Exadata Cloud Infrastructure instances using the new Exadata Cloud Infrastructuree instance resource model.
- Open the navigation menu. Click Oracle Database, then click Oracle Exadata Database Service on Dedicated Infrastructure
- Choose your Compartment.
- Click Exadata VM Clusters under Oracle Exadata Database Service on Dedicated Infrastructure.
-
In the list of Exadata VM Clusters resources, click the name of the VM Cluster you're interested in
- On rthe Infrastructure Details page, click More Actions and Update Display Name .
- In the Update Display Name dialog, Enter the New display name, and the current display name as instructed.
- Click Update Display Name.
Using the Console to Terminate a VM Cluster
Before you can terminate a VM cluster, you must first terminate the databases that it contains.
You cannot terminate a VM cluster from an infrastructure with less than 5 storage servers
To view details about private DNS configuration
- Open the navigation menu. Under Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Choose the Region that contains your Exadata infrastructure.
- Choose the Compartment that contains your Exadata infrastructure.
- Click VM Clusters.
- Click the name of the VM cluster that is configured with a private DNS you want to view.
- Under the Network section, Private DNS and Private Zone are displayed, if a private DNS is configured.
- Click the Private View name to edit the configuration.
Related Topics
To Attach a Virtual IP Address
Attach a Virtual IP address from a VM cluster using this procedure.
- Open the navigation menu. Under Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Choose the Region that contains your Exadata infrastructure.
- Choose the Compartment that contains your Exadata infrastructure.
- Click VM Clusters.
- Under the Resources, click Virtual IP Address.
- Click Attach Virtual IP Address.
- In the Attach Virtual IP Address dialog:
- Select a subnet from the Subnet drop-down list.
- Enter a hostname for the Virtual IP Address in the Virtual IP Address Hostname field.
- (Optional) Enter an IP address in the Virtual IP address field.
- (Optional) You can enter a VIrtual Machine name to be the default attachment in the Virtual Machine field.
- Click Attach.
To Detach a Virtual IP Address
Attach a Virtual IP address from a VM cluster using this procedure.
- Open the navigation menu. Under Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Choose the Region that contains your Exadata infrastructure.
- Choose the Compartment that contains your Exadata infrastructure.
- Click VM Clusters.
- Under the Resources, click Virtual IP Address.
- Click the Actions icon (three dots) to the right of the Virtual IP Address you wish to detach.
- Click Detach.
- In the Detach Virtual IP Address dialog, confirm by entering the VIP Address that you wish to detach and click Detach.
Overview of Automatic Diagnostic Collection
By enabling diagnostics collection and notifications, Oracle Cloud Operations and you will be able to identify, investigate, track, and resolve guest VM issues quickly and effectively. Subscribe to Events to get notified about resource state changes.
-
Enable Diagnostic Events
Allow Oracle to collect and publish critical, warning, error, and information events to you. For more information, see Database Service Events.
-
Enable Health Monitoring
Allow Oracle to collect health metrics/events such as Oracle Database up/down, disk space usage, and so on, and share them with Oracle Cloud operations. You will also receive notification of some events. For more information, see Health Metrics.
-
Enable Incident Logs and Trace Collection
Allow Oracle to collect incident logs and traces to enable fault diagnosis and issue resolution. For more information, see Incident Logs and Trace Files.
Diagnostics Collection is:
- Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files (all three options).
- Disabled: When you choose not to collect diagnostics, health metrics, incident logs, and trace files (all three options).
- Partially Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files (one or two options).
Disabling diagnostic events and health monitoring will only stop the collection and notification of data/events from the time you uncheck the checkboxes tied to the options. However, historical data will not be purged from Oracle Cloud Operations data repositories.
Incident Logs and Trace Files
This section lists all of the files that can be collected by Oracle Support if you opt-in for incident logs and trace collection.
- Oracle will create a service request (SR) against the infrastructure Customer Support Identifier (CSI) when an issue is detected and needs customer interaction to resolve.
- The customer's Oralce CLoud Infrastructure tenancy admin email will be used as the CSI contact to create SR and attach logs to it. Ensure tenancy admin is added as a CSI contact in My Oracle Support (MOS).
Oracle Trace File Analyze (TFA) Component Driven Logs Collections
The directories are generally assigned to a component and that component can then be used to guide TFA to the files it needs to collect, for example, requesting the CRS component would tell TFA to look at directories mapped to the CRS component and find files that match the required collection time frame.
If have previously opted in for incident log and trace file collection and decide to opt out when Oracle Cloud operations run a log collection job, then the job will run its course and will not cancel. Future log collections won't happen until you opt-in again to the incident logs and trace file collection option.
TFA is shipped with scripts that run when a particular component
is requested, for example, for CRS component,
crscollect.pl
will run a number
of crsctl
commands and gather the input. By
default, TFA does not redact collected logs.
Table 5-2 Oracle Trace File Analyze (TFA) Component Driven Logs Collections
Component | Script | Files/Directories |
---|---|---|
|
|
|
|
|
|
|
No DB Specific Script - runs |
|
Cloud Tool Logs
- Creg files:
/var/opt/oracle/creg/*.ini
files with masked sensitive info - Cstate file:
/var/opt/oracle/cstate.xml
-
Database related tooling logs:
If
dbName
specified,/var/opt/oracle/log/<dbName>
, else collect logs for all databases/var/opt/oracle/log/
If
dbName
specified,/var/opt/oracle/dbaas_acfs/log/<dbName>
, else collect logs for all databases/var/opt/oracle/log/<dbName>
- Database env files: If
dbName
specified,/home/oracle/<dbName>.env
, else collect logs for all databases/home/oracle/*.env
- Pilot logs:
/home/opc/.pilotBase/logs
- List of log directories:
/var/opt/oracle/log
/var/opt/oracle/dbaas_acfs/log
/var/opt/oracle/dbaas_acfs/dbsystem_details
/var/opt/oracle/dbaas_acfs/job_manager
/opt/oracle/dcs/log
DCS Agent Logs
/opt/oracle/dcs/log/
Tooling-Related Grid Infrastructure/Database Logs
- Grid Infrastructure:
GI_HOME/cfgtoollogs
- Database alertlog:
/u02/app/oracle/diag/rdbms/*/*/alert*.log
Health Metrics
Review the list of database and non-database health metrics collected by Oracle Trace File Analyzer.
Oracle may add more metrics in the future, but if you have already chosen to collect metrics, you need not update your opt-in value. It will remain enabled/disabled based on your current preference.
Guest VM Health Metrics List - Database Metrics
Table 5-3 Guest VM Health Metrics List - Database Metrics
Metric Name | Metric Display Name | Unit | Aggregation | Interval | Collection Frequency | Description |
---|---|---|---|---|---|---|
|
CPU Utilization |
Percentage |
Mean |
One minute |
Five minutes |
The CPU utilization is expressed as a percentage, which is aggregated across all consumer groups. The utilization percentage is reported with respect to the number of CPUs the database is allowed to use, which is two times the number of OCPUs. |
|
Storage Utilization |
Percentage |
Mean |
One hour |
One hout |
The percentage of provisioned storage capacity currently in use. Represents the total allocated space for all tablespaces. |
|
DB Block Changes |
Changes per second |
Mean |
One minute |
Five minutes |
The Average number of blocks changed per second. |
|
Execute Count |
Count |
Sum |
One minute |
Five minutes |
The number of user and recursive calls that executed SQL statements during the selected interval. |
|
Current Logons |
Count |
Sum |
One minute |
Five minutes |
The number of successful logons during the selected interval. |
|
Transaction Count |
Count |
Sum |
One minute |
Five minutes |
The combined number of user commits and user rollbacks during the selected interval. |
|
User Calls |
Count |
Sum |
One minute |
Five minutes |
The combined number of logons, parses, and execute calls during the selected interval. |
|
Parse Count |
Count |
Sum |
One minute |
Five minutes |
The number of hard and soft parses during the selected interval. |
|
Storage Space Used |
GB |
Max |
One hour |
One hour |
Total amount of storage space used by the database at the collection time. |
|
Storage Space Allocated |
GB |
Max |
One hour |
One hour |
Total amount of storage space allocated to the database at the collection time. |
|
Storage Space Used By Tablespace |
GB |
Max |
One hour |
One hour |
Total amount of storage space used by tablespace at the collection time. In the case of container databases, this metric provides root container tablespaces. |
|
Allocated Storage Space By Tablespace |
GB |
Max |
One hour |
One hour |
Total amount of storage space allocated to the tablespace at the collection time. In the case of container databases, this metric provides root container tablespaces. |
|
Storage Space Utilization By Tablespace |
Percentage |
Mean |
One hour |
One hour |
This indicates the percentage of storage space utilized by the tablespace at the collection time. In the case of container databases, this metric provides root container tablespaces. |
Guest VM Health Metrics List - Non-Database Metrics
Table 5-4 Guest VM Health Metrics List - Non-Database Metrics
Metric Name | Metric Display Name | Unit | Aggregation | Collection Frequency | Description |
---|---|---|---|---|---|
|
ASM Diskgroup Utilization |
Percentage |
Max |
10 minutes |
Percentage of usable space used in a Disk Group. Usable space is the space available for growth. DATA disk group stores our Oracle database files. RECO disk group contains database files for recovery such as archives and flashback logs. |
|
Filesystem Utilization |
Percentage |
Max |
One minute |
Percent utilization of provisioned filesystem. |
|
CPU Utilization |
Percentage |
Mean |
One minute |
Percent CPU utilization. |
|
Memory Utilization |
Percentage |
Mean |
One minute |
Percentage of memory available for starting new applications, without swapping.
The available memory can be obtained via the following command: |
|
Swap Utilization |
Percentage |
Mean |
One minute |
Percent utilization of total swap space. |
|
Load Average |
Number |
Mean |
One minute |
System load average over 5 minutes. |
|
Node Status |
Integer |
Mean |
One minute |
Indicates whether the host is reachable. |
|
OCPU Allocated |
Integer |
Max |
One minute |
The number of OCPUs allocated. |
Using the API to Manage Exadata Cloud Infrastructure Instance
For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.
Use these API operations to manage Exadata Cloud Infrastructure instance components.
Cloud Exadata infrastructure resource (new resource model):
- ListCloudExadataInfrastructures
- GetCloudExadataInfrastructure
- ChangeCloudExadataInfrastructureCompartment
- UpdateCloudExadataInfrastructure
- DeleteCloudExadataInfrastructure
Cloud VM cluster (new resource model)
- ListCloudVmClusters
- GetCloudVmCluster
- ChangeCloudVmClusterCompartment
- UpdateCloudVmCluster
- DeleteCloudVmCluster
DB systems (old resource model):
Virtual machines nodes (all Exadata Cloud Infrastructure instances):
Parent topic: Manage VM Clusters
Troubleshooting Virtual Machines Using Console Connections
You can troubleshoot malfunctioning virtual machines using console connections. For example, a previously working Guest VM stops responding.
Exadata System Software 23.1.13 is the minimum required version. Also, make sure to review all prerequisites stated below, including setting a password for either the
opc
or the root
user. Failure to make necessary changes to meet these requirements in advance will result in the inability to urgently connect to the serial console when the need arises when the VM is not otherwise accessible.
To connect to a running instance for administration and general use, use a Secure Shell (SSH). For more information, see Connecting to a Virtual Machine with SSH.
To make an SSH connection to the serial console, follow these configuration steps.
- Ensure that you have the correct permissions.
- Complete the prerequisites, including creating your SSH key pair (in case you don't have one yet).
- Create the Virtual Machine Serial Console.
- Connect to the serial console via SSH.
To check the DB server version installed, follow these steps:
- Open the navigation menu. Under Oracle Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Choose your Compartment.
- Click Exadata Infrastructure under Oracle Exadata Database Service on Dedicated Infrastructure.
- Click the name of the infrastructure that you are interested in.
- In the resulting Infrastructure Details page, go to the Version section to find the DB Server version installed.
- Required IAM Policies
An administrator must grant you secure access to the virtual machine console on the Oracle Exadata Database Service on Dedicated Infrastructure system through an IAM policy. - Prerequisites
You must install an SSH client and create SSH key pairs. - Create the Virtual Machine Serial Console Connection
Before you can make a local connection to the serial console, you need to create the virtual machine console connection. - Make an SSH Connection to the Serial Console
- Troubleshooting Virtual Machines from Guest VM Console Connections on Linux Operating Systems
- Exiting the Virtual Machine Serial Console Connection
Parent topic: Manage VM Clusters
Required IAM Policies
An administrator must grant you secure access to the virtual machine console on the Oracle Exadata Database Service on Dedicated Infrastructure system through an IAM policy.
This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tools. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment to work in.
To create virtual machine console connections, an administrator needs to grant user access to read and manage virtual machine console connections through an IAM policy. The resource name for virtual machine console connections is dbnode-console-connection
. The resource name for the virtual machine is db-nodes
. The following policies grant users the ability to create virtual machine console connections:
Allow group <group_name> to manage dbnode-console-connection in tenancy
Allow group <group_name> to read db-nodes in tenancy
Prerequisites
You must install an SSH client and create SSH key pairs.
Install an SSH Client and a Command-line Shell (Microsoft Windows)
Microsoft Windows does not include an SSH client by default. If you are connecting from a Windows client, you need to install an SSH client. You can use PuTTY plink.exe with Windows PowerShell or software that includes a version of OpenSSH such as:
The instructions in this topic frequently use PuTTY and Windows PowerShell.
If you want to make the console connection from Windows with Windows PowerShell, PowerShell might already be installed on your Windows operating system. If not, follow the steps at the link. If you are connecting to the instance from a Windows client using PowerShell, plink.exe is required. plink.exe is the command link connection tool included with PuTTY. You can install PuTTY or install plink.exe separately. For installation information, see http://www.putty.org.
Create SSH Key Pairs
To create the secure console connection, you need an SSH key pair. The method to use for creating key pairs depends on your operating system. When connecting to the serial console, you must use an RSA key. The instructions in this section show how to create an RSA SSH key pair.
Create the SSH key Pair for Linux
If you're using a UNIX-style system, you probably already have the ssh-keygen
utility installed. To determine whether the utility is installed, type ssh-keygen
on the command line. If the utility isn't installed, you can download OpenSSH for UNIX from http://www.openssh.com/portable.html and install it.
- Open a shell or terminal for entering the commands.
- At the prompt, enter
ssh-keygen
and provide a name for the key when prompted. Optionally, include a passphrase.The keys will be created with the default values: RSA keys of 2048 bits.
Alternatively, you can type a completessh-keygen
command, for example:ssh-keygen -t rsa -N "" -b 2048 -C "<key_name>" -f <path/root_name>
Argument Description -t rsa
Use the RSA algorithm. -N "<passphrase>"
A passphrase to protect the use of the key (like a password). If you don't want to set a passphrase, don't enter anything between the quotes. A passphrase is not required. You can specify one as a security measure to protect the private key from unauthorized use. If you specify a passphrase, when you connect to the instance you must provide the passphrase, which typically makes it harder to automate connecting to an instance.
-b 2048
Generate a 2048-bit key. You don't have to set this if 2048 is acceptable, as 2048 is the default. A minimum of 2048 bits is recommended for SSH-2 RSA.
-C "<key_name>"
A name to identify the key. -f <path/root_name> The location where the key pair will be saved and the root name for the files.
Create the SSH Key Pair for Windows Using PuTTY
If you are using a Windows client to connect to the instance console connection, use an SSH key pair generated by PuTTY.
- Find
puttygen.exe
in the PuTTY folder on your computer, for example,C:\Program Files (x86)\PuTTY
. Double-clickputtygen.exe
to open it. - Specify a key type of SSH-2 RSA and a key size of 2048 bits:
- In the Key menu, confirm that the default value of SSH-2 RSA key is selected.
- For the Type of key to generate, accept the default key type of RSA.
- Set the Number of bits in a generated key to 2048 if not already set.
- Click Generate.
- To generate random data in the key, move your mouse around the blank area in the PuTTY window.
When the key is generated, it appears under Public key for pasting into OpenSSH authorized_keys file.
- A Key comment is generated for you, including the date and timestamp. You can keep the default comment or replace it with your own more descriptive comment.
- Leave the Key passphrase field blank.
- Click Save private key, and then click Yes in the prompt about saving the key without a passphrase.
The key pair is saved in the PuTTY Private Key (PPK) format, which is a proprietary format that works only with the PuTTY tool set.
You can name the key anything you want but use the
ppk
file extension. For example,mykey.ppk
. - Select all of the generated keys that appear under Public key for pasting into OpenSSH
authorized_keys
file, copy it using Ctrl + C, paste it into a text file, and then save the file in the same location as the private key.Note
Do not use the Save public key option because it does not save the key in the OpenSSH format.You can name the key anything you want, but for consistency, use the same name as the private key and a file extension of
pub
. For example:mykey.pub
. - Write down the names and locations of your public and private key files. You need the public key when creating an instance console connection. You need the private key to connect to the instance console connection using PuTTY. For example:
$HOME\Documents\mykey.ppk
.
To create a connection using the SSH key pair generated using PuTTY
For more information about generating SSH key pairs, see Create the SSH Key Pair for Windows Using PuTTY
Do the following on the Create serial console access window:
- Paste the SSH Key generated from OpenSSH format or choose Upload SSH key file and provide the path of the public key saved at step 8 in Create the SSH Key Pair for Windows Using PuTTY.
- Once the connection is Active, click Copy serial console connection for Windows.
- Paste the connection string copied from the previous step into a text file.
- In the text file, replace
<
to point to your PuTTY Private Key (PPK) file path on your computer. For example, if you have savedPATH_FILE_PUTTY_PRIVATE.ppk
>.ppk
file at$HOME\Documents\mykey.ppk
. - Paste the modified connection string into the PowerShell window, and then press Enter to connect to the console.
Sign in to a Virtual Machine From the Serial Console
If you want to sign in to a virtual machine using a virtual machine console connection, you can use Secure Shell (SSH) connection to sign in. If you want to sign in with a username and password, you need a user account with a password. Oracle Exadata Cloud does not set a default password for the opc
or root
users. Therefore, if you want to sign in as the opc
or root
user, you need to create a password for the opc
or root
user. Otherwise, add a different user with a password and sign in as that user. This should be completed in advance, before a potential situation that might require you to log in to the serial console.
Connect Through Firewalls
If the client you will use to access the serial console is behind a firewall, you must ensure that this client can reach the required endpoint to access the serial console of the virtual machine. The client system connecting to the serial console must be able to reach the serial console server (for example, vm-console-ad1.exacs.us-ashburn-1.oci.oraclecloud.com
) over SSH using port 443, directly or through a proxy.
Create the Virtual Machine Serial Console Connection
Before you can make a local connection to the serial console, you need to create the virtual machine console connection.
channel 0: open failed: administratively prohibited: console access is limited to one connection at a time
- Open the navigation menu. Under Oracle Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Click the VM Cluster that you're interested in.
- In the resulting VM Cluster Details page, click the name of the virtual machine that you're interested in.
Under Resources, Console connection is selected by default.
- Click Create serial console access.
- In the resulting Create serial console access window, you have three options for adding the SSH key
- Generate a key pair for me: You can have Oracle Cloud Infrastructure generate an SSH key pair to use. If you are using PowerShell or PuTTY to connect to the instance from a Windows client, you cannot use the generated SSH key pair without first converting it to a
.ppk
file. - Upload public key file: Browse to a public key file on your computer. If you followed the steps in Creating SSH Key Pairs in the Prerequisites section to create a key pair, use this option to navigate to the
.pub
file. - Paste public key: Paste the content of your public key file into the text box.
- Generate a key pair for me: You can have Oracle Cloud Infrastructure generate an SSH key pair to use. If you are using PowerShell or PuTTY to connect to the instance from a Windows client, you cannot use the generated SSH key pair without first converting it to a
- Click Create console connection.
When the console connection has been created and is available, the state changes to Active.
Make an SSH Connection to the Serial Console
After you create the console connection for the virtual machine, you can connect to the serial console using a Secure Shell (SSH) connection. When making an SSH connection to the serial console, you must use an RSA key. You can use the same SSH key for the serial console that was used when you launched the instance, or you can use a different SSH key.
When you are finished with the serial console and have terminated the SSH connection, you should delete the serial console connection. If you do not disconnect from the session, Oracle Cloud Infrastructure terminates the serial console session after 24 hours and you must reauthenticate to connect again.
Validate Server Host Keys
When you first connect to the serial console, you're prompted to validate the fingerprint of the server host key. The fingerprint of the server host key is the SHA256 hash of the server host's public SSH key. The server SSH handshake response is signed with the associated private key. Validating the server host key's fingerprint protects against potential attacks.
When you make a manual connection to the serial console, the fingerprint of the server host key is not automatically validated. To manually validate the fingerprint, compare the fingerprint value displayed in the Oracle Cloud Infrastructure Console to the value of the RSA key fingerprint that appears in the terminal when you connect.
To find the fingerprint of the server host key in the Console, on the Virtual Machine details page, under Resources, click Console connection. The table displays the fingerprint of the server host key. The fingerprint in the Console should match the value of the RSA key fingerprint shown in the terminal when you connect to the serial console.
The server host keys are periodically rotated for security purposes. Key rotation reduces the risk posed when keys are compromised by limiting the amount of data encrypted or signed by one key version. When your key is rotated and you try to connect to the serial console, a warning appears indicating a potential attack. The warning includes an Host key verification failed error and a line number in your .ssh/known_hosts
file. Delete that line in your .ssh/known_hosts
file and then reconnect to the serial console. You are then prompted to accept a new server host key fingerprint.
Connect from Mac OS X and Linux Operating Systems
Use an SSH client to connect to the serial console. Mac OS X and most Linux and UNIX-like operating systems include the SSH client OpenSSH by default.
To connect to the serial console using OpenSSH on Mac OS X or Linux:
- Open the navigation menu. Under Oracle Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Click the VM Cluster that you're interested in.
- In the resulting VM Cluster Details page, click the name of the virtual machine that you're interested in.
- On the Virtual Machine details page in the Oracle Cloud Infrastructure Console, under Resources, click Console connection.
- Click the Actions menu (three dots), and then click Copy serial console connection for Linux/Mac.
- Paste the connection string into a terminal window on a Mac OS X or Linux system, and then press Enter to connect to the console.
If you are not using the default SSH key or ssh-agent, modify the serial console connection string to include the identity file flag,
-i
to specify the private key portion for the SSH key to use, for example,id_rsa
. Specify this flag for both the SSH connection and the SSH Proxy Command, as shown in the following line:ssh -i /<path>/<ssh_key> -o ProxyCommand='ssh -i /<path>/<ssh_key> -W %h:%p -p 443...
- If prompted, validate and accept the fingerprint of the server host key.
If you have previously accepted a fingerprint for the server host key but the key has been rotated, a warning appears indicating a potential attack. The warning includes an Host key verification failed error and a line number in your
.ssh/known_hosts
file. Delete the specified line in your.ssh/known_hosts
file and then reconnect to the serial console. Validate and accept the new server host key fingerprint. - Press Enter again to activate the console.
If the connection is active, a message appears in the console:
================================================= IMPORTANT: You are now connected to the serial console for this VM. This should be used in emergency situations only. See product documentation for more details and alternative connectivity options for normal operations =================================================
- Reboot your virtual machine.
You do not need to enter a user name or password. If the Virtual Machine is functional and the connection is active, the serial output appears in your console. If the serial output does not appear in the console, the Guest VM operating system is not booting.
For more troubleshooting options, see Troubleshooting Virtual Machines from Guest VM Console Connections on Linux Operating Systems.
- Go to the ExaDB-D VM Cluster Details page.
- Under Resources, click Virtual Machines.
- Select Reboot from the Actions menu (three dots) for the virtual machine that you want to reboot.
Connect from Windows Operating Systems
The steps to connect to the serial console from Microsoft Windows PowerShell are different from the steps for OpenSSH. The following steps do not work in the Windows terminal.
If you are connecting to the instance from a Windows client using PowerShell, plink.exe
is required. plink.exe
is the command link connection tool included with PuTTY. You can install PuTTY or install plink.exe
separately. For more information, see Installing an SSH Client and a Command-line Shell (Windows).
To connect to the serial console on Microsoft Windows:
- On the Virtual Machine details page in the Oracle Cloud Infrastructure Console, under Resources, click Console connection.
- Click the Actions menu (three dots).
Depending on which SSH client you are using, do one of the following:
- If you are using Windows PowerShell, click Copy serial console connection for Windows.
- If you are using OpenSSH, click Copy serial console connection for Linux/Mac.
Note
The copied connection string for Windows contains the parameter-i
specifying the location of the private key file. The default value for this parameter in the connection string references an environment variable that might not be configured on your Windows client, or it might not represent the location where the private key file is saved. Verify the value specified for the-i
parameter and make any required changes before proceeding to the next step. - Paste the connection string copied from the previous step into a text file so that you can add the file path to the private key file.
- In the text file, replace
$env:homedrive$env:homepath\oci\console.ppk
with the file path to the.ppk
file on your computer. This file path appears twice in the string. Replace it in both locations. - Paste the modified connection string into the PowerShell window or your OpenSSH client, and then press Enter to connect to the console.
- If prompted, validate and accept the fingerprint of the server host key.
If you have previously accepted a fingerprint for the server host key, but the key has been rotated, a warning appears indicating a potential attack. The warning includes a Host key verification failed error and a line number in your
.ssh/known_hosts
file. Delete the specified line in your.ssh/known_hosts
file and then reconnect to the serial console. Validate and accept the new server host key fingerprint. - Press Enter again to activate the console.
- Reboot your virtual machine.
You do not need to enter a user name or password. If the Virtual Machine is functional and the connection is active, the serial output appears in your console. If the serial output does not appear in the console, the Guest VM operating system is not booting.
For more troubleshooting options, see Troubleshooting Virtual Machines from Guest VM Console Connections.
- Go to the ExaDB-D VM Cluster Details page.
- Under Resources, click Virtual Machines.
- Select Reboot from the Actions menu (three dots) for the virtual machine that you want to reboot.
To create a connection using the SSH key pair generated using the OCI Console:
Do the following on the Create serial console access window:
- Click Generate a key pair for me.
- Click Save Private Key.
- Click Create console connection.
- Find
puttygen.exe
in the PuTTY folder on your computer, for example,C:\Program Files (x86)\PuTTY. Double-click puttygen.exe
to open it. - On the PuTTY Key Generator, click the Conversions menu and then click Import.
- On the Windows Explorer, select OCI Console generated SSH key (step 1) and then click Open.
PuTTY imports the key and displays information about the key on the PuTTY Key Generator window.
- Click Save private key.
- Click Yes when prompted about saving the key without a passphrase.
The key pair is saved in the PuTTY Private Key (PPK) format, which is the proprietary format that works only with the PuTTY tool set.
You can name the key anything you want but use the
.ppk
file extension. For example,$HOME\Desktop\key-vm-console.ppk
. - Use a text editor to change the command to point to your PuTTY Private Key (PPK) path. Replace
<PATH_FILE_PUTTY_PRIVATE.ppk>
to point to your PuTTY Private Key (PPK) file path on your computer. For example, if you have saved.ppk
file at$HOME\Desktop\key-vm-console.ppk
. - Paste the modified connection string into the PowerShell window, and then press Enter to connect to the console.
To convert a generated .key private key file:
- Open PuTTYgen.
- Click Load, and select the private key generated when you created the instance.
The extension for the key file is
.key
. - Click Save private key.
- Specify a name for the key.
The extension for the new private key is
.ppk
. - Click Save.
Troubleshooting Virtual Machines from Guest VM Console Connections on Linux Operating Systems
After you are connected with an instance console connection, you can perform various tasks, such as:
- Edit system configuration files.
- Add or reset the SSH keys for the
opc
user. - Reset the password for the
opc
user.
These tasks require you to boot into a Bash shell in maintenance mode.
To boot into maintenance mode
Default user and password:
- Account: Grub boot loader
- Username: root
- Default Password: sos1Exadata
- Account Type: Operating system user
For more information, see Default User Accounts for Oracle Exadata.
- Reboot the VM from the VM Cluster.
- For virtual machines running Oracle Linux 7.x or Oracle Linux 8.x, when the reboot process starts, switch back to the terminal window, and you see Console messages start to appear in the window. As soon as the GRUB boot menu appears, use the up/down arrow key to stop the automatic boot process, enabling you to use the boot menu.
- In the boot menu, highlight the top item in the menu, and press e to edit the boot entry.
- In edit mode, use the down arrow key to scroll down through the entries until you reach the line that starts with linux16.
- At the end of that line, add the following:
init=/bin/bash
- Reboot the instance from the terminal window by entering the keyboard shortcut CTRL+X.
When the instance has rebooted, you see the Bash shell command-line prompt, and you can proceed with the following procedures.
To edit the system configuration files
- From the Bash shell, run the following command to load the SElinux policies to preserve the context of the files you are modifying:
/usr/sbin/load_policy -i
- Run the following command to remount the root partition with read/write permissions:
/bin/mount -o remount, rw /
- Edit the configuration files as needed to try to recover the instance.
- After you have finished editing the configuration files, to start the instance from the existing shell, run the following command:
Alternatively, to reboot the instance, run the following command:exec /usr/lib/systemd/systemd
/usr/sbin/reboot -f
To add or reset the SSH key for the opc user
- From the Bash shell, run the following command to load the SElinux policies to preserve the context of the files you are modifying:
/usr/sbin/load_policy -i
- Run the following command to remount the root partition with read/write permissions:
/bin/mount -o remount, rw /
- From the Bash shell, run the following command to change to the SSH key directory for the
opc
user:cd ~opc/.ssh
- Include your public key entry to the
authorized_keys
file.Note
You can edit the file and remove your previous key if you want to. However, make sure to keep the cloud automation keys to prevent cloud automation from breaking.echo '<contents of public key file>' >> authorized_keys
- Restart the instance by running the following command:
/usr/sbin/reboot -f
To reset the password for the opc user
- From the Bash shell, run the following command to load the SElinux policies to preserve the context of the files you are modifying.
This step is necessary to sign in to your instance using SSH and the Console.
/usr/sbin/load_policy -i
- Run the following command to remount the root partition with read/write permissions:
/bin/mount -o remount, rw /
- Run the following command to reset the password for the
opc
user:sudo passwd opc
- Restart the instance by running the following command:
sudo reboot -f
Note
Setting aroot
password would be an acceptable alternative to setting anopc
password.
Exiting the Virtual Machine Serial Console Connection
To exit the serial console connection
When using SSH, the ~
character at the beginning of a new line is used as an escape character.
- To exit the serial console, enter:
~.
- To suspend the SSH session, enter:
~^z
The
^
character represents the CTRL key. - To see all the SSH escape commands, enter:
~?
To delete the serial console connection for a Virtual Machine
- Open the navigation menu. Under Oracle Database, click Oracle Exadata Database Service on Dedicated Infrastructure.
- Click the VM Cluster that you're interested in.
- In the resulting VM Cluster Details page, click the name of the virtual machine that you're interested in.
Under Resources, Console connection is selected by default.
- Click the Actions menu, and then click Delete. Confirm when prompted.