1. Administering Oracle Database Exadata Cloud at Customer (Gen 1/OCI-C)
  2. Administering Exadata Cloud at Customer
  3. Administering VM Clusters

Administering VM Clusters

If your Exadata system environment is enabled to support multiple virtual machine (VM) clusters, then you can define up to 8 clusters and specify how the overall Exadata system resources are allocated to them.

Topics

Viewing Information About Existing VM Clusters

To view information about existing VM clusters:

  1. Open the My Services dashboard.

    For detailed instructions, see Accessing the My Services Dashboard and the Oracle Database Cloud Service Console.

  2. Click the action menu (Menu icon) in the tile that is associated with Exadata Cloud at Customer and then choose View Details.

    The Service Details page is displayed, with the Overview tab showing.

  3. Locate your service instance in the list. Click the action menu (Menu icon) located beside the service instance name and choose Manage Clusters.

    The cluster administration page is displayed, which contains two tables:

    • The VM Clusters table displays information about all of the VM clusters that are defined on your Exadata Cloud at Customer system environment. Each row in the table outlines the resource allocation, configuration options, and status for an existing VM cluster.

    • The Resource Overview table outlines:

      • The resources that are allocated to existing VM clusters.

      • The resources that are not allocated to existing VM clusters and are available to be allocated to new VM clusters.

      • The total resources that are contained in your Exadata Cloud at Customer system environment.

Creating a VM Cluster

To create a VM cluster:

  1. Open the My Services dashboard.

    For detailed instructions, see Accessing the My Services Dashboard and the Oracle Database Cloud Service Console.

  2. Click the action menu (Menu icon) in the tile that is associated with Exadata Cloud at Customer and then choose View Details.

    The Service Details page is displayed, with the Overview tab showing.

  3. Locate your service instance in the list. Click the action menu (Menu icon) located beside the service instance name and choose Manage Clusters.

    The cluster administration page is displayed.

  4. In the cluster administration page, click Create VM Cluster.

    The Create VM Cluster dialog is displayed.

  5. Use the Create VM Cluster dialog to specify the following resource allocations and configuration options for the new VM cluster, factoring in any plans for more VM clusters. Then, click Create and confirm your intention in the resulting dialog.

    • Cluster Name — enter a name for the VM cluster.

    • Database backups on Exadata Storage — check this option to configure the Exadata storage to enable local database backups.

      Note:

      Take care when setting this option because your choice has a profound effect on the storage allocation and your backup options, which cannot be easily changed. See Exadata Storage Configuration for more information about the effects of each configuration alternative.

    • Create sparse disk group? — check this option to create a disk group that is based on sparse grid disks. You must select this option to enable Exadata Cloud at Customer snapshots. Exadata snapshots enable space-efficient clones of Oracle databases that can be created and destroyed very quickly and easily.

      Note:

      Take care when setting this option because your choice has a profound effect on the storage allocation and your ability to use snapshots, which cannot be easily changed. See Exadata Storage Configuration for more information about the effects of each configuration alternative. See also Creating and Managing Snapshots of a Database Deployment.

    • Database backups on ZDLRA — check this option to enable database backups on Oracle Zero Data Loss Recovery Appliance (ZDLRA) storage. If you do not select this option, then you cannot select ZDLRA as a backup location when you configure a database deployment.

    • Exadata Storage (TB) — specifies the total amount of Exadata storage that is allocated to the VM cluster. This storage is allocated evenly from all of the Exadata Storage Servers. Specify a value greater than 3 TB and up to the amount of remaining unallocated Exadata storage space.

    • Client Network — specifies the client network subnet that is allocated to the VM cluster. Choose from the list of available subnets. The choices in the list are based on the network definitions that were defined during the provisioning of your Exadata Cloud at Customer system. Your choice also automatically selects the backup network subnet for the VM cluster, which is associated with the selected client network subnet.

    • CPU Cores — specifies the number of CPU cores that are allocated to each active node in the VM cluster. Specify a value greater than 2 and up to the number of remaining unallocated CPU cores.

    • Memory (GB) — specifies the amount of memory that is allocated to each active node in the VM cluster. Specify a value greater than 30 GB and up to the amount of remaining unallocated memory, factoring in any plans for more VM clusters.

      Note:

      Take care when specifying the memory allocation because:
      • After the VM cluster is created, you cannot decrease the memory allocation; however, you may increase the memory allocation by using unallocated memory.
      • You cannot create another VM cluster unless there is 30 GB of remaining unallocated memory. In that case, you would need to delete an existing VM cluster before you can create another one.

    • DB Home Storage (GB) — specifies the amount of local disk storage that is allocated to each active node in the VM cluster. Specify a value greater than 60 GB and up to the amount of remaining unallocated local storage space, factoring in any plans for more VM clusters.

      Note:

      Take care when specifying the local disk storage because:
      • In addition to the storage specified in this attribute, each VM cluster requires 137 GB of local disk storage to support software images for the VM cluster. Consequently, the minimum amount of local disk storage consumed by a VM cluster is 197 GB (137 GB + 60 GB).
      • For Exadata Cloud at Customer configurations based on Oracle Exadata X7 systems, the total amount of local disk storage that can be allocated to VM clusters is 1237 GB. For Exadata Cloud at Customer configurations based on Oracle Exadata X6 systems, the total amount of local disk storage that can be allocated to VM clusters is 483 GB by default, or up to 1237 GB on systems with upgraded local disk storage.
      • After the VM cluster is created, you cannot modify the amount of local storage.
      • If all of the local disk storage is allocated, or if there is not at least 197 GB of remaining unallocated local disk storage, then you cannot create another VM cluster. In that case, you must delete an existing VM cluster before you can create another one.

    • The check boxes beside each node (node-1, node-2, and so on) enable you to specify the active compute nodes in the VM cluster. Use the check boxes to specify the nodes that are included in the cluster.

    After you complete the Create VM Cluster dialog, a message is displayed indicating that the VM cluster is being created.

Modifying an Existing VM Cluster

Note:

  • Before you modify an existing VM cluster you must create a database deployment in the VM cluster. If you attempt to modify a VM cluster before creating any database deployment, then the operation fails.
  • Ensure that you do not modify an existing VM cluster while another modify operation is progressing. If you attempt to perform concurrent VM cluster modifications, then operations may fail.

To modify an existing VM cluster:

  1. Open the My Services dashboard.

    For detailed instructions, see Accessing the My Services Dashboard and the Oracle Database Cloud Service Console.

  2. Click the action menu (Menu icon) in the tile that is associated with Exadata Cloud at Customer and then choose View Details.

    The Service Details page is displayed, with the Overview tab showing.

  3. Locate your service instance in the list. Click the action menu (Menu icon) located beside the service instance name and choose Manage Clusters.

    The cluster administration page is displayed.

  4. Click the action menu (Menu icon) that is associated with the VM cluster and choose Modify.

  5. Use the Modify VM Cluster dialog to modify the resource allocation for any of the following attributes. You can only modify one attribute at a time. Other attributes are displayed for reference purposes only and cannot be modified. After you make the change, click Modify and confirm your intention in the resulting dialog.

    • CPU Cores per Node — specifies the number of CPU cores that are allocated to each node in the VM cluster. Specify a value greater than 2 and up to the number of remaining unallocated CPU cores.

      Modifying the number of CPU cores allocated to each node in the VM cluster is an online operation. Compute nodes are not rebooted because of this operation.

      The Oracle Database instance caging feature is enabled by explicitly setting the CPU_COUNT database initialization parameter. That setting is not affected by modifying the number of CPU cores that are allocated to the VM cluster. Therefore, if you are using the instance caging feature you must explicitly alter the CPU_COUNT setting to match changes to the VM cluster. If CPU_COUNT is set to 0 (its default setting), then instance caging is disabled and Oracle Database continuously monitors the number of CPUs reported by the operating system and uses the current count.

    • Memory (GB) per Node — specifies the amount of memory (in GB) that is allocated to each node in the VM cluster. Specify a value greater than the current allocation and up to the amount of remaining unallocated memory.

      To process a memory change, the associated compute nodes are rebooted in a rolling manner, one compute node at a time, to minimize impact on the cluster.

      Note:

      You can only use the Modify VM Cluster dialog to increase the memory allocation.

    • Exadata Storage (TB) — specifies the total amount of Exadata storage (in TB) that is allocated to the VM cluster. This storage is allocated evenly from all of the Exadata Storage Servers. Specify a value greater than 3 TB and up to the amount of remaining unallocated Exadata storage space.

      You may reduce the Exadata storage allocation for a VM cluster. However, you must ensure that the new amount covers the existing contents, and you should also allow for anticipated data growth.

      Modifying the Exadata storage allocated to the VM cluster is an online operation. Compute nodes are not rebooted because of this operation.

    • Add/Remove Nodes — indicates that you want to add a node to the VM cluster or remove a node from the VM cluster.

      After you select this option, use the check boxes beside each node (node-1, node-2, and so on) to specify the nodes that are included in the cluster. If you check a previously unchecked box, then the corresponding node is added to the VM cluster. If you uncheck a previously checked box, then the corresponding node is removed from the VM cluster.

      Node addition and removal are online operations. Unchanged compute nodes are not rebooted because of node addition or removal.

      Note the following requirements, recommendations, and restrictions for node addition and removal:

      • You must always have at least one node in the VM cluster.

      • You can only add or remove one node at a time. To add or remove multiple nodes you must use the Modify VM Cluster dialog multiple times.

      • Before you add a new compute node to an existing VM cluster:

        • Update the cloud-specific tooling on the existing compute nodes in the VM cluster. See Updating the Cloud Tooling on Exadata Cloud at Customer.
        • Undo any operating system customization on the existing compute nodes in the VM cluster. The newly added node will use a standard software image with settings that may cause performance or stability issues if added to a customized cluster. Examples of potentially problematic customization include changing the timezone and NTP settings.
        • If you have upgraded the Exadata software release in the cluster, then check with Oracle to ensure that the Exadata software release version in your compute node user domains (DomU) matches the Exadata system software version in the management domains (Dom0). If you attempt to add a node with mismatched software versions, then the operation fails.

        • Ensure that there is at least one running database instance for every existing database deployment on the VM cluster.

          If you do not follow this recommendation and add a compute node to a VM cluster while a database is completely shut down, then you need to run the following command before you can start the database on the added compute node:

          #  dbaascli database configureinstance --dbname dbname --node nodename

          In the preceding command, dbname specifies the name of the database and nodename specifies the hostname of the new compute node. The command must be run as the root user on a compute node that hosts a running instance of the database.

        • If the existing VM cluster was created using Exadata Cloud at Customer before release 18.4.6, then ensure that Oracle Grid Infrastructure is correctly configured to add the new compute node:

          1. Connect to a compute node in the cluster as the opc user and start a command shell as the root user.

            See Connecting to a Compute Node Through Secure Shell (SSH).

          2. Examine the contents of the configuration file at /var/opt/oracle/creg/grid/grid.ini:

            If the file contains entries for the Oracle Automatic Storage Management (ASM) SYS and ASMSNMP user passwords, then take note of the passwords and proceed to the next step. If present, the entries look like: 

            sys=password
asmsnmp=password

            If the file does not contain the ASM SYS and ASMSNMP user passwords, then look for the wallet_loc entry. If the wallet_loc entry is not present, then proceed to the next step and supply passwords of your choice when prompted. If the wallet_loc entry is present, then your cluster is correctly configured to add new compute nodes and you do not need to proceed to the next step.

          3. Run the following command twice, once to set the ASM SYS user password and again to set the ASM ASMSNMP user password: 

            # dbaascli grid changepasswd --dbname grid

            When prompted, supply the ASM user name (SYS or ASMSNMP) and password.

        • Ensure that the file permissions are correctly configured on the existing nodes.

          On every existing node in the VM cluster:

          1. Connect to the compute node as the opc user and start a command shell as the root user.

            See Connecting to a Compute Node Through Secure Shell (SSH).

          2. Become the grid user: 

            # su - grid
$

          3. Observe the ORACLE_HOME directory location for Oracle Grid Infrastructure. For example: 

            $ echo $ORACLE_HOME
/u01/app/12.2.0.1/grid

          4. Exit the grid user session: 

            $ exit
#

          5. As the root user, set an environment variable that references the ORACLE_HOME directory location for Oracle Grid Infrastructure that you observed previously. For example: 

            # export GRID_HOME=/u01/app/12.2.0.1/grid

          6. Run the following command sequence to set the required file privileges:

            # chgrp oinstall $GRID_HOME/srvm/admin/logging.properties
# chmod 0644 $GRID_HOME/srvm/admin/logging.properties
# chgrp oinstall $GRID_HOME/bin/oradaemonagent
# chmod 0644 $GRID_HOME/bin/oradaemonagent
# chgrp oinstall $GRID_HOME/bin/osdbagrp0
# chmod 0644 $GRID_HOME/bin/osdbagrp0
# chgrp oinstall $GRID_HOME/bin/rawutl0
# chmod 0644 $GRID_HOME/bin/rawutl0
# chgrp oinstall $GRID_HOME/bin/*.ouibak
# chmod 0644 $GRID_HOME/bin/*.ouibak

        • If the existing VM cluster uses Exadata software release 19.2, then ensure that host naming is correctly configured on the existing nodes. You can determine the current Exadata software release by executing the imageinfo command on any compute node.

          On every existing node in the VM cluster:

          1. Connect to the compute node as the opc user and start a command shell as the root user.

            See Connecting to a Compute Node Through Secure Shell (SSH).

          2. Edit /etc/nsswitch.conf to ensure that the hosts entry concludes with myhostname.

            For example, the line in /etc/nsswitch.conf containing: 

            hosts: files dns

            changes to:

            hosts: files dns myhostname

      • After you add a new compute node to an existing VM cluster:

      • Before you remove a compute node, note the following:

        • When you remove a compute node, all database instances on the node are deleted.
        • You cannot remove a node if doing so would delete a database instance that is the only preferred instance for an Oracle Database service.
        • You cannot remove a node while a backup operation is running on it.

Deleting a VM Cluster

To delete an existing VM cluster:

  1. Open the My Services dashboard.

    For detailed instructions, see Accessing the My Services Dashboard and the Oracle Database Cloud Service Console.

  2. Click the action menu (Menu icon) in the tile that is associated with Exadata Cloud at Customer and then choose View Details.

    The Service Details page is displayed, with the Overview tab showing.

  3. Locate your service instance in the list. Click the action menu (Menu icon) located beside the service instance name and choose Manage Clusters.

    The cluster administration page is displayed.

  4. Click the action menu (Menu icon) that is associated with the VM cluster and choose Delete. Then, confirm the action in the resulting dialog.

    A message is displayed indicating that the cluster is being deleted.