Note:

Create a Multi-Cluster Oracle Cloud VMware Solution Software-Defined Data Center

Introduction

Oracle Cloud VMware Solution provides a customer-managed, native VMware-based cloud environment, installed within a customer’s tenancy. It offers complete control using VMware tools. Users can move or extend VMware-based workloads to the cloud without rearchitecting applications or retooling operations.

Oracle Cloud VMware Solution offers both Intel and AMD processor-based Compute Shapes. The service has two types of compute shapes, Dense I/O and Standard compute shapes. Dense I/O shapes include locally-attached NVMe-based SSDs, which are used to create a VMware vSAN storage solution for the Software-Defined Data Center (SDDC). Whereas with Standard shapes users attach OCI Block Storage as the Primary Virtual Machine File System (VMFS) based Datastore. Standard Shapes allows users to configure the cores per instance as required with disaggregated Oracle Cloud Infrastructure (OCI) block and file storage. This unique advantage allows the scaling of storage independent of compute while utilizing the OCI native features for fault isolation and supported Block Volume Service capabilities. Refer to this blog to understand the advantages of Standard Shapes for Oracle Cloud VMware Solution.

Multi-Cluster SDDC

In this tutorial, we will walk you through the steps of creating an SDDC with multiple clusters, selecting the appropriate compute shapes, networking and storage options. The Unified Management cluster comprises three Dense I/O-shaped hosts with vSAN storage, while the Workload cluster utilizes three Standard-shaped hosts with OCI Block Volume as primary storage. Refer to the blog to learn more about Multi-cluster SDDC.

Objective

Prerequisites

Task 1: Create a multi-cluster SDDC

To deploy an SDDC, log in to your OCI tenancy and select the appropriate region. In this tutorial, we are selecting the Germany Central (Frankfurt) region.

  1. In your Oracle Cloud Console, open the navigation menu on the top-left corner by clicking the hamburger icon.

  2. Select Hybrid and then select Software-Defined Data Centers under VMware Solution.

    OCI Console

  3. Select the appropriate Compartment from the drop-down list where you wish to deploy the SDDC, and then click Create SDDC.

    Create SDDC

  4. On the Create SDDC page, you can notice the workflow is split into 3 sections. In the Basic information section, enter the following details. When the Basic information section is completed, click Next to continue to the Define clusters section.

    Create SDDC Workflow - Basic Information

    • SDDC Name: Provide a name for the SDDC. This should be a unique name for SDDCs across the whole tenancy and the length of the name cannot be more than 16 characters.

    • SDDC Compartment: The compartment chosen earlier should be shown by default. If required, change it to the appropriate compartment.

    • VMware software version: Select a software version as per your requirements. This version contains the corresponding VMware products such as vSphere, NSX and HCX. For more information about the supported versions and builds that are deployed with each version, see VMware Software for Oracle Cloud VMware Solution.

    • HCX: Users have a choice to install the HCX plugin. By default, the provisioning workflow deploys HCX with Advanced license for all SDDCs. Users can upgrade or downgrade the license types after the SDDC is created. HCX licenses are based on a monthly commitment, so choose a license type as per your requirement. For Dense shaped hosts, choosing an Enterprise License will have an additional cost. For more information, see learn more about HCX and HCX Enterprise features..

      HCX Options for SDDC

      Note: Enabling HCX during SDDC creation is recommended as it cannot be enabled manually once the SDDC is created. This ensures flexibility for future workload mobility needs.

    • SSH Key: Select the Public Key of the SSH key pair created as explained in Get Started with Oracle Cloud VMware Solution.

      Choose SSH Key

    • Advanced Options (Optional): Add any tags to help organize your resources in the tenancy.

  5. In the Define clusters section, you can create vSphere clusters to fit your enterprise needs. To add the first cluster i.e., the Unified Management Cluster, select Define Management Cluster.

    Define Clusters

    Note: Users can create up to 6 vSphere clusters and at least one cluster is required which is the Unified Management Cluster i.e., Management Cluster. The management cluster holds both the cluster management tools (vCenter, NSX Manager, and HCX Manager) and the workload VMs. Optionally, customers can add up to 5 clusters into the same SDDC which are called Workload clusters.

Task 2: Define Unified Management Cluster

  1. The Hosts section is the first step in the Clusters workflow. Enter the required details as described below.

    • Cluster Name: Provide a name for the Management Cluster. This should be an unique name across the whole tenancy and the length of the name cannot be more than 16 characters.

    • Availability Domain: A cluster can be deployed in two models.

      • Single-AD: Hosts are spread across the three fault domains within a single Availability Domain. This provides high availability against a fault-domain failure. For Standard Shape clusters, this is the only available deployment model.

      • Multi-AD: For unique use cases, a cluster can be deployed across availability domains if the selected region has three ADs. This option is available only with Dense shape instances and is not valid for Standard Shape compute shape. Before deploying this scenario, it is recommended to work with an Oracle Cloud Architect to understand the pros and cons of this architecture.

    • Host Type: Indicate whether the SDDC is intended for production deployments or Proof-of-Concept testing only.

      • Multi host SDDC: If you are planning to deploy an SDDC for production use with multiple-clusters, this is the option.

      • Single host SDDC: This option is for Proof-of-Concept purposes only.This cannot be upgraded later to a multi-host SDDC. Deploy a single host SDDC for non-production use cases only.

    • Number of ESXi hosts: Specify the number of hosts that are required in the Unified Management Cluster.

      Note: The Unified Management Cluster requires a minimum of three hosts to ensure optimal performance and availability. For more information about the minimum and maximum host requirements for different cluster types and hardware configurations, please refer to the Get Started with Oracle Cloud VMware Solution documentation.

    • Prefix for ESXi Hosts (Optional): Specify a custom prefix if ESXi host names must differ from the cluster name.

    • Capacity type: The default option is On-demand Capacity. Users can reserve capacity if they do not want to rely on the available public pool. If reserved instances are available in your tenancy, select the required reserved instances. For more information, see Capacity Reservations.

      Define Unified Management Clusters

    • Cluster Hardware Type: Oracle Cloud VMware Solution supports both Intel and AMD-based compute shapes. While mixed-shape clusters are possible, there are certain restrictions on which shapes can be combined. Select Change Shape to choose an appropriate hardware type. For more information on mixed-shaped clusters, see Get Started with Oracle Cloud VMware Solution.

      Unified Management Clusters - Hardware Type

      Available Hardware Options

    • Select number of CPU cores: Choose the desired number of OCPUs per host in the cluster based on your workload requirements.

    • Shielded Instances: Enhance the security of your ESXi hosts by enabling Shielded Instances which provide features such as Secure Boot and Trusted Platform Module (TPM). For more information, see Oracle Cloud VMware Solution - Shielded Instances..

      Note: This option must be enabled when you create the cluster. All hosts created in the cluster will be shielded instances. You cannot enable this option later or for specific ESXi hosts. If you have already created a cluster without enabling shielded instances, and later want to use shielded instances, you must recreate the cluster.

    • Pricing Interval Commitment: Oracle Cloud VMware Solution offers four different pricing commitments: Hourly, Monthly, One Year, and Three Years. These commitments are applied to individual compute instances and are based on the compute shape. You must confirm the selected pricing interval and its terms by checking the checkbox. Click Next for the Networking section.

      Pricing Interval Commitments

      Note: Dense I/O Shapes support all four pricing commitments, while Standard Shapes only support Hourly, One Year, and Three Years commitments. The default commitment for Dense I/O Shapes is Monthly, while the default commitment for Standard Shapes is Hourly. If you choose a Monthly commitment for a Dense I/O Shape and then switch to a Standard Shape, the commitment will be automatically changed to Hourly. For more information about pricing commitment, see Get Started with Oracle Cloud VMware Solution. If you want to enable One Year or Three Years commitment for your tenancy, you can open a service request.

  2. In the Networking section, enter the following details and click Next.

    • Virtual Cloud Network (VCN): Select the VCN in which you wish to create the SDDC. Let’s select the VCN-Demo-1, with the CIDR block 10.0.0.0/16, which was created during the prerequisites section of the Get Started with Oracle Cloud VMware Solution.

    • Users can choose to either create new subnets and VLANs or utilize existing ones. The default option, Create new subnet and VLANs, will automatically generate the necessary networking components, while the Select existing subnet and VLANs option requires users to preconfigure the networking elements before initiating the SDDC deployment workflow. This latter option is only necessary for specific customization needs.

      Cluster Networking

    • SDDC Cluster Networks: To create new subnets and VLANs, specify a Cluster CIDR range for the cluster. We will use the CIDR block 10.0.0.0/21. Upon entering the CIDR, the workflow will automatically validate its availability within the VCN.

      Cluster Networks

      • Click Hide Network Details to see how the Cluster CIDR block will be segmented for various VMware functionalities.

      • If you enabled VMware HCX, the VCN you selected must have a NAT gateway attached to it. If the VCN already has a NAT gateway, its name, compartment, and public IP address will be displayed. If the VCN does not have a NAT gateway, the workflow will automatically create one for you.

        Cluster Networks - Details

      • (Optional) If you selected the Select existing subnet and VLANs option, choose the previously created one provisioning subnet and 10 VLANs and assign them to their respective purposes. For more information, see Creating VLANs for an SDDC.

    • Cluster Workload CIDR: Provide a workload CIDR block, this is optional. This CIDR block is used as the DHCP range by the VMs on the NSX Workload segment. If users do not provide a CIDR range for their workloads, they can create NSX segments manually within NSX Manager after the SDDC deployment is complete. This value must be /30 or larger and must not overlap with the VCN CIDR block. In this tutorial, we will use a CIDR block 172.16.1.0/24 for the workload NSX segment.

      Workload Network

      Note: By default, all clusters within the SDDC share the same overlay segment, which simplifies management and reduces overhead. This approach utilizes a single set of NSX edge nodes, a tier-0 Gateway, and tier-1 Gateway for all clusters within the SDDC. If specific requirements necessitate dedicated NSX infrastructure for a particular cluster, customers can deploy it as a day-2 operation.

  3. (Optional) If you selected a Standard Shape based cluster, proceed to add Datastores in this step. For Dense shape-based clusters, skip this step and move on to the next section.

    Note: When deploying a Standard Shape based cluster as a Unified Management Cluster, a dedicated datastore solely for VMware Management tasks is automatically created. This datastore features a Block Volume size of 8TB and a VPU setting of 10 (Balanced). It is not intended to host workload VMs. For workload VMs, add additional datastores as detailed in the following section. Each OCI Block Volume added will be attached to all hosts in the cluster and become available as a Datastore. Subsequently, all Datastores are incorporated into a single Datastore cluster. This enables users to leverage Storage DRS within a specific cluster.

    • To create a Datastore, click Create Block Volume and enter the following details. The workflow is identical to creating a Block Volume within OCI.

      Datastores

      • Name: Enter a descriptive name for the Block Volume.

      • Compartment: Select the compartment where the Block Volume will reside.

        Create Block Volume

      • Volume size and performance: Leave Default or select Custom to edit the volume details. To customize the volume, provide the below details.

        • Size (GB): Specify the desired size for the Datastore in gigabytes.

        • Target volume performance:

          • Performance-based auto-tune is not currently available for Oracle Cloud VMware Solution. Keep this option disabled.

          • Default VPUs/GB: Select a VPU (Volume Performance Unit) value between 0 and 50 for the volume, with increments of 10. For more information on VPUs, see Block Volume Performance. Oracle Cloud VMware Solution currently supports a maximum VPU of 50. Do not enter a VPU value greater than 50.

          • Detached volume auto-tune: If this feature is enabled then the volume is detached, the Block Volume service adjusts the performance level to Lower Cost (0 VPUs/GB) for these volumes. This feature enables cost savings and resource efficiency. When the volume is reattached, the performance is adjusted back to the performance level specified by the default VPUs/GB setting.

            Volume Size and Performance

          • Encryption: To secure data at rest, by default all Block Volumes are encrypted using the Oracle-provided encryption keys. Choose an option as required. For more information, see Securing Block Volume.

            Block Volume Encryption

    • Continue adding Block Volumes as required by repeating the above steps. As we selected a Dense Shape cluster for the Unified Management Cluster in this tutorial, we will skip this section and proceed to the Notifications section. Click Next.

  4. In the Notifications section, configure notifications as per your preferences. Enabling Notifications allow users to track the status of the SDDC Cluster. A Topic must be created and relevant Events selected for these Notifications to be delivered. To explore available notifications with Oracle Cloud VMware Solution, see OCVS Notifications. Below is a quick summary of available notifications.

    Service Resource(s) Events
    VMware Solution SDDC, ESXi Host Create, Delete, Update
      HCX License Upgrade, Downgrade, Cancellation
    Compute Instance Any Actions, Terminate or Change Compartment
    Networking VCN, Subnet, VLANs Delete, Update

    For this tutorial, we will disable notifications. Click Next to continue to the Review section.

    Notifications

  5. On the Review page, review all the information and click Edit to make any necessary modifications. Once satisfied with the details, click Submit to complete the process and advance to the Define Clusters page.

    Cluster Review

    The Define Clusters page now displays a newly created cluster along with an option to Add a workload cluster. Users can optionally add up to five additional workload clusters to isolate workloads for specific use cases.

    Define Clusters Summary

    The option to add additional Workload Clusters to the same SDDC exists. To Add clusters, click Add a workload cluster. This tutorial will demonstrate the feature by adding one workload cluster.

Task 3: Define Workload Cluster (Optional)

This workload cluster will utilize a Standard Shaped cluster to demonstrate the capability of adding different cluster shapes within the same SDDC and the flexibility to employ diverse storage configurations while maintaining separation between clusters. The workflow for the workload cluster remains largely identical to that of the Unified Management Cluster.

  1. In the Hosts section, enter the following details.

    Workload Cluster - Hosts

  2. The Networking section presents the preselected VCN, as all SDDC resources must reside within the same VCN as the initial cluster. Choose an option for subnet and VLANs.

    Workload Cluster - Networking

    • Since we are establishing new networking for the second cluster, specify an unused CIDR block within the same VCN for cluster networks. This block will be subdivided into subnets and VLANs.

      Workload Cluster - Cluster Networks

      Note: A Workload Cluster only requires one new subnet and six VLANs. The remaining four VLANs inherit the same CIDR ranges as the first cluster and are subsequently disregarded as they are not necessary for workload clusters.

    • (Optional) As a single NSX overlay supports all clusters within an SDDC, all workload clusters are mapped to the same NSX segment, provided it was created with the Unified Management Cluster. If a non-overlapping CIDR block is provided with any workload cluster(s), a new NSX segment is created and assigned to all the clusters.

      Workload Cluster - NSX Segment

  3. Add block volumes as required in the Datastores section. A summary of the created volumes for the workload cluster is provided below. These volumes will be exclusively attached to this specific workload cluster and will not be shared with any other clusters within the SDDC.

    Datastores Summary

  4. We will opt out of Notifications and click Submit to finalize the details on the Review section.

Task 4: Review SDDC

  1. A summary of the clusters added to the SDDC is provided below. Expand each cluster and click Next to make any necessary modifications.

    SDDC Clusters Summary

  2. On the Review and Create page, carefully review all the information, make any necessary edits to sections or clusters, and click Create SDDC to submit the task.

    SDDC Review Page

    • Once the task is submitted, the Provisioning Resources task begins showing the progress.

      SDDC Provisioning Resources

      Note: Occasionally, you may see a prompt to Retry Provisioning for VLANs if a task times out. Select the retry option and the provisioning will continue.

  3. The VMware Solution summary page provides a comprehensive overview of all the resources being provisioned for the SDDC, along with their current status.

    • The SDDC should be available in approximately two and a half hours.

    • Click SDDC-Prod-1 to view your SDDC details.

      VMware Solution summary page

  4. Navigate to the SDDC details page, you should see the status of the SDDC as CREATING.

    SDDC Details page

    • You can see all the resources corresponding to the SDDC. Select Work Requests to monitor the status of the deployment.

    • Click Log Messages to see a detailed log of all the events that occur during the creation of an SDDC.

      Log Messages

    • If you want to monitor the status, revisit this page as required.

    • Approximately after two and a half hours the status of the SDDC should have changed to ACTIVE.

  5. Once provisioned, access the SDDC details page to view and manage various aspects of the SDDC. This page provides a comprehensive overview of the SDDC’s configuration, resources, and status.

    • SDDC Information: This section provides an overview of the SDDC’s global configuration and parameters, including VMware software version, OCID, cluster count, NSX Edge IP address, and details of VMware management tools.

      SDDC Information

    • vSphere Clusters: This section provides an overview of the vSphere clusters within the SDDC. Each cluster listing includes details such as cluster type and host count. Selecting a cluster reveals its in-depth information, encompassing ESXi hosts, cluster networks, and datastores. The below screenshot showcases the Cluster Details page for one of the clusters within the deployed SDDC.

      Log Messages

    This detailed information provides a complete overview of the SDDC’s configuration and resources, enabling effective management and monitoring of your VMware SDDC deployment on Oracle Cloud Infrastructure.

Next Steps

Congratulations on deploying your Oracle Cloud VMware Solution SDDC! The resources within your SDDC are now provisioned and ready for use.

To securely access and manage these resources, you can establish connectivity from your on-premises environment or utilize secure connectivity options such as the OCI Bastion service or a jump host. For more information about securely accessing your SDDC resources, see Access your Oracle Cloud VMware Solution SDDC resources.

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.