Note:

Provision an Oracle Exadata Database Service on Cloud@Customer Virtual Machine Cluster with Ansible

Introduction

This tutorial explains how to use the OCI Ansible collection to provision Oracle Exadata Database Service on Cloud@Customer virtual machine (VM) clusters. To create an Oracle Exadata Database Service on Cloud@Customer VM cluster, it is mandatory to create the corresponding VM cluster network first.

Objectives

Prerequisites

For more information, see Prerequisites for Using Ansible with Oracle Cloud Infrastructure.

Task 1: Create an Inventory

  1. Connect to the properly configured Ansible control node.

  2. Create an inventory file named inventory_exacc.yml with the following content.

    localhost ansible_connection=local

Task 2: Create a Variable File

Create a variable file named variables_exacc.yml with all variables described in the following content.

# Your variables file

your_region: "eu-frankfurt-1"   # The OCI region where your resources will reside - This parameter is optional
your_exa_infra_id: "ocid1.exadatainfrastructure.oc1.eu-frankfurt-1.xxx"   # The OCI of your Exadata infrastructure

## Variables related to your VM cluster network

your_vm_net_comp_id: "ocid1.compartment.oc1..xxx"   # The compartment OCID of your VM cluster network
your_vm_net_name: "my_vm_cluster_network_name"   # The display name of your VM cluster network
your_vm_net_dns: "10.109.10.2"   # The list of your DNS server IP addresses. Maximum of 3 allowed.
your_vm_net_ntp: "10.109.10.3"   # The list of your NTP server IP addresses. Maximum of 3 allowed.
your_vm_net_scan_ips: ["203.101.10.123", "203.101.10.124", "203.101.10.125"]   # The list of your SCAN IP addresses. Three addresses should be provided.
your_vm_net_scan_listener_port_tcp: "1521"   # Your SCAN TCPIP port. Default is 1521.

### Variables related to your CLIENT network

your_cli_nodes_hostname_1: "your_1st_hostname"   # The hostname of your 1st node on the CLIENT network
your_cli_nodes_ip_1: "203.101.10.101"   # The IP address of your 1st node on the CLIENT network
your_cli_nodes_vip_hostname_1: "your_1st_hostname-vip"   # The node virtual IP (VIP) host name of your 1st node on the CLIENT network
your_cli_nodes_vip_1: "203.101.10.102"   # The node virtual IP (VIP) address of your 1st node on the CLIENT network
your_db_server_ocid_1: "ocid1.dbserver.oc1.eu-frankfurt-1.xxx"   # The Db server OCID associated with your 1st node on the CLIENT network

your_cli_nodes_hostname_2: "your_2nd_hostname"   # The hostname of your 2nd node on the CLIENT network
your_cli_nodes_ip_2: "203.101.10.103"   # The IP address of your 2nd node on the CLIENT network
your_cli_nodes_vip_hostname_2: "your_2nd_hostname-vip"   # The node virtual IP (VIP) host name of your 2nd node on the CLIENT network
your_cli_nodes_vip_2: "203.101.10.104"   # The node virtual IP (VIP) address of your 2nd node on the CLIENT network
your_db_server_ocid_2: "ocid1.dbserver.oc1.eu-frankfurt-1.yyy"   # The Db server OCID associated with your 2nd node on the CLIENT network

your_cli_vm_net_vlan_id: "123"   # Your CLIENT network VLAN ID
your_cli_vm_net_netmask: "your_client_netmask"   # Your CLIENT network netmask
your_cli_vm_net_gateway: "your_client_gateway_ip"   # Your CLIENT network gateway
your_cli_vm_net_domain_name: "your_client_network_domain_name"   # Your CLIENT network domain name

### Variables related to your BACKUP network

your_bck_nodes_hostname_1: "your_1st_hostname-bk"   # The hostname of your 1st node on the BACKUP network
your_bck_nodes_ip_1: "10.108.10.3"   # The IP address of your 1st node on the BACKUP network
your_bck_nodes_hostname_2: "your_2nd_hostname-bk"   # The hostname of your 2nd node on the BACKUP network
your_bck_nodes_ip_2: "10.108.10.4"   # The IP address of your 2nd node on the BACKUP network

your_bck_vm_net_vlan_id: "456"   # Your BACKUP network VLAN ID
your_bck_vm_net_netmask: "your_backup_netmask"   # Your BACKUP network netmask
your_bck_vm_net_gateway: "your_backup_gateway_ip"   # Your BACKUP network gateway
your_bck_vm_net_domain_name: "your_backup_network_domain_name"   # Your BACKUP network domain name

## Variables related to your VM cluster

your_vm_cluster_comp_id: "ocid1.compartment.oc1..yyy"   # The compartment OCID of your VM cluster
your_vm_cluster_name: "my_vm_cluster_name"
your_vm_clus_ssh_public_keys: [ "ssh-rsa AAAAB3NzaC...qScP4xMPQ9L5xWsqs" ]
your_license_model: "LICENSE_INCLUDED"   # The Oracle license model that applies to the VM cluster. The default is BRING_YOUR_OWN_LICENSE.

Task 3: Create a Playbook to Provision a VM Cluster Network

  1. Create a playbook file named playbook_exacc_vmCluster_network_creation.yml with the following content.

    - hosts: localhost
  collections:
    - oracle.oci
  vars_files:
    - variables_exacc.yml
  tasks:
    - name: Create VM cluster network
      oci_database_vm_cluster_network:
        region: "{{ your_region }}"
        compartment_id: "{{ your_vm_net_comp_id }}"
        exadata_infrastructure_id: "{{ your_exa_infra_id }}"
        display_name: "{{ your_vm_net_name }}"
        dns: "{{ your_vm_net_dns }}"
        ntp: "{{ your_vm_net_ntp }}"
        scans:
        - hostname: "{{ your_vm_clus_name }}-scan"
          ips: "{{ your_vm_net_scan_ips }}"
          scan_listener_port_tcp: "{{ your_vm_net_scan_listener_port_tcp }}"
        vm_networks:
        - network_type: CLIENT
          nodes:
          - hostname: "{{ your_cli_nodes_hostname_1 }}"
            ip: "{{ your_cli_nodes_ip_1 }}"
            vip_hostname: "{{ your_cli_nodes_vip_hostname_1 }}"
            vip: "{{ your_cli_nodes_vip_1 }}"
            db_server_id: "{{ your_db_server_ocid_1 }}"
          - hostname: "{{ your_cli_nodes_hostname_2 }}"
            ip: "{{ your_cli_nodes_ip_2 }}"
            vip_hostname: "{{ your_cli_nodes_vip_hostname_2 }}"
            vip: "{{ your_cli_nodes_vip_2 }}"
            db_server_id: "{{ your_db_server_ocid_2 }}"
          vlan_id: "{{ your_cli_vm_net_vlan_id }}"
          netmask: "{{ your_cli_vm_net_netmask }}"
          gateway: "{{ your_cli_vm_net_gateway }}"
          domain_name: "{{ your_cli_vm_net_domain_name }}"
        - network_type: BACKUP
          nodes:
          - hostname: "{{ your_bck_nodes_hostname_1 }}"
            ip: "{{ your_bck_nodes_ip_1 }}"
            vip_hostname: "{{ your_bck_nodes_vip_hostname_1 }}"
            vip: "{{ your_bck_nodes_vip_1 }}"
            db_server_id: "{{ your_db_server_ocid_1 }}"
          - hostname: "{{ your_bck_nodes_hostname_2 }}"
            ip: "{{ your_bck_nodes_ip_2 }}"
            vip_hostname: "{{ your_bck_nodes_vip_hostname_2 }}"
            vip: "{{ your_bck_nodes_vip_2 }}"
            db_server_id: "{{ your_db_server_ocid_2 }}"
          vlan_id: "{{ your_bck_vm_net_vlan_id }}"
          netmask: "{{ your_bck_vm_net_netmask }}"
          gateway: "{{ your_bck_vm_net_gateway }}"
          domain_name: "{{ your_bck_vm_net_domain_name }}"
        wait: true
      register: new_vmClusterNetwork

    Note the task keyword register, at the end of the invocation of module oci_database_vm_cluster_network: it means the output of the module will be registered in the new_vmClusterNetwork variable. This variable will contain all details of the newly created VM cluster network, including its OCID.

    Note: For this tutorial, there are only two nodes dictionaries for each network type, because the underlying Exadata infrastructure contains two db servers. To create a VM cluster with four or eight nodes, it is necessary to provide the adequate number of nodes dictionaries for each network type.

  2. In the playbook playbook_exacc_vmCluster_network_creation.yml, add the following task to retrieve the OCID of the newly created VM cluster network.

    - name: Get OCID of newly created VM cluster network
  set_fact:
    your_vm_net_ocid: "{{ item.id }}"
  loop:
  - "{{ new_vmClusterNetwork.vm_cluster_network }}"
  loop_control:
    label: "{{ item.display_name }}"

  3. In the playbook, playbook_exacc_vmCluster_network_creation.yml, add the following tasks to validate the newly created VM cluster network and to download its configuration.

        - name: Validate VM cluster network
      oci_database_vm_cluster_network_actions:
        exadata_infrastructure_id: "{{ your_exa_infra_id }}"
        vm_cluster_network_id: "{{ your_vm_net_ocid }}"
        region: "{{ your_region }}"
        action: validate


    - name: Download network configuration file of VM cluster network
      oci_database_vm_cluster_network_actions:
        exadata_infrastructure_id: "{{ your_exa_infra_id }}"
        vm_cluster_network_id: "{{ your_vm_net_ocid }}"
        action: download_vm_cluster_network_config_file
        config_file_dest: "/tmp/net_config_my_vm_cluster_network.json"
        region: "{{ your_region }}"

  4. In the playbook playbook_exacc_vmCluster_network_creation.yml, add the following tasks to display the details of the newly created VM cluster network after its validation has taken place.

        - name: Get facts of newly created VM cluster network
      oci_database_vm_cluster_network_facts:
        exadata_infrastructure_id: "{{ your_exa_infra_id }}"
        vm_cluster_network_id: "{{ your_vm_net_ocid }}"
        region: "{{ your_region }}"
      register: newly_created_VM_cluster_network_facts


    - name: Display details of newly created VM cluster network
      debug:
        msg : "{{ newly_created_VM_cluster_network_facts.vm_cluster_networks }}"

Task 4: Run the Playbook to Provision a VM Cluster Network

  1. Run the VM cluster network creation playbook with the verbose option.

    ansible-playbook playbook_exacc_vmCluster_network_creation.yml -i inventory_exacc.yml -vvv

    Note: Using the verbose option -vvv will increase the verbosity and help you understand all the underlying steps while the playbook is running.

  2. After a few minutes, a successful execution of the oci_database_vm_cluster_network module should return the following output.

    changed: [localhost] => {
[...],
    "vm_cluster_network": {
        "compartment_id": "ocid1.compartment.oc1..xxx",
        "defined_tags": {
            "Oracle-Tags": {
                "CreatedBy": "creation_user",
                "CreatedOn": "2025-01-09T09:30:30.631Z"
            }
        },
        "display_name": "my_vm_cluster_network_name",
        "dns": [
            "10.109.10.2"
        ],
        "dr_scans": null,
        "exadata_infrastructure_id": "ocid1.exadatainfrastructure.oc1.eu-frankfurt-1.xxx",
        "freeform_tags": {},
        "id": "ocid1.vmclusternetwork.oc1.eu-frankfurt-1.xxx",
        "lifecycle_details": null,
        "lifecycle_state": "REQUIRES_VALIDATION",
        "ntp": [
            "10.109.10.3"
        ],
        "scans": [
            {
                "hostname": "name_of_your-scan",
                "ips": [
                    "203.101.10.123",
                    "203.101.10.124",
                    "203.101.10.125"
                ],
                "port": 1521,
                "scan_listener_port_tcp": 1521,
                "scan_listener_port_tcp_ssl": 2484
            }
        ],
        "time_created": "2025-01-09T09:30:30.818000+00:00",
        "vm_cluster_id": null,
        "vm_networks": [
            {
                "domain_name": "your_client_network_domain_name",
                "gateway": "your_client_gateway_ip",
                "netmask": "your_client_netmask",
                "network_type": "CLIENT",
                "nodes": [
                    {
                        "db_server_id": "ocid_of_your_1st_db_server",
                        "hostname": "your_1st_hostname",
                        "ip": "203.101.10.101",
                        "lifecycle_state": "REQUIRES_VALIDATION",
                        "vip": "203.101.10.102",
                        "vip_hostname": "your_1st_hostname-vip"
                    },
                    {
                        "db_server_id": "ocid_of_your_2nd_db_server",
                        "hostname": "your_2nd_hostname",
                        "ip": "203.101.10.103",
                        "lifecycle_state": "REQUIRES_VALIDATION",
                        "vip": "203.101.10.104",
                        "vip_hostname": "your_2nd_hostname-vip"
                    }
                ],
                "vlan_id": "123"
            },
            {
                "domain_name": "your_backup_network_domain_name",
                "gateway": "your_backup_gateway_ip",
                "netmask": "your_backup_netmask",
                "network_type": "BACKUP",
                "nodes": [
                    {
                        "db_server_id": "ocid_of_your_1st_db_server",
                        "hostname": "your_1st_hostname-bk",
                        "ip": "10.108.10.3",
                        "lifecycle_state": "REQUIRES_VALIDATION",
                    },
                    {
                        "db_server_id": "ocid_of_your_2nd_db_server",
                        "hostname": "your_2nd_hostname-bk",
                        "ip": "10.108.10.4",
                        "lifecycle_state": "REQUIRES_VALIDATION",
                    }
                ],
                "vlan_id": "456"
            }
        ]
    }
}

Task 5: Create a Playbook to Provision a VM Cluster

  1. Create a playbook file named playbook_exacc_vmCluster_creation.yml with the following content.

    - hosts: localhost
  collections:
    - oracle.oci
  vars_files:
    - variables_exacc.yml
  tasks:

    - name: Get all existing VM cluster networks
      oci_database_vm_cluster_network_facts:
        exadata_infrastructure_id: "{{ your_exa_infra_id }}"
        compartment_id: "{{ your_vm_net_comp_id }}"
        region: "{{ your_region }}"
      register: get_all_vmClusterNetworks


    - name: Get OCID of the newly created VM cluster network
      set_fact:
        your_vm_net_ocid: "{{ item.id }}"
      loop: "{{ get_all_vmClusterNetworks.vm_cluster_networks }}"
      loop_control:
        label: "{{ item.display_name }}"
      when: item.display_name == your_vm_net_name

    These 2 tasks are used to retrieve the OCID of the newly created VM cluster network, because it will be necessary for the VM cluster creation.

  2. Add the VM cluster creation task in the playbook, playbook_exacc_vmCluster_creation.yml.

    - name: Create VM cluster
  oci_database_vm_cluster:
    region: "{{ your_region }}"
    compartment_id: "{{ your_vm_cluster_comp_id }}"
    display_name: "{{ your_vm_cluster_name }}"
    exadata_infrastructure_id: "{{ your_exa_infra_id }}"
    vm_cluster_network_id: "{{ your_vm_net_ocid }}"
    gi_version: "19.0.0.0.0"
    cpu_core_count: "4"
    ssh_public_keys: "{{ your_vm_clus_ssh_public_keys }}"
    time_zone: "Europe/Zurich"
    memory_size_in_gbs: "60"
    db_node_storage_size_in_gbs: "120"
    data_storage_size_in_tbs: "2"
    license_model: "{{ your_license_model }}"
    is_sparse_diskgroup_enabled: "false"
    is_local_backup_enabled: "false"
    data_collection_options:
      is_diagnostics_events_enabled: "false"
      is_health_monitoring_enabled: "false"
      is_incident_logs_enabled: "false"
    wait: true
    wait_timeout: 18000

    Note that the current accepted values for the gi_version parameter are 19.0.0.0.0 and 23.0.0.0.0. Otherwise, you will get the following error: Cannot create VM cluster because Oracle Grid Infrastructure version is invalid. Valid versions are: [19.0.0.0.0, 23.0.0.0.0].

    Note: The VM cluster creation usually takes up to 3 hours. This is why the wait: true and wait_timeout: 18000 instructions are added in the code.

Task 6: Run the Playbook to Provision a VM Cluster

  1. Run the VM cluster creation playbook with the verbose option.

    ansible-playbook playbook_exacc_vmCluster_creation.yml -i inventory_exacc.yml -vvv

    Note: Using the verbose option -vvv will increase the verbosity and help you understand all the underlying steps while the playbook is running.

  2. After a few minutes, a successful execution of the module oci_database_vm_cluster should return the following output.

    changed: [localhost] => {
[...],
    "vm_clusters": [
        {
            "availability_domain": "your_availability_domain",
            "compartment_id": "ocid1.compartment.oc1..yyy",
            "cpus_enabled": 4,
            "data_collection_options": {
                "is_diagnostics_events_enabled": false,
                "is_health_monitoring_enabled": false,
                "is_incident_logs_enabled": false
            },
            "data_storage_size_in_gbs": null,
            "data_storage_size_in_tbs": 2.0,
            "db_node_storage_size_in_gbs": 120,
            "db_servers": [
                "ocid1.dbserver.oc1.eu-frankfurt-1.xxx",
                "ocid1.dbserver.oc1.eu-frankfurt-1.yyy"
            ],
            "defined_tags": {
                "Oracle-Tags": {
                    "CreatedBy": "creation_user",
                    "CreatedOn": "2025-01-09T12:40:17.201Z"
                }
            },
            "display_name": "my_vm_cluster_name",
            "exadata_infrastructure_id": "ocid1.exadatainfrastructure.oc1.eu-frankfurt-1.xxx",
            "file_system_configuration_details": [
                {
                    "file_system_size_gb": 15,
                    "mount_point": "/"
                },
                [...],
                {
                    "file_system_size_gb": 16,
                    "mount_point": "swap"
                }
            ],
            "freeform_tags": {},
            "gi_software_image_id": null,
            "gi_version": "19.24.0.0.0",
            "id": "ocid1.vmcluster.oc1.eu-frankfurt-1.xxx",
            "is_local_backup_enabled": false,
            "is_sparse_diskgroup_enabled": false,
            "last_patch_history_entry_id": null,
            "license_model": "your_license_model",
            "lifecycle_details": null,
            "lifecycle_state": "AVAILABLE",
            "memory_size_in_gbs": 60,
            "ocpus_enabled": null,
            "shape": "your_ExadataCC_shape",
            "ssh_public_keys": null,
            "system_version": "24.1.6.0.0.241115",
            "time_created": "2025-01-09T12:40:17.256000+00:00",
            "time_zone": "Europe/Zurich",
            "vm_cluster_network_id": "your_newly_created_vmCluster_network_OCID"
        }
    ]
}

Task 7: Create a Playbook to Check the State of the Newly Created VM Cluster

Create a playbook file named playbook_exacc_vmCluster_post_creation_checks.yml with the following content.

- hosts: localhost
  collections:
    - oracle.oci
  vars_files:
    - variables_exacc.yml
  tasks:
    - name: Get newly created vmCluster facts
      oci_database_vm_cluster_facts:
        region: "{{ your_region }}"
        compartment_id: "{{ your_vm_cluster_comp_id }}"
        exadata_infrastructure_id: "{{ your_exa_infra_id }}"
        display_name: "{{ your_vm_cluster_name }}"
      register: output_cluster_facts


    - name: Set vmCluster facts
      set_fact:
        your_vm_cluster_id: "{{ item.id }}"
        your_vm_cluster_state: "{{ item.lifecycle_state }}"
      loop: "{{ output_cluster_facts.vm_clusters }}"
      loop_control:
        label: "{{ item.display_name }}"
      when: item.display_name == your_vm_cluster_name


    - name: Check VM cluster state
      debug:
        msg: "The VM cluster  your_vm_cluster_name  is in lifecycle state  your_vm_cluster_state ."

Task 8: Run the Playbook to Check the State of the Newly Created VM Cluster

  1. Run the VM cluster check playbook with the verbose option.

    ansible-playbook playbook_exacc_vmCluster_post_creation_checks.yml -i inventory_exacc.yml -vvv

    Note: Using the verbose option -vvv will increase the verbosity and help you understand all the underlying steps while the playbook is running.

  2. After a few seconds, a successful execution of the playbook playbook_exacc_vmCluster_post_creation_checks.yml should return the following output.

    ok: [localhost] => {
    "msg": "The VM cluster your_vm_cluster_name is in lifecycle state AVAILABLE."
}

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.