1. Maintenance Guide
  2. Managing Oracle VM User Domains
  3. Expanding an Oracle RAC Cluster on Oracle VM Using OEDACLI

5.16 Expanding an Oracle RAC Cluster on Oracle VM Using OEDACLI

You can expand an existing Oracle RAC cluster on Oracle VM by adding user domains using the Oracle Exadata Deployment Assistant command-line interface (OEDACLI).

OEDACLI is the preferred method if you have a known, good version of the OEDA XML file for your cluster.

Note:

During the execution of this procedure, the existing Oracle RAC cluster nodes along with their database instances incur zero downtime.

Use cases for this procedure include:

  • You have an existing Oracle RAC cluster that uses only a subset of the database servers of an Oracle Exadata Rack, and now the nodes not being used by the cluster have become candidates for use.
  • You have an existing Oracle RAC cluster on Oracle Exadata that was recently extended with additional database servers.
  • You have an existing Oracle RAC cluster that had a complete node failure and the node was removed and replaced with a newly re-imaged node.

Before preforming the steps in this section, the new database servers should have been set up as detailed in Adding a New Database Server to the Cluster, including the following:

  • The new database server is installed and configured on the network with a management domain.
  • Download the latest Oracle Exadata Deployment Assistant (OEDA); ensure the version you download is the July 2019 release, or later.
  • You have an OEDA configuration XML file that accurately reflects the existing cluster configuration. You can validate the XML file by generating an installation template from it and comparing it to the current configuration. See the OEDACLI command SAVE FILES.
  • Review the OEDA Installation Template report for the current system configuration to obtain node names and IP addresses for existing nodes. You will need to have new host names and IP addresses for the new nodes being added. The new host names and IP addresses required are:
    • Administration host names and IP addresses (referred to as ADMINNET) for the management domain and the user domains.
    • Private host names and IP addresses (referred to as PRIVNET) for the management domain and the user domains.
    • Integrated Lights Out Manager (ILOM) host names and IP addresses for the management domain.
    • Client host names and IP addresses (referred to as CLIENTNET) for the user domains.
    • Virtual IP (VIP) host names and IP addresses (referred to as VIPNET) for the user domains.
    • Physical rack number and location of the new node in the rack (in terms of U number)

  • Each management domain has been imaged or patched to the same image in use on the existing database servers. The current system image must match the version of the /EXAVMIMAGES/ System.first.boot.*.img file on the new management domain node.

    Note:

    The ~/dom0_group file referenced below is a text file that contains the host names of the management domains for all existing and new nodes being added.

    Check the image version across all management domains are the same. 

    dcli -g ~/dom0_group -l root "imageinfo -ver"

exa01adm01: 19.2.0.0.0.190225
exa01adm02: 19.2.0.0.0.190225
exa01adm03: 19.2.0.0.0.190225

    If any image versions differ, you must upgrade the nodes as needed so that they match.

    Ensure that the System.first.boot version across all management domains matches the image version retrieved in the previous step. 

    dcli -g ~/dom0_group -l root "ls  -1 /EXAVMIMAGES/System.first.boot*.img" 
exa01adm01:  /EXAVMIMAGES/System.first.boot.19.2.0.0.0.190225.img
exa01adm02:  /EXAVMIMAGES/System.first.boot.19.2.0.0.0.190225.img
exa01adm03:  /EXAVMIMAGES/System.first.boot.19.2.0.0.0.190225.img

    If any nodes are missing the System.first.boot.img file that corresponds to the current image, then obtain the required file. See the “Supplemental README note” for your Exadata release in My Oracle Support Doc ID 888828.1 and look for the patch file corresponding to this description, “DomU System.img OS image for V.V.0.0.0 VM creation on upgraded dom0s”

  • Place the klone.zip files (gi-klone*.zip and db-klone*.zip) in the /EXAVMIMAGES location on the freshly imaged management domain node you are adding to the cluster. These files can be found in the/EXAVMIMAGES directory on the management domain node from where the system was initially deployed.

The steps here show how to add a new management domain node called exa01adm03 that will have a new user domain called exa01adm03vm01. The steps show how to extend an existing Oracle RAC cluster onto the user domain using OEDACLI commands. The existing cluster has management domain nodes named exa01adm01 and exa01adm02 and user domain nodes named exa01adm01vm01 and exa01adm02vm01.

  1. Add the management domain information to the OEDA XML file using the CLONE COMPUTE command.

    In the examples below, the OEDA XML file is assumed to be in: unzipped_OEDA_location/ExadataConfigurations. 

    OEDACLI> LOAD FILE NAME=exa01_original_deployment.xml 

OEDACLI> CLONE COMPUTE SRCNAME=exa01adm01 TGTNAME=exa01adm03
OEDACLI> SET ADMINNET NAME=exa01adm03,IP=xx.xx.xx.xx
OEDACLI> SET PRIVNET NAME1=exa01adm03-priv1,IP1=xx.xx.xx.xx,NAME2=exa01adm03-priv2,IP2=xx.xx.xx.xx
OEDACLI> SET ILOMNET NAME=exa01adm03-c,IP=xx.xx.xx.xx
OEDACLI> SET RACK NUM=NN,ULOC=XX 

OEDACLI> SAVE ACTION
OEDACLI> MERGE ACTIONS FORCE
OEDACLI> SAVE FILE NAME=exa01_plus_adm03_node.xml

    At this point we have a new XML file that has the new compute node management domain in the configuration. This file will be used by the subsequent steps.

  2. Add the new user domain information to the OEDA XML file using the CLONE GUEST command and deploy the guest.
    OEDACLI> LOAD FILE NAME=exa01_plus_adm03_node.xml 

OEDACLI> CLONE GUEST SRCNAME=exa01adm01vm01 TGTNAME=exa01adm03vm01 WHERE STEPNAME=CREATE_GUEST
OEDACLI> SET PARENT NAME=exa01adm03
OEDACLI> SET ADMINNET NAME=exa01adm03vm01,IP=xx.xx.xx.xx
OEDACLI> SET PRIVNET NAME1=exa01db03vm01-priv1,IP1=xx.xx.xx.xx,NAME2=exa01db03vm01-priv2,IP2=xx.xx.xx.xx
OEDACLI> SET CLIENTNET NAME=exa01client03vm01,IP=xx.xx.xx.xx
OEDACLI> SET VIPNET NAME=exa01client03vm01-vip,IP=xx.xx.xx.xx

OEDACLI> SAVE ACTION
OEDACLI> MERGE ACTIONS
OEDACLI> DEPLOY ACTIONS

    If you prefer that OEDACLI runs all steps automatically, omit the WHERE STEPNAME=CREATE_GUEST clause from the CLONE GUEST command and skip step 3 below.

  3. Use OEDACLI to extend the cluster to the new user domain.

    Note:

    Continue using the same XML file, exa01_plus_adm03_node.xml in this example. You will continue to update this file as you proceed through these steps. At the very end of the procedure, this XML file will properly reflect the new state of the clusters. 
    OEDACLI> CLONE GUEST SRCNAME=exa01adm01vm01 TGTNAME=exa01adm03vm01 WHERE STEPNAME=CREATE_USERS

OEDACLI> SAVE ACTION
OEDACLI> MERGE ACTIONS
OEDACLI> DEPLOY ACTIONS

OEDACLI> CLONE GUEST SRCNAME=exa01adm01vm01 TGTNAME=exa01adm03vm01 WHERE STEPNAME=CELL_CONNECTIVITY

OEDACLI> SAVE ACTION
OEDACLI> MERGE ACTIONS
OEDACLI> DEPLOY ACTIONS

OEDACLI> CLONE GUEST SRCNAME=exa01adm01vm01 TGTNAME=exa01adm03vm01 WHERE STEPNAME=ADD_NODE

OEDACLI> SAVE ACTION
OEDACLI> MERGE ACTIONS
OEDACLI> DEPLOY ACTIONS

OEDACLI> CLONE GUEST SRCNAME=exa01adm01vm01 TGTNAME=exa01adm03vm01 WHERE STEPNAME=EXTEND_DBHOME

OEDACLI> SAVE ACTION
OEDACLI> MERGE ACTIONS
OEDACLI> DEPLOY ACTIONS

OEDACLI> CLONE GUEST SRCNAME=exa01adm01vm01 TGTNAME=exa01adm03vm01 WHERE STEPNAME=ADD_INSTANCE

OEDACLI> SAVE ACTION
OEDACLI> MERGE ACTIONS
OEDACLI> DEPLOY ACTIONS

    OEDACLI prints out messages similar to the following as each step completes: 

    Deploying Action ID : 39 CLONE GUEST SRCNAME=exa01adm01vm01 TGTNAME=exa01adm03vm01 where STEPNAME=ADD_INSTANCE 
Deploying CLONE GUEST 
Cloning Guest 
Cloning Guest  :  exa01adm03vm01.example.com_id 
Adding new instance for database [dbm] on exa01adm03vm01.example.com 
Setting up Huge Pages for Database..[dbm] 
Adding instance dbm3 on host exa01adm03vm01.example.com 
Successfully completed adding database instance on the new node [elapsed Time [Elapsed = 
249561 mS [4.0  minutes] Fri Jun 28 13:35:52 PDT 2019]] 
Done...
Done
  4. Save the current state of the configuration and generate configuration information.
    OEDACLI> SAVE FILES LOCATION=/tmp/exa01_plus_adm03_config

    The above command writes all the configuration files to the directory /tmp/exa01_plus_adm03_config. Save a copy of these files in a safe place since they now reflect the changes made to your cluster.

  5. Gather an Exachk report and examine it to ensure the cluster is in good health.

Related Topics

Parent topic: Managing Oracle VM User Domains