5.21 Removing an Oracle RAC Cluster Running in Oracle VM

You can remove all Oracle RAC nodes of an Oracle VM cluster, including the databases running within the cluster and all data stored on the Oracle Exadata Storage Server used by those databases.

To remove only a subset of user domains of an Oracle VM cluster, refer to the next section.

There are two main steps to remove an Oracle VM cluster:

  • Remove the user domain files from the management domain.

  • Remove the unused Oracle Exadata grid disks.

Note:

If the Oracle Exadata Deployment Assistant xml configuration files are to be reused later, then they will be not synchronized because the definition for the removed user domain still exists in Oracle Exadata Deployment Assistant files.

  1. Run the following example script as the grid software owner on any user domain to be removed.

    The example shell script generates two scripts, list_griddisk.sh and drop_griddisk.sh, that are run later in this procedure. Do not run the generated scripts until instructed.

    #!/bin/bash
     
    # Run this script as the Grid Infrastructure software owner.
    #
    # This script identifies griddisks used by this cluster and the cells to
    # which they belong, then creates two shell scripts - the list script to
    # show the current status, and the drop script to drop the griddisks.
    #
    # In order for the drop script to succeed, the griddisks must not be in use,
    # meaning databases and CRS are down, and the list script returns no output.
    #
    # The generated scripts are designed to run via dcli -x
     
    ORACLE_SID=$(awk -F: '/^+ASM/{print $1}' /etc/oratab)
    ORAENV_ASK=NO . oraenv >/dev/null
     
    listGriddiskScript=list_griddisk.sh
    dropGriddiskScript=drop_griddisk.sh
     
    rm -f $listGriddiskScript $dropGriddiskScript
     
    gridDiskList=$(asmcmd lsdsk --suppressheader | awk -F'/' '{print $NF}')
    if [[ ${PIPESTATUS[0]} != 0 ]]; then echo "asmcmd failed - exiting"; exit 1; fi
     
    cellList=$(echo "$gridDiskList" | awk -F_ '{print $NF}' | sort -u)
     
    for cell in $cellList; do
      myGriddisks=$(echo "$gridDiskList" | grep ${cell}$ | tr '\n' ',')
      echo "[[ \$(hostname -s) == ${cell} ]] && cellcli -e 'LIST GRIDDISK \
            ${myGriddisks%,} attributes name, asmDiskGroupName, asmModeStatus \
            where asmModeStatus != UNKNOWN'" >> $listGriddiskScript
      echo >> $listGriddiskScript
    done
     
    chmod +x $listGriddiskScript
     
    echo
    echo "Run the following command to list griddisks in use by this cluster:"
    echo
    echo "# dcli -l celladmin -c ${cellList//$'\n'/,} -x $listGriddiskScript"
    echo
     
    for cell in $cellList; do
      myGriddisks=$(echo "$gridDiskList" | grep ${cell}$ | tr '\n' ',')
      echo "[[ \$(hostname -s) == ${cell} ]] && cellcli -e 'DROP GRIDDISK \
            ${myGriddisks%,}'" >> $dropGriddiskScript
      echo >> $dropGriddiskScript
    done
     
    chmod +x $dropGriddiskScript
     
    echo
    echo "Stop CRS on all nodes in this cluster, then run the following"
    echo "command to drop all griddisks used by this cluster:"
    echo
    echo "# dcli -l celladmin -c ${cellList//$'\n'/,} -x $dropGriddiskScript"
    echo
     
    exit 
    
  2. Shut down the databases and Oracle Grid Infrastructure in all user domains that will be removed:
     # Grid_home/bin/crsctl stop crs -f
    
  3. Run the list_griddisk.sh script generated earlier from any user domain that will be removed.

    Note:

    • Run the script using the dcli command to connect as the celladmin user to all Oracle Exadata Storage Servers in the configuration.

    • Before running the dcli command, set up a passwordless SSH connection between the grid software owner on the database server and the celladmin user on the cells. Otherwise, the command will keep prompting you to enter the password.

    The following is an example of the command:

    $ dcli -l celladmin -c dm01celadm01,dm01celadm02,dm01celadm03  \
    -x list_griddisk.sh
    

    The list_griddisk.sh script should not output any grid disks. Grid disks returned from the list_griddisk.sh script are considered still in use.

    Do not proceed until the list_griddisk.sh script returns empty output indicating no grid disks are in use. Verify that Oracle Grid Infrastructure and the databases are shut down on all user domains to be dropped.

  4. Run the drop_griddisk.sh script generated earlier from any user domain that you want to remove.

    Run the script using the dcli command to connect as the celladmin user to all Oracle Exadata Storage Servers in the configuration.

    $ dcli -l celladmin -c dm01celadm01,dm01celadm02,dm01celadm03 \
    -x drop_griddisk.sh
    
  5. Run the exadata.img.domu_maker command from the management domain of each user domain you want to remove.

    This command removes the user domains, where DomainName is the name of the user domain.

    # /opt/exadata_ovm/exadata.img.domu_maker remove-domain DomainName
    

    In the following example, the commands remove the two user domains for a two-node Oracle VM RAC cluster in which the user domain dm01db01vm04 runs on the management domain dm01db01, and the user domain dm01db02vm04 runs on the management domain dm01db02.

    [root@dm01db01 ~] # /opt/exadata_ovm/exadata.img.domu_maker \
    remove-domain dm01db01vm04
    [INFO] Start with command line: /opt/exadata_ovm/exadata.img.domu_maker \
     remove-domain dm01db01vm04
    [INFO] Shutting down DomU dm01db01vm04
    [INFO] Autostart link for dm01db01vm04 deleted from /etc/xen/auto
    [INFO] Deleted OVM repository /OVS/Repositories/7bfd49d6bd5a4b2db2e46e8234788067 for DomU dm01db01vm04
    [INFO] Deleted guest vm /EXAVMIMAGES/GuestImages/dm01db01vm04 for \
    DomU dm01db01vm04
     
    [root@dm01db02 ~]# /opt/exadata_ovm/exadata.img.domu_maker \
    remove-domain dm01db02vm04
    [INFO] Start with command line: /opt/exadata_ovm/exadata.img.domu_maker \
    remove-domain dm01db02vm04
    [INFO] Shutting down DomU dm01db02vm04
    [INFO] Autostart link for dm01db02vm04 deleted from /etc/xen/auto
    [INFO] Deleted OVM repository /OVS/Repositories/1d29719ff26a4a17aca99b2f89fd8032 for DomU dm01db02vm04
    [INFO] Deleted guest vm /EXAVMIMAGES/GuestImages/dm01db02vm04  \
    for DomU dm01db02vm04