2.12.3 Remove the Database Server from the Cluster

If you are reimaging a failed server or repurposing a server, follow the steps in this task to remove the server from the cluster before you reimage it. If you are reimaging the server for a different reason, skip this task and proceed with the reimaging task next.

The steps in this task are performed using a working database server in the cluster. In the following commands, working_server is a working database server, and failed_server is the database server you are removing, either because it failed or it is being repurposed.

1. Log in as the oracle or grid user on a database server in the cluster.
   Log in as the user that owns the Oracle Grid Infrastructure software installation.
2. Disable the listener that runs on the failed server.

   $ srvctl disable listener -n failed_server
   $ srvctl stop listener -n failed_server
   

3. Delete the Oracle home from the Oracle inventory.

   In the following command, list_of_working_servers is a list of the servers that are still working in the cluster, such as dm01db02, dm01db03, and so on.

   In the following command, replace /u01/app/oracle/product/12.1.0.2/dbhome_1 with the location of your Oracle Database home directory.

   $ cd $ORACLE_HOME/oui/bin
   $ ./runInstaller -updateNodeList ORACLE_HOME= \
   /u01/app/oracle/product/12.1.0.2/dbhome_1 "CLUSTER_NODES=list_of_working_servers"
   

4. Log in as the grid user on the database server.
   The grid user refers to the operating system user that owns the Oracle Grid Infrastructure software installation. The $ORACLE_HOME variable should point to the location of the Grid home.
5. Verify the failed server is unpinned.

   $ olsnodes -s -t
   

   The following is an example of the output from the command:

   dm01adm05        Inactive        Unpinned
   dm01adm06        Active          Unpinned
   dm01adm07        Active          Unpinned
   dm01adm08        Active          Unpinned
   

6. Log in as the root user on the database server.
7. Stop and delete the VIP resources for the failed database server.

   # srvctl stop vip -i failed_server-vip
   PRCC-1016 : failed_server-vip.example.com was already stopped
   
   # srvctl remove vip -i failed_server-vip
   Please confirm that you intend to remove the VIPs failed_server-vip (y/[n]) y
   

8. Delete the server from the cluster.

   # crsctl delete node -n failed_server
   CRS-4661: Node dm01db01 successfully deleted.
   

   If you receive an error message similar to the following, then relocate the voting disks.

   CRS-4662: Error while trying to delete node dm01db01.
   CRS-4000: Command Delete failed, or completed with errors.
   

   To relocate the voting disks use the following steps:

   1. Determine the current location of the voting disks.

      # crsctl query css votedisk
      

      The following is an example of the output from the command. The current location is DBFS_DG.

      ##  STATE    File Universal Id          File Name                Disk group
      --  -----    -----------------          ---------                ----------
      1. ONLINE   123456789abab (o/192.168.73.102/DATA_CD_00_dm01cel07) [DBFS_DG]
      2. ONLINE   123456789cdcd (o/192.168.73.103/DATA_CD_00_dm01cel08) [DBFS_DG]
      3. ONLINE   123456789efef (o/192.168.73.100/DATA_CD_00_dm01cel05) [DBFS_DG]
      Located 3 voting disk(s).
      

   2. Relocate the voting disks to another disk group.

      # ./crsctl replace votedisk +DATA
      
      Successful addition of voting disk 2345667aabbdd.
      ...
      CRS-4266: Voting file(s) successfully replaced
      

   3. Relocate the voting disks to the original location using a command similar to the following:

      # ./crsctl replace votedisk +DBFS_DG
      

   4. Delete the server from the cluster.
9. Log in as the grid user on the database server.
   The grid user refers to the operating system user that owns the Oracle Grid Infrastructure software installation. The $ORACLE_HOME variable should point to the location of the Grid home.
10. Update the Oracle inventory.

    In the following command, replace /u01/app/12.1.0.2/grid with the location of your Oracle Grid Infrastructure home directory.

    $ cd $ORACLE_HOME/oui/bin
    $ ./runInstaller -updateNodeList ORACLE_HOME=/u01/app/12.1.0.2/grid \
      "CLUSTER_NODES=list_of_working_servers" CRS=TRUE
    

11. Verify the server was deleted successfully.

    $ cluvfy stage -post nodedel -n failed_server -verbose
    

    The following is an example of the output from the command:

    Performing post-checks for node removal
    Checking CRS integrity...
    The Oracle clusterware is healthy on node "dm01db02"
    CRS integrity check passed
    Result:
    Node removal check passed
    Post-check for node removal was successful.