Removing a Node From a Cluster

This section provides instructions on how to remove a node on a global cluster or a zone cluster. You can also remove a specific zone cluster from a global cluster. The following table lists the tasks to perform to remove a node from an existing cluster. Perform the tasks in the order shown.

---

Caution - If you remove a node using only this procedure for a RAC configuration, the removal might cause the node to panic during a reboot. For instructions on how to remove a node from a RAC configuration, see How to Remove Support for Oracle RAC From Selected Nodes in Oracle Solaris Cluster Data Service for Oracle Real Application Clusters Guide. After you complete that process, remove a node for a RAC configuration, follow the appropriate steps below.

---

Table 8-2 Task Map: Removing a Node

How to Remove a Node From a Zone Cluster

You can remove a node from a zone cluster by halting the node, uninstalling it, and removing the node from the configuration. If you decide later to add the node back into the zone cluster, follow the instructions in Table 8-1. Most of these steps are performed from the global-cluster node.

1. Assume the root role on a node of the global cluster.
2. Shut down the zone-cluster node you want to remove by specifying the node and its zone cluster.

   phys-schost# clzonecluster halt -n node zoneclustername

   You can also use the clnode evacuate and shutdown commands within a zone cluster.

3. Remove the node from all resource groups in the zone cluster.

   phys-schost# clrg remove-node -n zonehostname -Z zoneclustername rg-name

4. Uninstall the zone-cluster node.

   phys-schost# clzonecluster uninstall -n node zoneclustername

5. Remove the zone-cluster node from the configuration.

   Use the following commands:

   phys-schost# clzonecluster configure zoneclustername

   clzc:sczone> remove node physical-host=node

   clzc:sczone> exit

6. Verify that the node was removed from the zone cluster.

   phys-schost# clzonecluster status

How to Remove a Node From the Cluster Software Configuration

Perform this procedure to remove a node from the global cluster.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical.

1. Ensure that you have removed the node from all resource groups, device groups, and quorum device configurations and put it into maintenance state before you continue with this procedure.
2. Assume a role that provides solaris.cluster.modify RBAC authorization on the node that you want to remove.

   Perform all steps in this procedure from a node of the global cluster.

3. Boot the global-cluster node that you want to remove into noncluster mode.

   For a zone-cluster node, follow the instructions in How to Remove a Node From a Zone Cluster before you perform this step.

   • On SPARC based systems, run the following command.

     ok boot -x

   • On x86 based systems, run the following commands.

     shutdown -g -y -i0
     
     Press any key to continue

   1. In the GRUB menu, use the arrow keys to select the appropriate Oracle Solaris entry and type e to edit its commands.

      For more information about GRUB based booting, see Booting a System in Booting and Shutting Down Oracle Solaris 11.1 Systems.

   2. In the boot parameters screen, use the arrow keys to select the kernel entry and type e to edit the entry.
   3. Add -x to the command to specify system boot into noncluster mode.

      [ Minimal BASH-like line editing is supported. For the first word, TAB
      lists possible command completions. Anywhere else TAB lists the possible
      completions of a device/filename. ESC at any time exits. ]
      
      grub edit> kernel$ /platform/i86pc/kernel/#ISADIR/unix -B $ZFS-BOOTFS -x

   4. Press the Enter key to accept the change and return to the boot parameters screen.

      The screen displays the edited command.

   5. Type b to boot the node into noncluster mode.

      This change to the kernel boot parameter command does not persist over the system boot. The next time you reboot the node, it will boot into cluster mode. To boot into noncluster mode instead, perform these steps again to add the -x option to the kernel boot parameter command.

      ---

      Note - If the node to be removed is not available or can no longer be booted, run the following command on any active cluster node: clnode clear -F <node-to-be-removed>. Verify the node removal by running clnode status <nodename>.

      ---

4. Delete the node from the cluster.

   Run the following command from an active node:

   phys-schost# clnode clear -F nodename

   If you have resource groups that are have rg_system=true, you must change them to rg_system=false so that the clnode clear -F command will succeed. After you run clnode clear -F, reset the resource groups back to rg_system=true.

   Run the following command from the node you want to remove:

   phys-schost# clnode remove -F

   ---

   Note - If you are removing the last node in the cluster, the node must be in noncluster mode with no active nodes left in the cluster.

   ---

5. From another cluster node, verify the node removal.

   phys-schost# clnode status nodename

6. Complete the node removal.
   • If you intend to uninstall the Oracle Solaris Cluster software from the removed node, proceed to How to Uninstall Oracle Solaris Cluster Software From a Cluster Node. You could also choose to remove the node from the cluster and uninstall the Oracle Solaris Cluster software at the same time. Change to a directory that does not contain any Oracle Solaris Cluster files and type scinstall -r.
   • If you do not intend to uninstall the Oracle Solaris Cluster software from the removed node, you can physically remove the node from the cluster by removing the hardware connections as described in Oracle Solaris Cluster 4.1 Hardware Administration Manual.

Example 8-2 Removing a Node From the Cluster Software Configuration

This example shows how to remove a node (phys-schost-2) from a cluster. The clnode remove command is run in noncluster mode from the node you want to remove from the cluster (phys-schost-2).

[Remove the node from the cluster:]
phys-schost-2# clnode remove
phys-schost-1# clnode clear -F phys-schost-2
[Verify node removal:]
phys-schost-1# clnode status
-- Cluster Nodes --
                    Node name           Status
                    ---------           ------
  Cluster node:     phys-schost-1       Online

See Also

To uninstall Oracle Solaris Cluster software from the removed node, see How to Uninstall Oracle Solaris Cluster Software From a Cluster Node.

For hardware procedures, see the Oracle Solaris Cluster 4.1 Hardware Administration Manual.

For a comprehensive list of tasks for removing a cluster node, seeTable 8-2.

To add a node to an existing cluster, see How to Add a Node to an Existing Cluster.

How to Remove Connectivity Between an Array and a Single Node, in a Cluster With Greater Than Two-Node Connectivity

Use this procedure to detach a storage array from a single cluster node, in a cluster that has three-node or four-node connectivity.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical.

1. Back up all database tables, data services, and volumes that are associated with the storage array that you are removing.
2. Determine the resource groups and device groups that are running on the node to be disconnected.

   phys-schost# clresourcegroup status
   phys-schost# cldevicegroup status

3. If necessary, move all resource groups and device groups off the node to be disconnected.

   ---

   Caution (SPARC only) - If your cluster is running Oracle RAC software, shut down the Oracle RAC database instance that is running on the node before you move the groups off the node. For instructions, see the Oracle Database Administration Guide.

   ---

   phys-schost# clnode evacuate node

   The clnode evacuate command switches over all device groups from the specified node to the next-preferred node. The command also switches all resource groups from the specified node to the next-preferred node.

4. Put the device groups into maintenance state.

   For the procedure on putting a device group in maintenance state, see How to Put a Node Into Maintenance State.

5. Remove the node from the device groups.

   If you use a raw disk, use the cldevicegroup(1CL) command to remove the device groups.

6. For each resource group that contains an HAStoragePlus resource, remove the node from the resource group's node list.

   phys-schost# clresourcegroup remove-node -n node + | resourcegroup

   node

   The name of the node.

   See the Oracle Solaris Cluster Data Services Planning and Administration Guide for more information about changing a resource group's node list.

   ---

   Note - Resource type, resource group, and resource property names are case sensitive when clresourcegroup is executed.

   ---

7. If the storage array that you are removing is the last storage array that is connected to the node, disconnect the fiber-optic cable between the node and the hub or switch that is connected to this storage array.

   Otherwise, skip this step.

8. If you are removing the host adapter from the node that you are disconnecting, and power off the node.

   If you are removing the host adapter from the node that you are disconnecting, skip to Step 11.

9. Remove the host adapter from the node.

   For the procedure on removing host adapters, see the documentation for the node.

10. Without booting the node, power on the node.
11. If Oracle RAC software has been installed, remove the Oracle RAC software package from the node that you are disconnecting.

    phys-schost# pkg uninstall /ha-cluster/library/ucmm 

    ---

    Caution (SPARC only) - If you do not remove the Oracle RAC software from the node that you disconnected, the node panics when the node is reintroduced to the cluster and potentially causes a loss of data availability.

    ---

12. Boot the node in cluster mode.

    • On SPARC based systems, run the following command.

      ok boot

    • On x86 based systems, run the following commands.

      When the GRUB menu is displayed, select the appropriate Oracle Solaris entry and press Enter.

13. On the node, update the device namespace by updating the /devices and /dev entries.

    phys-schost# devfsadm -C 
     cldevice refresh

14. Bring the device groups back online.

    For information about bringing a device group online, see How to Bring a Node Out of Maintenance State.

How to Correct Error Messages

To correct any error messages that occurred while attempting to perform any of the cluster node removal procedures, perform the following procedure.

1. Attempt to rejoin the node to the global cluster.

   Perform this procedure only on a global cluster.

   phys-schost# boot

2. Did the node successfully rejoin the cluster?

   • If no, proceed to Step b.

   • If yes, perform the following steps to remove the node from device groups.

   1. If the node successfully rejoins the cluster, remove the node from the remaining device group or groups.

      Follow procedures in How to Remove a Node From All Device Groups.

   2. After you remove the node from all device groups, return to How to Uninstall Oracle Solaris Cluster Software From a Cluster Node and repeat the procedure.
3. If the node could not rejoin the cluster, rename the node's /etc/cluster/ccr file to any other name you choose, for example, ccr.old.

   # mv /etc/cluster/ccr /etc/cluster/ccr.old

4. Return to How to Uninstall Oracle Solaris Cluster Software From a Cluster Node and repeat the procedure.