Removing a Node from a Cluster

To safely remove a node from a cluster, perform the following steps:

  1. Before you remove a node, ensure that each index has at least one replica. You can use the Elasticsearch Interact page to verify the replica count by choosing _cat/indices?v option in the Cluster API drop-down list.

    Alternatively, you can use the Elasticsearch Cluster Dashboard to view the replica information in PeopleSoft Health Center.

  2. Identify the IP address of the Elasticsearch node that needs to be removed from the cluster.

    Open a command prompt window and execute the following command where X.X.X.X stands for the IP address of the node that needs to be removed and also replace the host:port with the IP or port of any of the nodes in the cluster.

    curl -XPUT “http://<username:password@host:port>/_cluster/settings” -d 
    '{
      "transient" :{
          "cluster.routing.allocation.exclude._ip" : "X.X.X.X"
       }
    }'
  3. When the command is executed, Elasticsearch tries to move the existing shards out of the node that will be removed and moves it to other nodes in the cluster.

  4. In a command prompt window, execute the following command to ensure that the relocation is complete and to validate that the relocating_shards attribute shows value as 0 (zero). Alternatively, you may use the _cluster/health option from the Cluster API drop-down list on the Elasticsearch Interact page.

    curl -XGET http://<username:password@host:port>/_cluster/health?pretty

    For example, $>curl -XGET http://esadmin:esadmin@slc10uax.us.oracle.com:9200/_cluster/health?pretty.

     {
      "cluster_name" : "ES_CLUSTER1",
      "status" : "green",
      "timed_out" : false,
      "number_of_nodes" : 3,
      "number_of_data_nodes" : 3,
      "active_primary_shards" : 630,
      "active_shards" : 635,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 0,
      "delayed_unassigned_shards" : 0,
      "number_of_pending_tasks" : 0,
      "number_of_in_flight_fetch" : 0,
      "task_max_waiting_in_queue_millis" : 0,
      "active_shards_percent_as_number" : 100.0
    }
  5. Shutdown the node that was identified for removal.

  6. Set the exclusion rules to empty so that, in the future, the removed node can be added back to the cluster and can be used to hold the shards.

    curl -XPUT “http://username:password@host:port/_cluster/settings” -d 
    '{
      "transient" :{
          "cluster.routing.allocation.exclude._ip" : ""
       }
    }'
  7. On the Search Instance Properties page, delete the row corresponding to the removed node.

    See Working With Search Instances.