Dennis Zheleznyak

Dennis Zheleznyak

DevOps Engineer bigpanda

© 2021

Move shards between Elasticsearch nodes

Since one of our nodes used most of its disk space, we needed to move some shards to other nodes.

List all the nodes on your cluster:

curl -X GET "localhost:9200/_cat/nodes"

Retreive the node id

curl -X GET "localhost:9200/_nodes/NodeName/id?pretty"

{
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "7wkfBrYbT9KgErV8_geseg" : {   <========= Node ID
      "name" : "prod-elasticsearch-12",
      "transport_address" : "xxxxxxxxxxxx",
      "host" : "prod-elasticsearch-12",
      "ip" : "xxxxxxxx",
      "version" : "xxxxxx",
      "build" : "xxxxx",
      "http_address" : "xxxxxxx",
      "attributes" : {
        "max_local_storage_nodes" : "xx",
        "rack" : "xxxx", 
        "master" : "xxxx"
      } 
    }
  }
}

Find a big index:

curl -X GET 'localhost:9200/_cat/indices?v&s=store.size:desc'

Make sure it is located on the overloaded server:

curl -X GET 'localhost:9200/_cat/shards/IndexName?v'

index                    shard prirep state      docs  store ip        node                  
IndexName$2018_3 0     p      STARTED 7998483 17.1gb 127.0.0.1 prod-elasticsearch-19 
IndexName$2018_3 0     r      STARTED 7998483 17.1gb 127.0.0.1 prod-elasticsearch-11 
IndexName$2018_3 1     p      STARTED 8000627 17.1gb 127.0.0.1 prod-elasticsearch-16 
IndexName$2018_3 1     r      STARTED 8000627 17.1gb 127.0.0.1 prod-elasticsearch-12  <==

Use the commands above to choose a destination node.

Move the replica shard 1 to the chosen server:

curl -X POST -H "Content-Type: application/json" 'http://localhost:9200/_cluster/reroute' -d '{
    "commands" : [ {
  "move" : {
   "index" : "IndexName$2018_3", "shard" : 1, "from_node" : "7wkfBrYbT9KgErV8_geseg", "to_node" : "V9xn31PsSwSlq31OTsCEqg"
  }
 } ]
}'

Monitor the cluster status:

curl -X GET 'localhost:9200/_cluster/health?pretty'

{
  "cluster_name" : "elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 9,
  "number_of_data_nodes" : 9,
  "active_primary_shards" : 5789,
  "active_shards" : 9703,
  "relocating_shards" : 1,     <==== Monitor this 
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0
}