5.3 Using the Heketi CLI

This section shows you how to create a cluster, and create and manage volumes using the Heketi CLI. The steps in this section should be performed on the node on which you installed the Heketi client.

Heketi cannot retrieve information about an existing cluster. New clusters must be created for them to be managed by Heketi. You can create multiple clusters with various disk types (SSD, SAS, or SATA) to suit your needs.

After Heketi is set up to manage a cluster, only use heketi-cli commands or the Heketi API to manage the cluster and volumes. You should not use gluster commands to manage clusters or volumes managed by Heketi, as it may cause inconsistencies in the Heketi database.

The RAW devices used by Heketi to create volumes must not be formatted.

The hostnames and IP addresses uses in the examples in this section are:

  • node1.example.com (192.168.1.51)

  • node2.example.com (192.168.1.52)

  • node3.example.com (192.168.1.53)

When you run heketi-cli commands, you need to specify the Heketi API server location, and if authentication has been set up, the authentication information. You can do this either by passing those options when running heketi-cli commands, or set environment variables. The heketi-cli syntax to use is:

heketi-cli --server=URL --user=username --secret=key command

If you would prefer to use environment variables, the environment variable names are as shown in this example.

# export HEKETI_CLI_SERVER=http://node1.example.com:8080
# export HEKETI_CLI_USER=admin
# export HEKETI_CLI_KEY=key

The examples in this section use environment variables to make the commands easier to read.

5.3.1 Creating a Cluster

This section discusses creating a cluster with the Heketi CLI.

To create a cluster:

  1. Create Heketi topology configuration file to set up the cluster. Copy the /usr/share/heketi/topology-sample.json to a new file, for example:

    # cp /usr/share/heketi/topology-sample.json /usr/share/heketi/topology-mycluster.json
  2. The topology file is in JSON format, and can contain an array of clusters. Each cluster contains an array of nodes. Each node contains the node hostname and IP address, the devices available on the node, and the failure domain (zone) on which the node should be included.

    Edit the /usr/share/heketi/topology-mycluster.json file to to suit your environment. For example:

    {
      "clusters": [
        {
          "nodes": [
            {
              "node": {
                "hostnames": {
                  "manage": [
                    "node1.example.com"
                  ],
                  "storage": [
                    "192.168.1.51"
                  ]
                },
                "zone": 1
              },
              "devices": [
                {
                  "name": "/dev/sdb",
                  "destroydata": false
                }
              ]
            },
            {
              "node": {
                "hostnames": {
                  "manage": [
                    "node2.example.com"
                  ],
                  "storage": [
                    "192.168.1.52"
                  ]
                },
                "zone": 1
              },
              "devices": [
                {
                  "name": "/dev/sdb",
                  "destroydata": false
                }
              ]
            },
            {
              "node": {
                "hostnames": {
                  "manage": [
                    "node3.example.com"
                  ],
                  "storage": [
                    "192.168.1.53"
                  ]
                },
                "zone": 1
              },
              "devices": [
                {
                  "name": "/dev/sdb",
                  "destroydata": false
                }
              ]
            }
          ]
        }
      ]
    }
  3. Load the Heketi topology file into Heketi using the heketi-cli topology load command to create a cluster. The clusters, nodes and disks specified in the JSON file are created.

    # heketi-cli topology load --json=heketi-mycluster.json 
    Creating cluster ... ID: 7c1cf54ff4b5ab41f823ac592ba68ca5
    	Allowing file volumes on cluster.
    	Allowing block volumes on cluster.
    	Creating node node1.example.com ... ID: c35ba48b042555633b511f459f5aa157
    		Adding device /dev/sdb ... OK
    	Creating node node2.example.com ... ID: 7c899bc9f50e46efc993dc22263549e4
    		Adding device /dev/sdb ... OK
    	Creating node node3.example.com ... ID: 32755ad123c325f75c91aa963c4312f3
    		Adding device /dev/sdb ... OK
  4. You can get a list of clusters managed by Heketi using the heketi-cli cluster list command:

    # heketi-cli cluster list
    Clusters:
    Id:7c1cf54ff4b5ab41f823ac592ba68ca5 [file][block]
  5. You can get more information about each cluster managed by Heketi using the heketi-cli topology info command:

    # heketi-cli topology info
    
    Cluster Id: 7c1cf54ff4b5ab41f823ac592ba68ca5
    
        File:  true
        Block: true
    
        Volumes:
    
        Nodes:
    
    	Node Id: 32755ad123c325f75c91aa963c4312f3
    	State: online
    	Cluster Id: 7c1cf54ff4b5ab41f823ac592ba68ca5
    	Zone: 1
    	Management Hostnames: node3.example.com
    	Storage Hostnames: 192.168.1.53
    	Devices:
    		Id:5917085ef4a7beca4f7c61138d152460   Name:/dev/sdb            State:online    
                 Size (GiB):500     Used (GiB):0       Free (GiB):500     
    			Bricks:
    
    	Node Id: 7c899bc9f50e46efc993dc22263549e4
    	State: online
    	Cluster Id: 7c1cf54ff4b5ab41f823ac592ba68ca5
    	Zone: 1
    	Management Hostnames: node2.example.com
    	Storage Hostnames: 192.168.1.52
    	Devices:
    		Id:855490c8fb09e4f21caae9f421f692b0   Name:/dev/sdb            State:online
                 Size (GiB):500     Used (GiB):0       Free (GiB):500     
    			Bricks:
    
    	Node Id: c35ba48b042555633b511f459f5aa157
    	State: online
    	Cluster Id: 7c1cf54ff4b5ab41f823ac592ba68ca5
    	Zone: 1
    	Management Hostnames: node1.example.com
    	Storage Hostnames: 192.168.1.51
    	Devices:
    		Id:fbf747dc6ccf811fce0196d8280a32e3   Name:/dev/sdb            State:online
                 Size (GiB):500     Used (GiB):0       Free (GiB):500     
    			Bricks: