2.1 Zookeeper Setup

This topic provides the systematic instructions to install and setup the Zookeeper.

Kafka uses ZooKeeper to manage the cluster. ZooKeeper is used to coordinate the brokers/cluster topology. ZooKeeper is a consistent file system for configuration information. ZooKeeper gets used for leadership election for Broker Topic Partition Leaders. Here we are going to start a node of 2 zookeeper ensemble on 2 servers each.

1. Extract the kafka installation files in /tools/kafka on both the servers.
2. Navigate to config folder in /tools/kafka/conf.
3. Duplicate the zoo_sample.cfg and rename it to zookeeper1.cfg.
4. Open zookeeper1.cfg and modify the following properties.

   DataDir= <kafka home directory>/data
   tickTime=2000
   clientPort= Zookeeper client Port value (2181)
   initLimit=10
   syncLimit=5
   
   server.1=<hostname>:<peer port>:<leader port>
   #1 is the id that we put in myid file.
   
   server.2=<hostname>:<peer port>:<leader port>
   #2 is the id that we will put in myid file of second node.
   
   server.3=<hostname>:<peer port>:<leader port>
   #3 is the id that we will put in myid file of third.

   Example:

   
                        tickTime=2000
                     

   
                        initLimit=5
                     

   
                        syncLimit=2
                     

   
                        clientPort=2181
                     

   
                        dataDir=/tmp/zookeeper-oblm/zookeeper-node1
                     

   
                        server.1=server1-IP:2666:3666
                     

   
                        server.2=server2-IP:2667:3667
                     

   Note:

   Update the IP value with the respective server IP.
5. Duplicate the zoo.cfg file and rename it as zookeeper2.cfg in the same directory on Server 2 (Other names can also be used). These configuration files used for each of the zookeeper nodes.
6. Open zookeeper2.cfg and modify the following properties.

   
                        clientPort=2182
                     

   
                        dataDir=/tmp/zookeeper-oblm/zookeeper-node2
                     

   
                        server.1=server1-IP:2666:3666
                     

   
                        server.2=server2-IP:2667:3667
                     

   Note:

   Update the IP value with the respective server IP.
7. Copy the zookeeper1.cfg and zookeeper2.cfg and paste it in the local.
8. Open the directory /tmp/zookeeper-oblm/zookeeper-node1 on server 1 and create a file named myid, open with text editor and write 1, save and close.
9. Open the directory /tmp/zookeeper-oblm/zookeeper-node2 on server 2 and create a file named myid, open with text editor and write 2, save and close.
10. Run the command to start the zookeeper nodes.
    On Server 1:

    
                         nohup ./bin/zkServer.sh start conf/zookeep
                      

    On Server 2:

    
                         nohup ./bin/zkServer.sh start conf/zookeep