4.2.4 Upgrading Kafka

Creating Backup for Kafka Log Files

1. Create a backup of the Kafka data:
   1. cd /u02/kafka
   2. mkdir backup
   3. cp -r kafka-logs backup/
   4. cp -r zookeeper backup/

Upgrading Kafka Broker

1. Go to Kafka home folder: cd /u01/app/kafka.
2. Stop Kafka broker services by running the command: sudo systemctl stop kafka.service.
3. Create a backup folder, by running the command, mkdir backup, and move all files into the backup folder.
4. Copy the Kafka 3.4.0 zip and extract the folder contents.
5. Move all the extracted files to Kafka parent folder: /u01/app/kafka.
6. Copy config folder files from the backup to the config folder of the new kafka.
7. Open the server.property file under the config folder and add the following parameters:

   • inter.broker.protocol.version=<version_you_are_upgrading_from> 
             

     For example,

     inter.broker.protocol.version=2.1.1

   • log.message.format.version=<version_you_are_upgrading_from>

     For example,

     log.message.format.version=2.1.1

8. Start the broker by running the command, sudo systemctl restart kafka.service, and wait for a few minutes for the cluster to stabilize; check logs for any issues.
9. Stop the broker again by running the command, sudo systemctl stop kafka.service, and update the protocol again.
10. Stop the broker by running the command, sudo systemctl stop kafka.service, and update the message format.
11. Start the broker by running the command, sudo systemctl restart kafka.service, and wait for a few minutes for the cluster to stabilize; check logs for any issues.
12. Start the broker by running the command , sudo systemctl restart kafka.service, and wait for a few minutes for the cluster to stabilize; check logs for any issues.

Now you have the latest Kafka broker running.

Upgrading Zookeeper

1. Go to Kafka parent folder i.e /u01/app/kafka
2. Stop kafka broker and kafka-zookeeper by running the following commands:

   sudo systemctl stop kafka.service

   sudo systemctl stop kafka-zookeeper.service

3. Go to config folder and open zookeeper.properties file
4. Add admin.serverPort property in the file and assign it to some port number which is not being used by any other services:

   admin.serverPort=9876

   or to disable this feature, set

   admin.enableServer=false

5. Start the zookeeper by running the command, sudo systemctl restart kafka-zookeeper.service.

   • Check if the zookeeper services are up, by running the command, ggsa-services.
   • If the services are not up, check the zookeeper logs for errors.
   • If the error is related to snapshot, then download the snapshot file from https://issues.apache.org/jira/browse/ZOOKEEPER-3056, and copy it to the location, /u02/kafka/zookeeper/version-2/.
   • Open the zookeeper.properties file, add the property, zookeeper.snapshot.trust.empty=true, and restart the zookeeper by running the command, sudo systemctl restart kafka-zookeeper.service.
6. After the zookeeper gets started, start the Kafka broker by running the command, sudo systemctl restart kafka.service.
7. Check if the services kafka.service and kafka-zookeeper.service are running fine.

Now you have an upgraded Kafka.