4 Increase Replication Factor for an existing topic

This topic provides the systematic instruction to increase Replication Factor for an existing topic.

In case, a topic is already created, and the user want to increase the replication factor.

Then, follow the below steps. Explanation is given below with an example and desired output for easier understanding.

Increase the replicas for the topic structure-closed in partition 0 from only on broker id 0 to broker id 0, 1. i.e. increase replication factor of 1 to 2.

  1. Dowload a Increasing Replication Factor file and save to the local.
  2. Command to increase the replication factor.
    ./kafka-reassign-partitions.sh --zookeeper zookeeper-server --reassignment-json-file jsonFilePath --execute
    Example:

    ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file D:\kafka\kafka_2.12-2.3.1\config\increase-replication-factor.json –execute

    Output:

    Current partition replica assignment

    {"version":1,"partitions":[{"topic":"structure-closed","partition":1,"replicas":[0],"log_dirs":["any"]},{"topic":"structure-closed","partition":0,"replicas":[1],"log_dirs":["any"]}]}

    Save this to use as the --reassignment-json-file option during rollbackSuccessfully started reassignment of partitions.

  3. Command to increase the replication factor.
    ./kafka-reassign-partitions.sh --zookeeper zookeeper-server --reassignment-json-file jsonFilePath --execute
    Example:

    ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file D:\kafka\kafka_2.12-2.3.1\config\increase-replication-factor.json –execute

    Output:

    Current partition replica assignment:

    {"version":1,"partitions":[{"topic":"structure-closed","partition":1,"replicas":[0],"log_dirs":["any"]},{"topic":"structure-closed","partition":0,"replicas":[1],"log_dirs":["any"]}]}

    Save this to use as the --reassignment-json-file option during rollbackSuccessfully started reassignment of partitions.

  4. Command to check the status of the partition reassignment.
    ./kafka-reassign-partitions.sh --zookeeper zookeeper-server --reassignment-json-file jsonFilePath --verify
    Example:

    ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file D:\kafka\kafka_2.12-2.3.1\config\increase-replication-factor.json –verify

    Output:

    Status of partition reassignment:

    Reassignment of partition structure-closed-0 completed  successfully.

  5. Describe and check the topic.
    ./kafka-topics.sh --describe --zookeeper zookeeper-server --topic topic-name
    Example:

    ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic structure-closed

    Output:

    Topic: structure-closed PartitionCount: 2  ReplicationFactor: 1 Configs:
      Topic: structure-closed Partition: 0 Leader: 1 Replicas: 0,1 Isr: 1,0
      Topic: structure-closed Partition: 1 Leader: 0 Replicas: 0  Isr: 0