Kafka - Broker: Group Coordinator Not Available

The group coordinator is not available-Kafka

I faced a similar issue. The problem was when you start your Kafka broker there is a property associated with it, "KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR". If you are working with a single node cluster make sure you set this property with the value '1'. As its default value is 3. This change resolved my problem. (you can check the value in Kafka.properties file)
Note: I was using base image of confluent kafka version 4.0.0 ( confluentinc/cp-kafka:4.0.0)

Kafka - Broker: Group coordinator not available

Try to add properties into the server.conf and clean zookeeper cache.
It should help

offsets.topic.replication.factor=3
default.replication.factor=3

Root cause of this issue is impossibility to distribute topic offsets between nodes.

Auto generated topic:
__consumer_offsets

You can check it by

$ ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic __consumer_offsets

Pay attention to this article:
https://kafka.apache.org/documentation/#prodconfig

by default it creates __consumer_offsets with RF - 1

Important thing is to configure replication factor before the kafka/cluster start.
Otherwise it can bring some issues with re configuring instances like in your case.

Group coordinator lookup failed: The coordinator is not available

It is indeed a cluster problem. There is a problem with the __consumer_offsets topic data of kafka. It is good to restart kafka after deleting.

Kafka Connect Distributed mode The group coordinator is not available

After writing a connector in Go, I came across the same issue. I was forced to solve it myself.

When a connector connects to kafka it automatically writes to the topics and to __offset_topics. When a connector crashes, it leaves trace of itself in those tables as the coordinator. When a new connector starts up it finds the record in the table and attempts to communicate with the coordinator. The coordinator fails to respond and the connector never works.

You can fix this one of two ways, Delete all the topics (connect-configs, connect-offsets, connect-status, __offset_topics) and restart the cluster. The other method is to remove the coordinator from the topics, which I am currently unsure how to perform.

---

Related Topics