Kafka: Failed to Update Metadata After 60000 Ms

Kafka : Failed to update metadata after 60000 ms with only one broker down

Posting the answer for someone who might face this issue -

The reason is older version of Kafka Producer. The kafka producers take bootstrap servers as list. In older versions, for fetching metadata, producers will try to connect with all the servers in Round Robin fashion. So, if one of the broker is down, the requests going to this server will fail and this message will come.

Solution:

• Upgrade to newer producer version.
• can reduce metadata.fetch.timeout.ms settings: This will ensure the main thread is not getting blocked and send will fail soon. Default value is 60000ms. Not needed in higher version

Note: Kafka send method is blocked till the producer is able to write to buffer.

org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms

I was having this same problem today. I'm a newbie at Kafka and was simply trying to get a sample Java producer and consumer running. I was able to get the consumer working, but kept getting the same "topic not present in metadata" error as you, with the producer.

Finally, out of desperation, I added some code to my producer to dump the topics. When I did this, I then got runtime errors because of missing classes in packages jackson-databind and jackson-core. After adding them, I no longer got the "topic not present" error. I removed the topic-dumping code I temporarily added, an it still worked.

In apache Kafka, who would populate the RecordMetadata and where does this information reside once the producer got it?

You have no records, so it's actually topic metadata.

As answered on your previous question, you'll get these logs if you cannot connect to the bootstrap addresses.

Zookeeper holds topic metadata

I'm getting Topic not present in metadata after 60000 ms message on some computers

192.168.10.10:9092

This seems to be an internal IP. Check if the clients where you cannot access are within its network range i.e. whether they can access this IP.

Try doing a telnet from your client machine..

telnet 192.168.10.10 9092

If you are not able to telnet then give the IP which can be accessed by your clients and ensure the same in the advertised.listeners also.

Also check your advertised.listeners config. When we connect to a url given in the bootstrap.servers that typically should par with those in the advertised.listeners configuration.

Topic metadata not present means that your client is unable to fetch any information about the given topic i.e. it cannot get metadata through the given bootstrap.servers property.

---

Related Topics