Kafka producer throws error

Hi,
I am getting below error when I send anything through producer program in Kafka.
I was able to create the topic without issues but unable to send any messages via producer as it is throwing below error. The error is very long hence I have pasted only a small portion of it.
I created the topic using following commands as you had mentioned in another discussion.

export PATH=$PATH:/usr/hdp/current/kafka-broker/bin kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic GurmeetKafka
ERROR:

WARN Fetching topic metadata with correlation id 0 for topics [Set(GurmeetKafka)] from broker [BrokerEndPoint(0,localhost,9092)] failed (
kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:122)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:77)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:76)
at kafka.producer.SyncProducer.send(SyncProducer.scala:121)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:68)
at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:79)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.CoreUtils$.swallowError(CoreUtils.scala:51)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:68)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2018-03-04 14:36:18,773] ERROR fetching topic metadata for topics [Set(GurmeetKafka)] from broker [ArrayBuffer(BrokerEndPoint(0,localhost,9092))] failed (kafka.ut
ils.CoreUtils$)

HI Gurmeet,

Instead of localhost try using the zookeeper ip add with the port number.

  1. export PATH=$PATH:/usr/hdp/current/kafka-broker/bin

  2. kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic GurmeetKafka

Amol

1 Like