Not able to run Building Real-Time Analytics Dashboard Using Apache Spark project code

Hi,

When I was trying to do the project of the Building Real-Time Analytics Dashboard Using Apache Spark

I got the below error in the web console. Could you please help me out?

[2017-06-23 17:48:09,639] WARN Fetching topic metadata with correlation id 379 for topics [Set(order-data)] from broker [BrokerEndPoint(0,ip-172-31-13-154.ec2.int
ernal,6667)] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:122)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:77)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:76)
at kafka.producer.SyncProducer.send(SyncProducer.scala:121)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49)
at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:188)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:152)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:151)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:151)
at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:96)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:73)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2017-06-23 17:48:09,639] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(order-data)] from broker [ArrayBuff
er(BrokerEndPoint(0,ip-172-31-13-154.ec2.internal,6667))] failed (kafka.producer.async.DefaultEventHandler)

Hi Shanmukh,

Can you please create a new topic and try pushing order data to that topic?

Let’s say your username is shanmukh123 and you want to push data to shanmukh123-orderdata topic, please run below commands in web console to create a topic

export PATH=$PATH:/usr/hdp/current/kafka-broker/bin
kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic shanmukh123-orderdata

Hope this helps

HI Abhinav,

Thanks for the correction,Will try and let you know if any issue while executing again.