Kafka spark streaming program is not working

Technical Discussions Apache Kafka
#1

Hi Sir,

I have followed below video to run example of kafka spark streaming :

However I didnt get any consumer console after running program through sparksubmit command

[madhavink145361@ip-172-31-38-146 word_count_kafka]$ spark-submit --class “KafkaWordCount” --jars spark-streaming-kafka-assembly_2.
10-1.5.2.jar target/scala-2.12/kafkawordcount_2.12-0.1.0-SNAPSHOT.jar ip-172-31-20-247.ec2.internal:2181 spark-streaming-consumer-g
roup madhavi_test 2>/dev/null

Program terminates without reading any data

Could you please suggest ?

#2

Hi @MADHAVI_NAIK,

Probably you will have to try different spark-streaming-kafka-assembly JAR which is compatible with the Spark version which you are using. We do not have Spark 1.5 in the lab now.

Hope this helps.

Thanks

#3

Hi Abhinav,
I see we are using spark version 2 in lab, which jar version for kafka streaming version is expected?
producer is up on one terminal and I am running my program on another terminal.
program gets finished and not pompting for any error.

could you please help?

Thanks,
Madhavi Naik

#4

Hi @MADHAVI_NAIK,

Just curious if you had checked the official documentation for the same. If not then this official documentation will help you with the same.

Hope this helps.

Thanks