Not able to find consumer_group_id to receieve data from Kafka topic

Technical Discussions Spark Streaming
#1

I have created topic in Kafka,Its tested , working fine using console producer and consumer. Now I am trying to do spark streaming using scala/spark/kafka.

I use the code as follows :
…
…
val topicMap = “partha_topic1”.split(",").map((, 1)).toMap
val lines = KafkaUtils.createStream(ssc, “ip-172-31-38-183.ec2.internal:2181”, “consumer_group_id”, topicMap).map(._2)

Can anyone help me to find out “consumer_group_id” as this value need to pass to KafkaUtils ?

#2

Hi @Partha_Sarathi_Das ,

You can give any label to your consumer group.

Please note that your message will be delivered to only one consumer in your consumer group.

Please go through this video on Spark streaming and Kafka integration.

Hope this helps.

#3

Thanks Abhinav , Its helpful video .