Kafka Topic not listed in /kafka-logs

#1

Hi ,

I have been trying to ingest the data through streaming in kafka topic by running the following two command

./tail_logs.sh | kafka-console-producer.sh --broker-list cxln1.c.thelab-240901.internal:6667,cxln4.c.thelab-240901.internal:6667
–topic ashu_retail1

kafka-console-consumer.sh --bootstrap-server cxln1.c.thelab-240901.internal:6667,cxln4.c.thelab-240901.internal:6667 --topic ashu_reta
il1 --from-beginning

But how can i look for the data present in kafka topic.I tried to find it in /kafka-logs directory as mentioned in ambari but there my topic is not listed.But when i run kafka binaries to list the Kafka topics i cn see my topic created.

So my doubt is how to see the log files created inside kafka topic

#2

Hi,
Kindly follow the below steps.

  1. export PATH=$PATH:/usr/hdp/current/kafka-broker/bin
  2. kafka-topics.sh --create --zookeeper cxln1.c.thelab-240901.internal:2181 --replication-factor 1 --partitions 1 --topic Your_topic_name.
  3. kafka-topics.sh --list --zookeeper cxln1.c.thelab-240901.internal:2181
  4. kafka-console-producer.sh --broker-list cxln4.c.thelab-240901.internal:6667 --topic Your_topic_name
  5. ls /brokers/ids

All the best!

#3

Hi satyajit.
I tried running all the commands as suggested by you but the last command was not working so I started
zookeeper-client and then ran the following command

get /brokers/topics/ashu_topic_name
but this shows only the metadata about the partition.How to see the actual data

2020-05-06_17h40_09
2020-05-06_17h40_09841×308 11 KB