HDFS FILE PATH not found in spark,


#1

i am tryin to read file from HDFS in spark,

scala> val input1=sc.textFile(“hdfs://ip-172-31-35-141.ec2.internal:8020/user/kumarrupesh2389619/input1.txt”)

its saying file path not found.


#2

Hi @Rupesh_Kumar,

Does file really exist in that path?

what does below command say?

hadoop fs -ls hdfs://ip-172-31-35-141.ec2.internal:8020/user/kumarrupesh2389619/input1.txt


#3

@Rupesh_Kumar You can also directly mention the path in following way

val input1 = sc.textFile(“hdfs:///user/kumarrupesh2389619/input1.txt”)