File handling in scala

#1

Please find the below screenshot for the command I used to access file in scala

image
image964Ă—228 8.7 KB

its throwing the error, file not found nut file is there in HUE.

#2

The method that you have use to refer to the file would access the file on the local file system . This would not access the file on the HDFS. You need to change the url to refer to the file on HDFS.

1 Like
#3

You got the answer to this ?

#4

This is the method to access the content
val iput = sc.textFile(“hdfs://user/<user_name> / filePath”)
This is the method to access the file
more information

#5

This is SPARK , My requirement is not having SPARK. Its NO-SPARK Programe.

#6

Hi utkarsh,

I got , from stand alone scala programe we cannot read file from HDFS, for reading from HDFS we need to sparkcontext or any other Frame work.

#7

:smiley: I was about to tell you the answer but had not implemented the solution on my terminal.

Source can only be used to read local system.

For reading HDFS we have to use some java APIs. I was trying that out but is required to put the HADOOP related jars on the scala REPL classpath and did not know that. Can you tell how you did it ??
This is the link to the answer I am referring to

Jar file should be added using the link giving below:

https://alvinalexander.com/scala/scala-how-add-jar-file-scala-repl-classpath-command-line

1 Like
#8

Not necessary we can read from HDFS using scala code, but the method needs to be found out according the answer mentioned in the source code shown in the link.

#9

Thanks Man, will check later ,now my requirement is also not having HDFS.