java.lang.IllegalArgumentException: java.net.UnknownHostException: user in Spark Scala Program

Technical Discussions Apache Spark
#1

Hi Lap Support Team,

I am getting the below error when i run collect() command in the Spark Scala window. Can you please take a look on it??

val TestFile2 = sc.textFile("//user/karthimechatronics7605/Scala/Source/HiveTestFile2.txt")

val TestFileColumnSplit = TestFile2.map(e=>e.split("|"))

TestFileColumnSplit.collect().

Error Message:
java.lang.IllegalArgumentException: java.net.UnknownHostException: user
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:406)
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:678)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:619)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)

#2

There is an extra slash in the beginning of the file name. It should start with one slash instead of two slashes.

If it notices two slashes, it thinks that you are trying to load data from network.

1 Like