Cannot load main class from JAR file


#1

I am trying to execute “spark-submit labtest.jar --master yarn --class com.lab.test.labtest”. Getting error " Cannot load main class from JAR file:/home/aswiniharidas194641/labtest.jar".

Please help me solve this error. jar file uploaded in hue.

My jar file contains below code:
package com.lab.test
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.rdd.RDD.rddToPairRDDFunctions

object labtest {
def main(args: Array[String]) = {

val conf = new SparkConf().setAppName(“WordCount”).setMaster(“local”)
val sc = new SparkContext(conf)
val input = sc.textFile(“inputforsparkwordcountapp.txt”)
input.flatMap { line => line.split(" ")
} //for each line split the line into words.
.map { word => (word, 1)}
.reduceByKey(_ + _)
.saveAsTextFile(“outputfiletest”)
sc.stop()
}

}

Thanks,
Aswini


#2

Hi, Aswini.

I understand that this is your own test .jar file that you have uploaded into your hdfs and now submitting to the spark for word count.

This is the basic syntax for this.
spark-submit --class “WordCount” --master “local[2]” target/scala-2.10/word-count_2.10-1.0.jar
I see that your class name is labtest, but you given labtest.jar in class name.
Can you check it once?

All the best.


#3

Thanks for the reply Satyajit.

spark-submit labtest.jar --master yarn --class com.lab.test.labtest -> this is the spark submit command i ran.
class name:com.lab.test.labtest
jar file name:labtest.jar.

Jar file uploaded to hdfs using hue.