Spark-submit class not found exception

Hi Team,

I am trying to run this simple spark application with the spark submit command.

package org.apache.spark.SparkProgram


  • Created by Maheshn on 8/17/2017.
    import org.apache.spark.SparkConf
    import org.apache.spark.SparkContext
    import org.apache.spark.SparkContext._

object SparkTest {
def main(args: Array[String]): Unit = {

    val conf = new SparkConf().setAppName("count")
    val sc = new SparkContext(conf)
    val list = List("Hello world", "Hello Mahesh howe are you", "Hello s", "sql context is available as sqlcontext")
    val parallel = sc.parallelize(list)
    val record = parallel.flatMap(rec => rec.split(" ")).map(rec => (rec, 1)).reduceByKey(_ + _).map(rec => rec.productIterator.mkString("\t"))



When i try to run it with below command

spark-submit --class " SparkTest" --master local[4] sparkdemo_2.11-1.0.jar

I encounter classNotFoundException .

Please help .Thank you

Resolved …

it was an issue in giving below command

spark-submit --class " SparkTest" --master local[4] sparkdemo_2.11-1.0.jar

Instead of above i gave command

spark-submit --class org.apache.spark.SparkProgram.SparkTest --master yarn /home/user/sparkdemo_2.11-1.0.jar


Thank Mahesh for posting the answer here so that others could also use it.

For others, you need to specify the fully qualified name in -class argument.

Hi Mahesh/Sandip Sir,

Please help on the below issue while executing the spark application using spark-submit.
spark-submit --class org.apache.spark.project.Demo2 --master yarn /home/mdakshita01218889/SparkApp1-0.0.1-SNAPSHOT.jar


21/08/17 13:42:59 INFO EventLoggingListener: Logging events to hdfs:///spark2-history/local-1629207777822
21/08/17 13:43:03 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0) unexpected exception type

java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/runtime/LambdaDeserialize
at org.apache.spark.project.Demo2$.$deserializeLambda$(Demo2.scala)

in my laptop scala version is 2.13, spark dependent packages are as below in pom.xml file


Please help on the above. I have struggling with this issue since so many days.
Below is my spark application program it is running well in eclipse ide.
package org.apache.spark.project
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions._

object Demo2{
case class Emp(id:Int,name:String)
def main(args:Array[String]){
val spark = SparkSession.builder().master(“local”).appName(“exampleOfUdf”).getOrCreate


import spark.implicits._
val l1 = List((101,"madhu"),(102,"sruthi"))
val rdd1 = spark.sparkContext.parallelize(l1)
val df1 =>Emp(x._1,x._2))
val ds1 = df1.toDS()

val ds2 =>{
       val list1 = List(,
  val list1 = List(x._1,x._2)