Spark-submit cluster mode - NoClassDefFoundError


#1

i am getting below error while submitting spark job in cluster mode.
–deploy-mode client model is working fine
/usr/spark2.0.2/bin/ spark-submit --name hello --master yarn --deploy-mode client --executor-memory 1g --executor-cores 1 --num-executors 1 --class com.first.load.Hello /home/udaychitukula6587/first.load/target/scala-2.10/helloworld_2.10-1.0.jar

Below one throwing error:
/usr/spark2.0.2/bin/ spark-submit --name hello --master yarn --deploy-mode cluster --executor-memory 1g --executor-cores 1 --num-executors 1 --class com.first.load.Hello /home/udaychitukula6587/first.load/target/scala-2.10/helloworld_2.10-1.0.jar
ERROR:
:: problems summary ::
:::: ERRORS
_ unknown resolver null_

:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
:: retrieving :: org.apache.spark#spark-submit-parent
_ confs: [default]_
_ 0 artifacts copied, 17 already retrieved (0kB/19ms)_
Exception in thread “main” java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig
_ at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)_
_ at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:181)_
_ at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:168)_

this program is to print welcome message

please help me in resolving this
thanks in advance


#2

could someone please look into the issue???


#3

Hi @Uday_Reddy,

Spark 2.x will not work in YARN mode currently. It is a pending issue which will solve during the cluster upgrade.

We will let you know as soon as we will upgrade the cluster. Extremely sorry for the inconvenience. Please run Spark 2.x in local mode for now.

Thanks


#4

Hi,
Is still this issue exists, I am getting same issue.


#5

I have tested the yarn mode multiple times. It seems to work. Please share the code.