Error not able to import sql Spark session import org.apache.spark.sql.SparkSession

Lab Support
#1

Getting error while importing sparksession on spark-shell.
Is it due to spark version? I believe sql.sparksession doesn’t support version > 2.0.
Please confirm.

import org.apache.spark.sql.SparkSession

error is :
object SparkSession is not a member of package org.apache.spark.sql

#2

SparkSession is introduced in Spark 2.0. Your need to use Spark 2.0.0 or later to import it.
I think the default version on cloudxlab is not 2.0
Try this for launching shell and give a try
#if using scala spark shell
/usr/spark2.0.1/bin/spark-shell

#if using python - PySpark
/usr/spark2.0.1/bin/pyspark

Reference : https://cloudxlab.com/blog/access-spark-cloudxlab/

1 Like