Hi
As I want to work with hiveContext but when entered in spark1.6/bin/spark-shell.
I found a sqlContext point to SQLContext since I need to be in hiveContext I tried below the line
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
but I am getting an error while loading hiveContext. can you please help on the same and if I do something wrong please guide me .
Thank you
It’s working good with spark 2.0.1 but I want to be on spark 1.6
I was doing a mistake.
When we run /usr/spark1.6/bin/spark-shell it gives two object automatically created spark 1.6 ( sc and sqlContext) and spark 2.x ( sc and spark )
when we run sqlContext it points to HiveContext by default.
But my bad I was initiated sqlContext with SQLContext by below lines.
import org.apache.spark.sql.SQLContext
val sqlContext = new SQLContext(sc)
so sqlContext started pointing to SQLContext after running above code. and I got an error when again I loaded sqlContext as HiveContext as per below line
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
If anyone faces the same issue then the need to open fresh spark-shell terminal.
Great Mahesh!
I am very happy that you were able to resolve your problem by yourselves and also shared with others.
This will definitely help others.