Monitor Spark SQL Jobs in Spark History Server


#1

I’m running few spark SQL commands and i would like to view stages/tasks/jobs. I couldn’t find them neither on HUE job browser not Spark History Server UI. Request you to please help on this


#2

You can view them in spark UI. Notice the port number when the spark-shell or pyspark starts and open the same using the public URL.