SPARK UI unavailable

Technical Discussions Apache Spark
#1

when I start spark shell using command
spark-shell

I see as below

SPARK_MAJOR_VERSION is set to 2, using Spark2
Setting default log level to “WARN”.
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://10.142.1.4:4040
Spark context available as ‘sc’ (master = local[*], app id = local-1570732770075).
Spark session available as ‘spark’.

but the Spark UI url is not accessible. I have tried 4-5 times I got different port numbers also, but still could not connect.

Can you please explain this ?

#2

Launch spark shell and notice the port number and IP address. In my case port number is 4041 and ip address is 10.142.1.4.

image
image1149×600 60.1 KB

From “My Lab”, figure out the public IP address:

image
image1236×533 45.1 KB

Now, using the public IP address and port number, open the url in the browser.

image
image1438×620 61.5 KB

#3

I used the same process but I got

error
error883×199 13.2 KB

Is it blocked ? Please help me solve this

#4

You can check your job status in Hue job browser

#5

Thanks for the quick reply!!
But I don’t find all the information that I used to get from Spark UI.
For example: Job, tasks level information and also SQL / Streaming tabs. Which would really help us in debug/tune our codes