How to access Spark 2.1?

senthiljdpm · July 17, 2018, 10:38am

Hi,

I am trying to access Spark 2.1 but Cloudxlab supports version upto 2.0. I am trying to implement Watermark concept with Spark Structured Streaming which is available from version 2.1 onwards. May i know how to access Spark 2.1 in Cloudxlab?

utkarsh_rathor · July 17, 2018, 4:32pm

The /usr directory has many versions of spark in it. You can choose the one that you want.

senthiljdpm · July 21, 2018, 3:52am

got it. thanks a lot