How can connect IntelliJ with CloudxLab

Course Discussions Big Data with Hadoop & Spark
#1

How can connect IntelliJ with cloudexlab

#2

Hi Kumar,

We do not have IntelliJ on CloudxLab as Linux servers generally do not have x window.

Generally, we write code on the local machine using our preferred IDE such as Eclipse and IntelliJ, unit test it, build the JAR and copy the JAR to CloudxLab web console using WinSCP or SCP.

Then we run the JAR either using Hadoop or Spark on CloudxLab.

We’re in the process of writing an end to end guide on how to write applications on the local machine and deploy it on any production server as well as on CloudxLab.

I’ll update you as soon as we will publish it.

If you know any better way, feel free to suggest

2 Likes
#3

How can we connect to our Intellij SBT project to our cloudx lab hdfs file system?

#4

Hi Rupesh,

You can upload the folder to CloudxLab home directory and run the sbt command inside the folder from console.

If you want to read the data from HDFS, you can simply use the location of HDFS in your code. By default, it reads from HDFS for example:

var myvar = sc.textFile("/data/mr/wordcount/input") // The location /data/mr/wordcount/input is in HDFS.