How can connect IntelliJ with cloudexlab
We do not have IntelliJ on CloudxLab as Linux servers generally do not have x window.
Generally, we write code on the local machine using our preferred IDE such as Eclipse and IntelliJ, unit test it, build the JAR and copy the JAR to CloudxLab web console using WinSCP or SCP.
Then we run the JAR either using Hadoop or Spark on CloudxLab.
We’re in the process of writing an end to end guide on how to write applications on the local machine and deploy it on any production server as well as on CloudxLab.
I’ll update you as soon as we will publish it.
If you know any better way, feel free to suggest
How can we connect to our Intellij SBT project to our cloudx lab hdfs file system?
You can upload the folder to CloudxLab home directory and run the
sbt command inside the folder from console.
If you want to read the data from HDFS, you can simply use the location of HDFS in your code. By default, it reads from HDFS for example:
var myvar = sc.textFile("/data/mr/wordcount/input") // The location /data/mr/wordcount/input is in HDFS.