How to execute HMaster of Hbase using start-hbase.sh command in lab

mearupmukherjee · July 16, 2020, 3:39pm

Hi

Could you please let me know the path of Hbase and how to run command below
start-hbase.sh to run HMaster in lab.

I want to include Hbase jars while starting spark to include Hbase from spark.

Thanks
Arup Mukherjee

satyajit_das · July 16, 2020, 4:46pm

Hi,
Just type hbase shell in your web-console and you will get the Hbase terminal.
KIndly refer the below for include Hbase jars while starting spark to include Hbase from spark.

mearupmukherjee · July 16, 2020, 6:46pm

Hi
Thanks
Actually Could you please provide me Hbase Jar file path of Clodexlab, I need that path while calling spark-shell. The document you mentioned that is generalized one but I need specific to Cloudexlab so I can use it in this lab.

Thanks
Arup Mukherjee

satyajit_das · July 17, 2020, 4:32am

Hi,

You can find the Hbase related file in the path “/usr/hdp/2.6.2.0-205/”
But you do not have permissions to do any changes for security reasons.
That is common folders for all learners where the required files are stored.

mearupmukherjee · August 15, 2020, 3:05pm

Hi

I have tried to open spark-shell with HBASE classpath but not getting success.

I am trying as per the below tutorial

Could you plz check.

$ ../hbase-0.98.9-hadoop2/bin/start-hbase.sh

$ HBASE_PATH=…/hbase-0.98.9-hadoop2/bin/hbase classpath``

$ bin/spark-shell --driver-class-path $HBASE_PATH

scala>

You are now ready to enter scala code in the interactive shell.

scala> import org.apache.hadoop.hbase.HBaseConfiguration

scala> val conf = HBaseConfiguration.create()

scala> val tableName="spark_table"

scala> import org.apache.hadoop.hbase.mapreduce.TableInputFormat

scala> conf.set(TableInputFormat.INPUT_TABLE,tableName)

scala> import org.apache.hadoop.hbase.client.HBaseAdmin

scala> val admin = new HbaseAdmin(conf)

scala> admin.isTableAvailable(tableName)

res2: Boolean = false

satyajit_das · August 17, 2020, 7:04pm

Hi,

If you want to open hbase then type hbase shell in your web-console.
If you want to open the spark then type spark-shell in your web-console.

It does not matter how you open.
What to do that matter.

All the best!

mearupmukherjee · August 17, 2020, 7:38pm

Hi

Actually I want to work Spark with Hbase that’s why I am trying to call those jars while opening spark shell.

If you know how to work Hbase from spark plz do let me know.I don’t see any option here in cloudex lab .You have spark and you have Hbase but if someone want to connect hbase table from spark then how to do that.

Thanks

mearupmukherjee · August 17, 2020, 8:43pm

Hi

Could you plz try with copying hbase jar file into spark jar directory.This will be helpful for other resources also.

Thanks

satyajit_das · August 18, 2020, 2:40am

The files “…/hbase-0.98.9-hadoop2/bin/start-hbase.sh” and “HBASE_PATH=` …/hbase-0.98.9-hadoop2/bin/hbase classpath” are not present in my web-console so cannot start.
the classpath need to be mentioned.

also getting an error " error: object hbase is not a member of package org.apache.hadoop".

mearupmukherjee · August 18, 2020, 6:18am

Hi
There are 2 way we can connect Hbase from spark.

1.Could you please copy Hbase jar files to Spark Jar file location
Or
2. Put Hbase jar files in separate folder and give path like jar= while opening Spark in shell
Please check Jar option to call external jars while invoking spark.

This is an good facility where people can work with hbase and spark togeather.
Please give a try ,I don’t have access all these folder so I can not.
I am asking this thing almost one month but still not got any solution

Thanks