DiskSpace quota exceeded error while running Hive queries

Lab Support
#1

Hello Team,

When I’m trying to run this simple query in hive

select avg(salary) from emp_demo;,

It is giving me following error:

org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /user/pratik58892973 is exceeded: quota = 4294967296 B = 4 GB but diskspace consum
ed = 4458084547 B = 4.15 GB

While I am checking for the same on my home directory, it is showing me the following:

[pratik58892973@cxln4 ~]$cd ~; du -hs
7.3M .

Can you please let me know, why am I getting this error? though my home directory shows only 7 Mb of data.
Looking forward to your reply on an urgent basis.

#2

@sandeepgiri Can you please help here as I’m still facing the problem.

#3

Hi, Pratik.

May I know in which database is the “emp_demo” table is located?
and which exercise of Hive you are working on?
I am looking into it.

All the best!

#4

Hi Satyajit,

I am using pk581989 database.
I am just trying to run a normal aggregate function on a column.
Also while I tried working on bucketing, it showed the same error.
Thanks in advance. :slightly_smiling_face:

#5

Hi, Pratik.

Yes, I have checked it.
I am trying to find the cause for it.

You can carry on with the course, once resolved. I will let you know.

All the best!

#6

Hi pratik,

Your hdfs /user/userid is having 4gb of user quota which can we increase by admin. If you will run du -sh will show your local home directory space i.e /home for hdfs run hdfs dfs -du -sh /user/userid.
You can clean some unwanted file in your hdfs home directory and then run this query…hope this will work…!!

Regards,
Kapil Varshney
Bigdata Hadoop admin at TCS

#7

Hi Kapil,

Thanks for your reply.
Since the disk quota is 3GB per user, mine is even less than 1GB on hdfs path and less than 10 mb on local path.

[pratik58892973@cxln4 ~]$ hadoop fs -du -s -h /user/pratik58892973
919.7 M /user/pratik58892973

[pratik58892973@cxln4 ~]$ du -sh
7.4M

And I’m still facing this error. It would be really great if this gets solved asap as I’m unable to use hive and sqoop because of the same.
Thanks in advance.

#8

Kindly refer to the cause for this in this discussions.

Each file is getting replicated more no. of times and consuming the space leads to StackOverflow/Heap in main memory. ( my assumption).

#9

Hello Everyone,

I deleted the .Trash folder and now the hive query is working fine.
Thanks everyone Satyajit, Kapil for your support.