DiskSpace quota - ERROR hdfs.DFSClient: Failed to close inode

Lab Support
#1

Hi,
I am executing below mapreduce command

hadoop jar /home/essujit8176/mrlog.jar DriverLogAnalysis /user/essujit8176/console /user/essujit8176/logoutput ERROR

Got error
18/09/16 12:27:22 ERROR hdfs.DFSClient: Failed to close inode 1651645
org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /user/essujit8176 is exceeded: quota = 4294967296 B = 4 GB but diskspace consumed = 5071125108 B = 4.72 GB

HDFS folder size I am reading is 1.2 GB, is there any limit to memory/disk size.

I verified hdfs trash is empty.

Note: I was able to execute same file with Spark submit job.

Should I resize my input folder to lesser size in HDFS input? Could you help me resolve?

#2

Delete large HDFS file and was able to run.

My bad for uploading large file.

1 Like
#3

Hi @Sujith_Surendran,

Please see our fair usage policy here. You can not upload more than 4.5 GB data in HDFS with the replication factor of 3.

If you want you can decrease the replication factor to 1 to utilize all the 4.5 GB space