Expunge , deleting files from hadoop trash

Technical Discussions Hadoop
#1

Hi,

  1. Does .Trash directory exist for all users individually or is it one .Trash for all users?

  2. How to delete my deleted files from the trash directory ? Below command did not work -
    [amit3tyagi2316@cxln4 ~]$ hadoop fs -expunge -fs .Trash/210324050639/user/amit3tyagi2316

  3. [amit3tyagi2316@cxln4 ~]$ hadoop fs -expunge
    Above command gives below error -
    Access denied for user amit3tyagi2316. Superuser privilege is required

Thanks and Regards,
Amit

#2

It is better explained here: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#expunge

#3

Thanks Sandeep !!!

I had come here after reading that link also searched internet in vain.
Below is also not working -

[amit3tyagi2316@cxln4 ~]$ hadoop fs -expunge -immediate
-expunge: Illegal option -immediate

#4

Hi Amit,

Yes, you are right. The expunge command is to be run only by the superuser/root.

But if you delete “.Trash” folder it is equivalent to removing the file completely. Here is a short experiment I just did at CloudxLab

# let's check the storage consumed by my home folder
$ hadoop fs -du -s -h /user/sandeep
222.7 M  /user/sandeep

# Now, lets delete some files
$ hadoop fs -rm -r /user/sandeep/word*

# If you see the storage consumed by my home folder is still same even after deletion
$ hadoop fs -du -s -h /user/sandeep
222.7 M  /user/sandeep

# Now, lets delete .Trash folder
$ hadoop fs -rm -r /user/sandeep/.Trash/
Deleted /user/sandeep/.Trash

# Now, the space consumed by my home folder has dropped
$ hadoop fs -du -s -h /user/sandeep
201.5 M  /user/sandeep
1 Like
#5

Thanks Sandeep for quick reply !!!

I just find out that if we delete the .Trash directory, it is automatically created whenever any file is deleted. Experiment screenshot is attached.

DeletingTrash
DeletingTrash1122Ă—520 144 KB

1 Like
#6

Yes, that’s right. So, no worries in deleting it.