Advanced Search

Find duplicate file and remove

/ Published in: Bash
Save to your folder(s)
Perl 1 liner + additional linux command to find duplicate files and remove it - only spare 1 file.

Expand | Embed | Plain Text

Copy this code and paste it in your HTML 
  1. #!/bin/sh
  2.  
  3. find "$@" -type f -print0 | xargs -0 md5sum | sort | perl -ne
  4.  
  5. 'chomp;$x=$y;($y,$z)=split(/\s+/,$_,2); print "$z"."\x00" if ($x eq $y)' | xargs -0 rm -v --
  6.  
  7. #OR
  8. #if wanna find out size of all files to calculate space storage 
  9. #change the 
  10. # xargs -0 stat -c %s | awk'{SUM+=$1}END{print SUM/1024/1024}'

Report this snippet


Comments

RSS Icon Subscribe to comments

You need to login to post a comment.