Posted By

tm on 01/18/10


Tagged

Bash arrays files duplicate find associative


Versions (?)

Who likes this?

1 person have marked this snippet as a favorite

jarnaldich


Find duplicate files recursively (bash4, assoc. arrays)


 / Published in: Bash
 

  1. # bash4 - Use Associative Arrays to find duplicate files.
  2. # Generate a file "rmdups" which contains rm commands to remove the duplicates.
  3. # Check this file and then source it from the current (!) directory
  4. # to actually remove the duplicates. Works equally with the (deprecated)
  5. # md5sum program instead of sha*sum.
  6. unset flist; declare -A flist
  7. while read -r sum fname; do
  8. if [[ ${flist[$sum]} ]]; then
  9. printf 'rm -- "%s" # Same as >%s<\n' "$fname" "${flist[$sum]}"
  10. else
  11. flist[$sum]="$fname"
  12. fi
  13. done < <(find . -type f -exec sha256sum {} +) >rmdups

Report this snippet  

You need to login to post a comment.