Snipplr comments feed https://snipplr.com/ Sun, 05 Apr 2026 19:04:44 +0000 https://snipplr.com/view/57573/find-duplicate-files-using-sha1-hash <p>## First version: $ time for i in $(find . -type f -exec sha1 -r {} \; |tee .hashes.tmp |awk '{print $1}' |sort |uniq -c |awk '{print $1, $2}'|grep -v "^1"|sort -rn |awk '{print $2}'); do grep $i .hashes.tmp; echo; done; [...] real 0m2.336s user 0m0.600s sys 0m1.820s # New version: $ time for i in $(find . -type f -exec sha1 -r {} \+ |tee .hashes.tmp |awk '{print $1}' |sort |uniq -d); do grep $i .hashes.tmp; echo; done; [...] real 0m0.256s user 0m0.104s sys 0m0.167s</p> Thu, 04 Aug 2011 11:05:02 UTC https://snipplr.com/view/57573/find-duplicate-files-using-sha1-hash