Sometimes I get a big list of things, and some of the things occur multiple times in the same list. To make the list easier to read, I want to delete the duplicate lines.
A good example is a list of files that have errors (maybe excerpted from an application sever's log files). In that case you have a newline-delimited list of file paths, and depending upon the situation, the same file path might be listed 4 or 5 times or more. Often, it is useful to have a list of just the files that are faulty, which can be produced by deleting all the duplicate lines. This script is for filtering just those kinds of list files.
Of course, for Emacs users there is a much easier way to remove duplicate lines:, if you have
uniq installed on your system.
M-x sort-lines RET C-x h M-x shell-command-on-region RET uniq RET
CommentsSubscribe to comments
You need to login to post a comment.