Posted By

tm on 06/02/10

Tagged

file
Bash

Versions (?)

Last Edited at 06/02/10 06:38pm

Statistics

Viewed 875 times

Favorited by 0 user(s)

Related snippets

Count Word Frequency in a File (bash4, awk)

/ Published in: Bash

Expand | Embed | Plain Text

Copy this code and paste it in your HTML

# "word" is defined as "space delimited token" - i.e. "one" and "one." are dfferent
# words. The awk-expression is used as "Tokenizer", its results are used to build an
# associative array. Blank lines are assigned to the array-entry a[Blank Line] (which
# cannot result from the tokenizing, since it has a space)
unset a; declare -A a;
while read -r; do
    ! [[ $REPLY ]] && REPLY="Blank Line"
    ((a[\$REPLY]++))
done < <(awk -v OFS=\\n '{$1=$1} 1' ./file)
# print the results, sorted by frequency
for word in "${!a[@]}"; do
    printf '%d\t%s\n' "${a[$word]}" "$word"
done | sort -n
 
# NOTE: The "\$REPLY" (i.e. the backslash) is needed because for bash4
# otherwise a "[" or "]" would break the expansion/evaluation.
# NOTE ALSO: This is mainly sort of a "proof of concept". It is very slow and would better
# be implemented in e.g. awk completely!