Awk Samples

Count rows and compute sum for a column

grep "Summary" *.log|awk '{sum += $7} {count +=1} END {print count, sum}'

This counts each line containing the word "Summary" in all files ending in .log
Value in column 7 is summed.

Sample data:
                                        *               
Summary - Thread 145    20      16      4       16      4
Summary - Thread 148    20      15      5       15      5
Summary - Thread 147    20      15      5       15      5

* this is column 7

Conditional Sum

awk '$5 == 2 {{s += $8} {c += 1}} END {print 2, s/c }' timing.txt >> sum.txt

What this does:

* read file 'timing.txt'
* if 5th column has value 2:
- add value of column 8 to s
- add 1 to c (count occurrences)
* after last record read:
- print literal 2
- print average (sum/count)
* output: append to file 'sum.txt'

Count with sort and uniq

Question: Find all unique IP address that appear in a certain log message, and count how many times they appear.


Answer:

grep "log message" logfile.log | awk '{ print $18 }' | sort | uniq -c

Example:

[cygwin] $ grep "Missing post body" mpi.log |awk '{print $18}' |sort |uniq -c |sort -rn
      6 82.1.184.81
      6 212.121.212.149
      4 62.49.164.229
      2 62.140.196.160
 [cygwin] $

Note: the "sort -rn" at the end sorts results by number of occurrence

Unix find

# find all files containing "string", starting in current directory, and print their path
find . -exec grep “string” {} /dev/null \;

Note: /dev/null shows name of the file before text that is found

For more details see http://en.wikipedia.org/wiki/Find and man find