Let's say that your UNIX server is configured to rotate your email logs nightly. After the log rotation you want to setup a script that will give a quick summary of the top email recipients on your server.
Here is some example output from the log:
[root@server1 root]# tail -n 40 /var/log/maillog
Nov 17 09:40:28 server1 postfix/qmgr[31224]: E602E44C002: removed
Nov 17 09:40:35 server1 postfix/pickup[6300]: DDD6C44C002: uid=0 from=
Nov 17 09:40:35 server1 postfix/cleanup[6045]: DDD6C44C002: message-id=<20061117144035.ddd6c44c002@server1>
Nov 17 09:40:35 server1 postfix/qmgr[31224]: DDD6C44C002: from=
Nov 17 09:40:36 server1 postfix/smtp[6060]: DDD6C44C002: to=johnsmith@webmail.us relay=192.168.127.1[192.168.127.1], delay=1, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as E091B44C17E)
...
The only line I'm interested in is the last line, so I can easily 'grep' for that data:
[root@server1 root]# grep "postfix/smtp" /var/log/maillog
Nov 17 09:22:33 server1 postfix/smtp[4077]: A63D444C002: to=johnsmith@webmail.us relay=192.168.127.1[192.168.127.1], delay=0, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as ABB8E44C188)
Nov 17 09:22:58 server1 postfix/smtp[4077]: AAD5244C002: to=jsmith@gmail.com relay=192.168.127.1[192.168.127.1], delay=0, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as ADF86C1DC)
...
That's great, but what I really need is only the email address part of that line, that's where 'awk' is very useful:
[root@server1 root]# grep "postfix/smtp" /var/log/maillog | awk '/.*/ {print $7}'
to=johnsmith@webmail.us
to=jsmith@gmail.com
to=johnsmith@webmail.us
to=vvv@webmail.us
...
Almost there, but I only need the part after the 'to=', here comes 'sed' to the rescue:
[root@server1 root]# grep "postfix/smtp" /var/log/maillog | awk '/.*/ {print $7}' | sed {s/to=//}
johnsmith@webmail.us
jsmith@gmail.com
johnsmith@webmail.us
...
Now, the tricky part, I want to group all of the duplicate lines together and get a count of the number of 'johnsmith@webmail.us' entries, luckily I stumbled upon this great awk script which helps me do this:
# histsort.awk -- compact a shell history file
# Arnold Robbins, arnold@gnu.ai.mit.edu, Public Domain
# May 1993
# Thanks to Byron Rakitzis for the general idea
{
if (data[$0]++ == 0)
lines[++count] = $0
}
END {
for (i = 1; i <= count; i++)
print data[lines[i]], lines[i]
}
[root@server1 root]# grep "postfix/smtp" /var/log/maillog | awk '/.*/ {print $7}' | sed {s/to=//} | awk -f history.awk
17 johnsmith@webmail.us
1 jsmith@gmail.com
16 vvv@webmail.us
Now it's easy to sort the list, numerically, in reverse order:
[root@server1 root]# grep "postfix/smtp" /var/log/maillog | awk '/.*/ {print $7}' | sed {s/to=//} | awk -f history.awk | sort -r -n
17 johnsmith@webmail.us
16 vvv@webmail.us
1 jsmith@gmail.com
And finally, let's email the results to myself:
[root@server1 root]# grep "postfix/smtp" /var/log/maillog | awk '/.*/ {print $7}' | sed {s/to=//} | awk -f history.awk | sort -r -n | mail -s "Daily Summary" jsmith@gmail.com
Oh, I almost forgot, I need to gunzip the nightly version an put it in cron:
[root@server1 root]# gunzip -c /var/log/maillog.1.gz | grep "postfix/smtp" | awk '/.*/ {print $7}' | sed {s/to=//} | awk -f history.awk | sort -r -n | mail -s "Daily Summary" jsmith@gmail.com
Combining all of these really simple tools allows you to create just about any type of report imaginable.
