Question

I’m maintaining some website code that will soon dump all its errors and warnings into a log file. In order to make this a bit more pro-active I plan to parse this log file daily, summarize the warnings and errors (i.e. count the occurrence of each specific one and group by either warning/error) and then email this to the devs on the project.

This would likely admittedly be rather trivial with a hash and some further fiddling, I wondered if there is a suitable module on CPAN that I could use to do this task.

It would either be one that summarizes specifically Perl error/warnings logs or one that summarizes arbitrary text files. Any suggestions?

Edit:

The site I’m maintaining was inherited in a state where it generates 50MB of warnings per day. I’m ONLY looking for a bandaid i can apply to point out the most prolific ones. Log4Perl can come in once I run out of critical stuff to fix, however right now it’s not an option.

Answer 1

I think looking on CPAN for something as specific and simple as this might be overkill. Assuming the log file in in the default apache error format:

[Mon Apr 26 15:39:34 2010] [error] [client 69.12.220.202] syntax error at /var/www/cgi-bin/errortest.cgi line 8, near "{}"

Here’s a quick one liner to mail the errors sorted by highest occurrence to an address. Easily changed to mail multiple addresses (or make an alias that goes to multiple addresses and send to that.

cat LOGFILE |
perl -ple 's/\[\w+\s\w+\s\d+\s\S+\s\d+\]\s\[[^\]]+\]\s\[[^\]]+\]\s//;' |
sort |     # Sort errors after
uniq -c |  # Uniqify with count
sort -rn | # Sort line counts
mail -s "Error list" EMAIL@ADDRESS

You can easily exclude lines by throwing in a grep -v at whatever point you find most appropriate. Throw it into cron for daily reports, or put it into a script and add to logrotate.