Question

I’m writing a simple application in vb.net and need to output some information to a log file for diagnostic purposes.

In order to ensure that the log file doesn’t get too big, I need to trim it to a certain number of lines (say 5000). For performance reasons, the trimming doesn’t have to occur each time a log entry is written out nor is it critical that trimming occurs (for example, if the application crashes prematurely).

However I’m not really sure the best way to trim it. Loading the entire log into memory and then re-writing only the last 5000 lines back out would work, but I’m sure there is a cleaner and better way to do this.

Can anyone please suggest a more efficient method? I’m fairly new at vb.net so code examples would be really appreciated.

Answer 1

Rather than worrying about modifying data I’ve already written, I would just follow the time-honored tradition of starting a new log file at a certain point, either when the current one reaches a certain size, or on a schedule basis (hourly, daily or otherwise).

Just rename the current logfile from xyzzy.log to xxyyz-YYYYMMDDHHMMSS.log then start a brand new copy of xyzzy.log.

This also gives you a decent trade-off between maintaining a full audit of logs and preserving disk space. You can set up procedures to delete log files more than 30 days old, or keep the aggregate size below some threshold, or even hive off older logs to a secondary storage area.

These procedures don’t even have to be part of your program – I have, in the past, simply had a scheduled task which cleans up old log files, a task that shared nothing with the program proper, other than the directory where the script resided.

This is probably the easiest way to do it short of integrating a third-party product (which isn’t always possible) and, since log files are almost invariably for the programmer rather than the user, the naming standards shouldn’t be too onerous.

---

A very simple solution I’ve used in the past (admittedly with Linux) is to actually not worry about renaming at all. Simply have the log function open the file using the current date, and remember what the date was. Every time you call the log function after that, check if the date has changed and, if so, open a new log file for subsequent logging. That gives you one log file per day with very little effort by the program.

Then I just had scripts using find -mtime to delete those over a certain age.