We’ve currently got four web servers in a farm generating IIS web logs about 100Mb per day. These can be compressed pretty effieciently down to somewhere around 5% of their size.
We are planning to use waRmZip to move them off the servers and onto a SAN. After a week or so we can be confident we don’t have any technical issues to investigate so the only other thing would be using them for trend analysis as a compliment to Google Analytics.
What retention periods do people recommend? Are there any legal requirements to keep this data?
Legal requirements will depend on your country, how much you’re logging, and quite possibly the nature of your business. Talk to your company’s lawyers – legal advice on SO is likely to be worth what you pay for it.
If you’re only storing 5MB per day, you should be able to store them for basically as long as you want without worrying on the technical front.