Question

What is the best direction to go if we are logging every user action in a web application that could potentially have 20K users on at the same time? Should we build a list cache of the log data, and persist it to the database after a certain amount of time or threshold? Are there any good libraries or techniques that can do this?

Answer 1

Should we build a list cache of the log data, and persist it to the database after a certain amount of time or threshold

If you cache and you crash, goodbye logs.

Are there any good libraries or techniques that can do this?

It is best to use a flexible logging mechanism that you can turn on and filter at will, that has ultra high performance when disabled.

I recommend using an existing library. You don’t want bugs in your logging to mask bugs in production.

NLog (and other such logging libraries) can be used to do this fairly easily. It is designed to suit the needs of a large range of projects, and has techniques that allow you to have a very low performance impact when you have disabled it. It also has built-in multi-threaded/asynchronous logging support.

That being said, there are a lot of reasonable quality .Net logging libraries out there. Check out this question for some quality comparison opinions (more libraries are represented than the title suggests): log4net vs. Nlog