Question

I am writing a simple utility to take application logs and display them to the user. It neeeds to take log entries from multiple files and merge them into a single internal storage. The merging is based on the log entry time and so it is possible for multiple entries to have the same time. Note that the size of the logs is fairly small and so the maximum size is likely to be 10 files containing 2,000 log entries each.

I also want to present this to the user but allow them to filter based on different logging levels (debug, warning, error etc…) and also filter based on the file it came from. This sounds like a very common type of problem to solve and so others must have experience about the best approach. I had thought of the following…

1, Use a DataSet to store the log entries and then use a DataView to apply filtering, sorting.

2, Use a couple of List<> instances, a master and then a copy that is filtered to the required entries and apply a custom IComparer function to the filtered list to get the correct ordering.

Thoughts?

Answer 1

I assume that the data is small enough to fit in memory, and you’d rather not have the overhead of a database and an external file.

Either of the two should work well. I’m more comfortable with List<T>, so would prefer that, but can understand the attraction of using DataTable, etc.

Another option would be to use LINQ on whatever internal data storage you choose. That might simplify the queries.

I suspect that a List<T> or whatever collection you choose will use less memory than a DataTable. However, unless your log entries are huge, I don’t expect that memory usage will be a problem.