Question

I will preface the question by saying that I am somewhat new to the .NET world, and might be missing something entirely obvious. If so, I’d love to hear what that is!

I often find myself writing small program that all do more or less the same thing:

1. Read in data from one or more files
2. Store this data in memory, in some sort of container
3. Crunch data, output analysis results to a text file and quit

I often find myself creating monstrous-looking containers to store said data. E.g.:

Dictionary<DateTime, SortedDictionary<ItemType, List<int>>> allItemTypesAndPropertiesByDate =
            new Dictionary<DateTime, SortedDictionary<ItemType, List<int>>>();

This works, in the sense that the data structure describes my intent more or less accurately – I want to be able to access item types and properties by date. Nevertheless, I feel that the storage container is too tightly bound to the output data format (if tomorrow I decide that I’d like to find all dates on which items with certain properties were seen, this data structure becomes a liability). Generally, making input and output changes down the line is time-consuming and error-prone. Plus, I have to keep staring at these ugly-looking declarations – and code to iterate over them is not pretty either.

On the other end of the complexity spectrum, I can create a SQL database with schema that describes input in a more flexible format, and then run queries (using SQL or LINQ to SQL) against the database. This certainly works, but feels like too big of a hammer – I write many programs like these, and don’t want to create a database for each one, manage the SQL dependency (even if it is SQL express on local machine), etc. I don’t need to actually persist the data – just to read it in, keep it in memory, make a few queries and quit. Even using an in-memory SQLite instance feels like an overkill. I am not overly concerned with runtime performance – these are usually just little local machine experiments – but it just feels wrong.

Ideally, what I would like is to have a low-overhead, in memory row store with a loosely-defined schema that is easily LINQ-queryable, and takes only a few lines of code to set up and use. Does the Microsoft .NET 4 stack include something like this? If you found yourself in a similar predicament, what would you do?

Your thoughts are appreciated – thanks!

Alex

Answer 1

If you find a database structure easier to work with, one option might be to create a DataSet with DataTables representing your schema which you can then query using Linq 2 DataSets