Question

I’m writing a data access layer where I want to return just the data required for each request. Let’s say there is a total of 100 properties that you could retrieve, but a normal request will probably just need two or three of them.

So I’ve added a parameter to the request that specifies the properties you want to return and only this data will be pulled from the DB.

A normal request will return a list of 1,000 – 10,000 rows. My big concern now is how to return them efficiently.

The data will be used “internally” in my .NET-project but may also be serialized to XML and JSON, and therefore I don’t want to have 98 empty properties when in fact there is just 2 of them that are populated with data.

So, what would be the best technique (in case of performance and “overhead size”) to use for creating this model?

Update:
So a property could actually have three states. If it never was asked for, it should somehow be removed, and if it was asked for it should be either null or the actual value.

Answer 1

While I agree that Andrei’s answer is convenient, it can definitely have a performance impact. If your data is flat (and it is, since you are ultimately pulling from a database), then you can have an IEnumerable<IDictionary<string, object>> where each row in the IEnumerable<T> is a IDictionary<string, object> contains the values from the row.

It’s a more unwieldy to begin with, but it is dynamic; the addition or removal of attributes from the result set will still work.

It’s also more performant, in that you won’t have (1,000 to 10,000) * N calls to reflection (1,000 to 10,000) rows, times N, the number of attributes on each row (if you have 100 attributes per row, then that’s 100,000 to 1,000,000 calls to Reflection, which will add up). The DynamicDictionary ultimately stores the key/value pairs in a lookup table, but it uses Reflection (optimized through the DLR, but that’s still at it’s core) to get to the lookup table.

And you do know what the properties are (you said you are using it internally for your project, so you have to know what the properties you want to use are). To that end, you really should be using a strongly-typed data transfer object with properties that are all nullable that represent the values from the database; anything that is null or doesn’t exist in the result set gets set to null. Values that don’t exist in the result set never get set.

This way, you have no Reflection, you get compile-time type checking (which is very important, using DynamicObject will result in run-time exceptions), and better performance.

Serialization in XML or JSON is simple with either of these approaches, you simply serialize null values, or don’t serialize the property at all (if using XML, just make sure that your schema supports the elements as being options).