Question

I have a utility class that translates objects into DataTables and vice versa. We’re having problems loading large amounts of data when converting from a DataTable to a List of Objects. We use a custom attribute to determine and relate column information. Here’s the pseudocode:

For each row in the table
    For each property in an object
        For each attribute on that property
            If the attribute is our column information attribute
                Grab the data from the table and insert the value into the objects property
            End
        End
    End
End

For DataTable results that have hundreds of rows, though, this process is taking minutes… and that’s simply unacceptable in a web app.

So, my question is: Is there some easy way to translate a DataTable and a .NET (custom) data object back and forth that doesn’t require a lot of reflection (which is probably where all the overhead is in this case)?

Edit: Turns out it was another issue within the data object itself. Still, I did optimize the loader a bit with the reflection calls, so thank you all.

Answer 1

Does it have to be dynamic? If not, you can use LINQ to DataSets:

from tr in table.AsQueryable()
select new MyObject
{
   A = tr.Field<string>("A"),
   .
   .
}

Otherwise, it has to be reflection dependent. Make sure you aren’t getting the type and property info within the loop, as that can take some time. Same thing with the attributes: consider building an index of the properties that have the attribute before the loop, then use this index of property names to get the properties that you need. This way, the only reflection is the setting of values into the object.

Alternatively, you can try to offload work by using the async feature in ASP.NET, or consider using the Parallel extensions to use multi-core processing for the conversion.