Question

I need a data structure of some sort to do the following:

One “set” composed of many types such as string, integer, datetime and double.
Many sets are added dynamically
The sets are retrieved dynamically where information is pulled

Now the obvious solution is to use a DataTable. Define the datatable structure, and add a new row each time you need to add a new set. Pull data from the datatable when you need to.

Actually I have implemented it already using a datatable, but the problem is it is extremely slow for some reason. Since this is done thousands to millions of times performance can be problematic.

Is there an alternative datatable type of data structure with better performance that I can use or should I build my own class using Lists<> ?

Answer 1

Depending on your use case I would recommend using List<object[]> (since you mentioned dynamic schema) as central data structure, but you will need to maintain the schema info yourself if you need it later on.

If you need to bind the UI to the data this approach will add a lot of extra manual work, it’s better suited for background processing of large amounts of data.

We have used this approach in the past and were able to save 2/3 of memory and 80% of execution time when bulk handling data compared to data tables.