Question

I have a huge dataTable (around 500k-600k rows). I wanted to compute rows based on some specific columns.
Ex: I have 3 columns name ID, type and value. I wanted to compute ‘value’ column based on ‘Type’. I have done it using DataRow Filter – first get the unique ‘ID’, then for each ‘type’ compute value. This logic gets really complex and take longer to process. I’m not very good in LINQ, so i was wondering if i can do it better using LINQ or any other way?

DataTable:

ID       type      value  
--------------------------------
2         100         5

2         100         6

2         200         10

3         200         8

3         200         9

4         100         10

4         200         15

The output i’m looking for is:

ID     Type          Value

2       100            11

2       200            10

3       200            17

4       100            10

4       200            15

Answer 1

I think what you’re looking for is something like this. Obviously, where I’ve used <int>, you would need to replace with proper types as appropriate.

var output = from row in table.AsEnumerable()
             let id = row.Field<int>("ID")
             let type = row.Field<int>("type")
             group row by new { id, type } into grp 
             select new 
             {
                 ID = grp.Key.id,
                 Type = grp.Key.type,
                 Value = grp.Sum(r => r.Field<int>("value"))
             };

This is going to result in rather simple code, but it should not arguably be more efficient than a well written loop (and, of course, if you can offload this to the database instead, you will generally be better off). However, all things held equal, Linq code is pretty well optimized and efficient. If you have doubt about efficiency, measure. Run both your existing code (if you have it) and code from answers and see where you stand.