Question

I have a trouble while making header from Datatable’s values. I have a text file, which I search for some strings. split them and put them in Datatable. After that I use some linq codes to make column “Name” as header of this table. For small text file it works great but when i got little bigger files, my compiler throws MemoryOutOfExeption. So I’m asking how can I fix my code?

My Code:

        DataTable evHeader00 = resAtt.Copy();

        var prds1 = evHeader00.AsEnumerable() // make Name Values as header
                        .GroupBy(c => c["Name"])
                        .Where(g => !(g.Key is DBNull))
                        .Select(g => (string)g.Key)
                        .ToList();

        prds1.ForEach(p => evHeader00.Columns.Add(p, typeof(string))); // Here i got MemoryOutOfExeption

        foreach (var row in evHeader00.AsEnumerable())
        {
            if (!(row["Name"] is DBNull))
            {
                row[(string)row["Name"]] = row["Products"];
            }
        }

DataTable 1:

 ID    Name    Products
 1      a        A
 2      b        B
 3      c        C
...    ...      ...

Result:

ID     a   b   c   ...
1      A
2          B
3              C
...   ... ... ...  ...    

Answer 1

1. Why do you need to copy the whole DataTable here?

   DataTable evHeader00 = resAtt.Copy();
   

2. Why do you create a new List<String> here with all your data?

   var prds1 = evHeader00.AsEnumerable() // make Name Values as header
                   .GroupBy(c => c["Name"])
                   .Where(g => !(g.Key is DBNull))
                   .Select(g => (string)g.Key)
                   .ToList();
   

So this should be more scalable:

IEnumerable<string> data = resAtt.AsEnumerable()
                                 .Where(r => !r.Field<string>("Name") == null)
                                 .GroupBy(r => r.Field<string>("Name"))
                                 .Select(g => g.Key);

or (as @Rawling has commented) by using Enumerable.Distinct which should be even more “memory-friendly”:

IEnumerable<string> data = resAtt.AsEnumerable()
                                 .Where(r => !r.Field<string>("Name") == null)
                                 .Select(r => r.Field<string>("Name"))
                                 .Distinct();

Now you can use a foreach to enumerate the result.