I need to compare particular content of 2 SQL tables located in different servers: Table1 and Table2.
I want to compare each row from Table1 against the whole content of Table2.
Comparison logic is kind of complicated so I want to apply a logical operator that I will wrinte in C#. So I don’t want to do the comparison on the SQL query itself.
My concern is the size of the data I will work on will be around 200 MB.
I was thinking to load the data into a DataTable by using ADO.Net and do the comparison on the memory.
What would you recommend? Is there already a pattern like approach to compare massive data?
200 MB should not be a problem. A .NET application can handle much more than that at once.
But even so, I would probably use a forward-only data reader for Table 1, just because there’s no good reason not to, and that should reduce the amount of memory required. You can keep table 2 in memory with whatever structure you are accustomed to.