Question

Ok so here it goes. The requirement I am facing is that I have a table in the database say “MyDbTable” which has a lot of data in it.

Lets say it has three columns like:

ColA    ColB    ColC

1           a        ab

2           b        bc
3           c        cd

ColA being the identity

Now I call a webservice and expected result consists of around 1500 rows.

the schema is same of the returned result say:

ColA    ColB    ColC

1           a        xy

3           c        yz
4           c        yz

Now what I really want to do is check for the existing records I have in the service results and update them in MyDbTable, in this case, its gonna be the records with 1 and 3 in ColA. I will have to update them. For the record with ColA value 4 in the webservice result is new so I would have to insert it.

Now the problem is that the MyDbTable have thousands of rows and the service also returns a number of rows.

I know the simplest way to do it is the brute force where we iterate over each record, check it and then handle it, either through application or a stored procedure.

What I really wanna know is how to do this in the most optimized way.

Answer 1

This is what I’d assume to be the most efficient way (haven’t measured it, though):

1. Copy the whole table returned from the web service into a temporary table of your database, using a bulk copy operation such as offered by the SqlBulkCopy Class.

2. Use a single SQL Server MERGE statement to make an “update-or-insert” operation from the temporary table into your real table.

If you are using SQL Server version older than 2008, you can use an UPDATE statement with an OUTPUT clause instead of a MERGE.