Question

I have a once-a-day ingestion case in which I will be getting a large file via FTP which contains the up-to-date versions of 4 database tables.

For each table, I would like to:

1. Truncate table in staging database
2. BCP the FTP’d file into that table
3. Find diffs (IUD) between staging table and production table
4. Make any required IUDs to production table so it matches staging table

I’m sure this is a reasonably common problem, but I’m not 100% sure as to the best way to approach it.

Are there any built in T-SQL features for this kind of problem, or do I just need to do various joins to find the inserted/updated/deleted records and execute them manually? I’m sure I can manage to do it this second way, but any suggestions are greatly appreciated none-the-less (not looking for working code).

Answer 1

Since nobody ever put it as a real answer, the MERGE command as mentioned by Mikael Eriksson in the comment is the right way to go, it worked great.

Here’s a simple example usage:

MERGE dbo.DimProduct AS Target
USING (SELECT ProductID, ProductName, ProductColor, ProductCategory FROM dbo.StagingProduct) AS Source
ON (Target.ProductID = Source.ProductID)
WHEN MATCHED THEN
    UPDATE SET Target.ProductName = Source.ProductName
WHEN NOT MATCHED BY TARGET THEN
    INSERT (ProductID, ProductName, ProductColor, ProductCategory)
    VALUES (Source.ProductID, Source.ProductName, Source.ProductColor, Source.ProductCategory)
OUTPUT $action, Inserted.*, Deleted.*;

from: http://www.bidn.com/blogs/bretupdegraff/bidn-blog/239/using-the-new-tsql-merge-statement-with-sql-server-2008

which helped me.