Question

I receive a CSV file weekly that I import into my SQL database using BULK INSERT. I insert into a temporary table and then merge that with the main table by inserting new records and updating any records that have been modified. The code is:

BULK INSERT dbo.temp
FROM 'C:\Users\Administrator\Documents\20120125.csv'
WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '\n' );

MERGE dbo.main AS TargetTable                            
USING dbo.temp AS SourceTable                    
ON (TargetTable.name = SourceTable.name)                
WHEN NOT MATCHED BY TARGET                              
    THEN INSERT (name, age, gender, father, mother, teacher, mathsrating, historyrating, sciencerating, englishrating)
        VALUES(SourceTable.name, SourceTable.age, SourceTable.gender, SourceTable.father, SourceTable.mother, SourceTable.teacher, SourceTable.mathsrating, SourceTable.historyrating, SourceTable.sciencerating, SourceTable.englishrating)
WHEN MATCHED                                            
    THEN UPDATE SET
        TargetTable.name = SourceTable.name,
        TargetTable.age = SourceTable.age,
        TargetTable.gender = SourceTable.gender,
        TargetTable.father = SourceTable.father,
        TargetTable.mother = SourceTable.mother,
        TargetTable.teacher = SourceTable.teacher,
        TargetTable.mathsrating = SourceTable.mathsrating,
        TargetTable.historyrating = SourceTable.historyrating,
        TargetTable.sciencerating = SourceTable.sciencerating,
        TargetTable.englishrating = SourceTable.englishrating;

DELETE FROM dbo.temp

What I want to achieve is to have the records that are overwritten by the update stored in a new table with their ‘previous’ values so that I have a history of what’s changed. I’m fairly new to SQL but having researched a little it seems a Trigger may be the approach to take but would welcome any suggestions on how to approach this.

Thanks

Matt

Answer 1

The merge statement has an output clause that will output affected rows either to a table, or as a result of the query.

You can also specify extra criteria in the When Matched part. In this case use it to ensure that rows don’t get updated if all the values are the same as the existing row. If the columns are nullable this is slight tricky because (1 != Null) returns Null. For your ratings, which I’m assuming can’t be negative, you can do IsNull(s.rating, -1) != IsNull(t.rating, -1) to see if it has changed

As the output clause can be the result of the query, you can nest it as an inner query. I’ve used this to add an UpdateTimestamp to your history table.

Example: http://sqlfiddle.com/#!6/1651a/2

Example of why null check is necessary: http://sqlfiddle.com/#!6/bd99b/2

Insert Into 
  history_table  
Select
  GetDate(), [name], age, gender, father, mother, teacher, 
  mathsrating, historyrating, sciencerating, englishrating
From (
  Merge
    dbo.main As t
  Using
    dbo.temp AS s
      On (t.[name] = s.[name])
  When Not Matched By Target Then
    Insert (
      [name], age, gender, father, mother, teacher, 
      mathsrating, historyrating, sciencerating, englishrating
    ) values (
      s.[name], s.age, s.gender, s.father, s.mother, s.teacher, 
      s.mathsrating, s.historyrating, s.sciencerating, s.englishrating
    )
  When Matched And -- assume ratings can't be negative, but can be null
    t.age != s.age Or
    t.gender != s.gender Or
    t.father != s.father Or
    t.mother != s.mother Or
    t.teacher != s.teacher Or
    IsNull(t.mathsrating, -1) != IsNull(s.mathsrating, -1) Or
    IsNull(t.historyrating, -1) != IsNull(s.historyrating, -1) Or
    IsNull(t.sciencerating, -1) != IsNull(s.sciencerating, -1) Or
    IsNull(t.englishrating, -1) != IsNull(s.englishrating, -1)
  Then Update
    Set
      t.[name] = s.[name],
      t.age = s.age,
      t.gender = s.gender,
      t.father = s.father,
      t.mother = s.mother,
      t.teacher = s.teacher,
      t.mathsrating = s.mathsrating,
      t.historyrating = s.historyrating,
      t.sciencerating = s.sciencerating,
      t.englishrating = s.englishrating
  Output
      $action as Action, deleted.*
  ) as Updates