Question

I have a Delete Query that each day needs to run deleting any data that is greater than 7 days old which is about 6 Million Records.

My table should only store data for the last 7 days.

Here is the query I am running:

DELETE FROM [tblTSS_DataCollection]
Where [DatapointDate] < DATEADD(D, -7, GETDATE()) 

This query takes 5.5 minutes to execute. I have an index setup that includes this so I don’t think it should be taking this long to execute:

CREATE UNIQUE NONCLUSTERED INDEX [IX_tblTSS_DataCollection] ON [dbo].  [tblTSS_DataCollection] 
(
[DataPointID] ASC,
[DatapointDate] ASC,
[AssetID] ASC
)

Is there a better way to delete this data? It takes forever and I really need to be able to delete this data quickly.

Answer 1

Your index doesn’t really support the query since the query doesn’t reference the leading key column. So, it has to scan the entire table with or without this index. You might consider an index on DataPointDate alone to support this delete operation if it’s something you run often.

If DataPointID is an IDENTITY column, and DataPointDate is entered sequentially, you might also consider this variation:

DECLARE @maxID INT;

SELECT @maxID = MAX(DataPointID) 
  FROM dbo.tblTSS_DataCollection
  WHERE [DatapointDate] < DATEADD(D, -7, GETDATE());

DELETE dbo.tblTSS_DataCollection
  WHERE DataPointID <= @maxID;

Another thing you might consider doing (if it’s the delete and not the scan contributing to the slowness) is (a) making sure your log has enough room to accommodate the delete, and isn’t killing you with a bunch of autogrows, and (b) performing the work in batches:

BEGIN TRANSACTION;

SELECT 1;

WHILE @@ROWCOUNT > 0
BEGIN
  COMMIT TRANSACTION;

  DELETE TOP (1000) dbo.tblTSS_DataCollection WHERE ...
END

COMMIT TRANSACTION;