Question

I have two tables and I’m looking for the rows in one table where a time column is not near any of the values in another table’s time column. (Near is defined as within a minute).

Here’s a code sample:

create table temp1
(
    id int identity primary key,
    value datetime not null 
)
GO

create index ix_temp1 on temp1(value, id);
GO

set nocount on
insert temp1 (value) values (DATEADD(second, rand() * 1000000, '20100101'))
GO 15000

table temp2 is set up identical:

create table temp2
(
    id int identity primary key,
    value datetime not null 
)
GO

create index ix_temp2 on temp2(value, id);
GO

set nocount on
insert temp2 (value) values (DATEADD(second, rand() * 1000000, '20100101'))
GO 15000

And here’s my first crack at it (which is very inefficient)

SELECT t1.id, t1.value
FROM temp1 t1
LEFT JOIN temp2 t2
    ON t1.value between DATEADD(MINUTE, -1, t2.value) and DATEADD(MINUTE, 1, t2.value)
WHERE t2.value is null

I’m looking for ways to do this more efficiently. All solutions will be considered (new indexes, SSIS solution, CLR solutions, temp tables, cursors etc…)

Answer 1

The LEFT JOIN/IS NULL isn’t as efficient on SQL Server as NOT IN or NOT EXISTS when columns are not nullable – see this link for details.

That said, this:

SELECT t1.id,
       t1.value
  FROM temp1 t1
 WHERE NOT EXISTS(SELECT NULL
                    FROM temp2 t2
                   WHERE t2.value BETWEEN DATEADD(MINUTE, -1, t1.value)  
                                      AND DATEADD(MINUTE, 1, t1.value))

…still has a problem in that function use (IE: DATEADD) renders the index useless. You’re altering the data of the column (temporarily, without writing it back to the table) while the index is on the original value.

I’m at a loss for options if you want the precision. Otherwise, if you alter the datetime before it’s inserted into the temp table then you gain:

1. ability to straight compare: t1.value = t2.value
2. ability to use the index, assuming optimizer believes it can be of use