Question

I have two tables, X and Y, with identical schema but different records. Given a record from X, I need a query to find the closest matching record in Y that contains NULL values for non-matching columns. Identity columns should be excluded from the comparison. For example, if my record looked like this:

------------------------
id | col1 | col2 | col3
------------------------
0  |'abc' |'def' | 'ghi'

And table Y looked like this:

------------------------
id | col1 | col2 | col3
------------------------
6  |'abc' |'def' | 'zzz'
8  | NULL |'def' | NULL

Then the closest match would be record 8, since where the columns don’t match, there are NULL values. 6 WOULD have been the closest match, but the ‘zzz’ disqualified it.

What’s unique about this problem is that the schema of the tables is unknown besides the id column and the data types. There could be 4 columns, or there could be 7 columns. We just don’t know – it’s dynamic. All we know is that there is going to be an ‘id’ column and that the columns will be strings, either varchar or nvarchar.

What is the best query in this case to pick the closest matching record out of Y, given a record from X? I’m actually writing a function. The input is an integer (the id of a record in X) and the output is an integer (the id of a record in Y, or NULL). I’m an SQL novice, so a brief explanation of what’s happening in your solution would help me greatly.

Answer 1

There could be 4 columns, or there could be 7 columns…. I’m actually writing a function.

This is an impossible task. Because functions are deterministic, so you cannot have a function that will work on an arbitrary table structure, using dynamic SQL. A stored procedure, sure, but not a function.

However, the below shows you a way using FOR XML and some decomposing of the XML to unpivot rows into column names and values which can then be compared. The technique used here and the queries can be incorporated into a stored procedure.

MS SQL Server 2008 Schema Setup:

-- this is the data table to match against
create table t1 (
    id int,
    col1 varchar(10),
    col2 varchar(20),
    col3 nvarchar(40));
insert t1
select 6, 'abc', 'def', 'zzz' union all
select 8, null , 'def', null;

-- this is the data with the row you want to match
create table t2 (
    id int,
    col1 varchar(10),
    col2 varchar(20),
    col3 nvarchar(40));
insert t2
select 0, 'abc', 'def', 'ghi';
GO

Query 1:

;with unpivoted1 as (
    select n.n.value('local-name(.)','nvarchar(max)') colname,
           n.n.value('.','nvarchar(max)') value
    from (select (select * from t2 where id=0 for xml path(''), type)) x(xml)
    cross apply x.xml.nodes('//*[local-name()!="id"]') n(n)
), unpivoted2 as (
    select x.id,
           n.n.value('local-name(.)','nvarchar(max)') colname,
           n.n.value('.','nvarchar(max)') value
    from (select id,(select * from t1 where id=outr.id for xml path(''), type) from t1 outr) x(id,xml)
    cross apply x.xml.nodes('//*[local-name()!="id"]') n(n)
)
select TOP(1) WITH TIES
       B.id,
       sum(case when A.value=B.value then 1 else 0 end) matches
from unpivoted1 A
join unpivoted2 B on A.colname = B.colname
group by B.id
having max(case when A.value <> B.value then 1 end) is null
ORDER BY matches;

Results:

| ID | MATCHES |
----------------
|  8 |       1 |