Question

I have a simple temp-table defined in SQL Server 2008 R2 representing a parent-child relationship. There can be multiple levels of hierarchy (say, up to 10). I’m using a CTE to find children in my table that are at least level 3 in the descendency hierarchy – in other words have at least a parent AND a grandparent.

Here is a script to demonstrate the set up and the CTE I am using:

set nocount on
create table #linkage(entity_key bigint, parent_key bigint)
--alter table #linkage add foreign key (parent_key) references #linkage(entity_key)

insert into #linkage values(1, 1), (2, 2), (3, 3), (4, 1), (5, 4), (6, 5)
print 'all data:' select * from #linkage

print 'level 3+ descendents:' 
;with r(entity_key, parent_key, level) as
(
    select entity_key, parent_key, 1
        from #linkage
        where entity_key = parent_key
    union all
    select p.entity_key, r.parent_key, r.level + 1
        from #linkage p
        inner join r on p.parent_key = r.entity_key 
        where p.entity_key <> r.entity_key
)
select entity_key, parent_key as ultimate_parent_key
from r
where r.level > 2

The correctly outputs the following:

all data:
entity_key           parent_key
-------------------- --------------------
1                    1
2                    2
3                    3
4                    1
5                    4
6                    5

level 3+ descendents:
entity_key           ultimate_parent_key  level
-------------------- -------------------- -----------
5                    1                    3
6                    1                    4

The problem is that I need this to work with large data sets. When I run this against 12 million rows, it takes over 3 minutes to complete, which I’m hoping to significantly reduce.

I have tried creating various combinations of clustered and non-clustered indexes (entity_key), (entity_key, parent_key), etc. but nothing seems to help (indeed, some seem to slow it down).

Here is the execution plan against the 12 million rows with no indexes:

  |--Filter(WHERE:([Recr1014]>(2)))
       |--Index Spool(WITH STACK)
            |--Concatenation
                 |--Compute Scalar(DEFINE:([Expr1015]=(0)))
                 |    |--Compute Scalar(DEFINE:([Expr1004]=(1)))
                 |         |--Table Scan(OBJECT:([tempdb].[dbo].[#linkage]), WHERE:([tempdb].[dbo].[#linkage].[entity_key]=[tempdb].[dbo].[#linkage].[parent_key]))
                 |--Assert(WHERE:(CASE WHEN [Expr1017]>(100) THEN (0) ELSE NULL END))
                      |--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1017], [Recr1008], [Recr1009], [Recr1010]))
                           |--Compute Scalar(DEFINE:([Expr1017]=[Expr1016]+(1)))
                           |    |--Table Spool(WITH STACK)
                           |--Compute Scalar(DEFINE:([Expr1011]=[Recr1010]+(1)))
                                |--Filter(WHERE:([tempdb].[dbo].[#linkage].[entity_key] as [p].[entity_key]<>[Recr1008]))
                                     |--Index Spool(SEEK:([p].[parent_key]=[Recr1008]))
                                          |--Table Scan(OBJECT:([tempdb].[dbo].[#linkage] AS [p]))

Here is the same plan in XML format in case you’re into that kind of thing:

http://pastebin.com/Kx559C10

I should also note that this box has 12 CPUs, so if there is some way we can introduce some parallelism then this may help.

Can anyone recommend a method to speed this up?

Answer 1

Have you tried an index on parent_key and adding entity_key as an included column?

Marking root nodes with NULL parents, rather than pointing back to themselves, ought to help:

declare @linkage table (entity_key bigint, parent_key bigint null) 

insert into @linkage values
  (1, NULL), (2, NULL), (3, NULL), (4, 1), (5, 4), (6, 5), (7, 3), (8, 7), (9, 5) 

;with r(entity_key, immediate_parent, root, level) as 
(
  -- Faster search for NULL to find roots.
  select entity_key, entity_key as immediate_parent, entity_key as root, 1 
    from @linkage 
    where parent_key is NULL
  union all
  -- No WHERE clause needed.
  select p.entity_key, r.entity_key, r.root, r.level + 1 
    from r inner join
      @linkage as p on p.parent_key = r.entity_key  
) 
select *
  from r