This question is linked to my previous one ( posted as an anonymous user – now I have an account ) and, before I begin, I would like to give the credit to Rob Farley for providing the right indexing schema.
But the problem is not the indexing schema.
It’s the Query Optimizer !
The query :
SELECT s.ID_i
, s.ShortName_v
, sp.Path_v
, ( SELECT TOP 1 1 -- has also user access on subsites ?
FROM SitePath_T usp
, UserSiteRight_t usr
WHERE usr.SiteID_i = usp.SiteID_i
AND usp.Path_v LIKE sp.Path_v + '%_'
AND usr.UserID_i = 1 )
FROM Site_T s
, SitePath_T sp
WHERE sp.SiteID_i = s.ID_i
AND s.ShortName_v LIKE '[a-y]%'
AND s.ParentID_i = 1
AND EXISTS ( SELECT *
FROM SitePath_T usp
, UserSiteRight_t usr
WHERE usr.SiteID_i = usp.SiteID_i
AND usp.Path_v LIKE sp.Path_v + '%'
AND usr.UserID_i = 1 )
… runs in :
CPU Reads Writes Duration
2073 49572 0 2241 -- more than 2 sec
Execution plan :
|--Compute Scalar(DEFINE:([Expr1014]=[Expr1014]))
|--Nested Loops(Left Outer Join, OUTER REFERENCES:([sp].[Path_v]))
|--Nested Loops(Left Semi Join, OUTER REFERENCES:([Expr1016], [Expr1017], [Expr1018], [Expr1019]))
| |--Merge Join(Inner Join, MERGE:([sp].[SiteID_i])=([s].[ID_i]), RESIDUAL:([dbo].[SitePath_T].[SiteID_i] as [sp].[SiteID_i]=[dbo].[Site_T].[ID_i] as [s].[ID_i]))
| | |--Compute Scalar(DEFINE:([Expr1016]=[dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%', [Expr1017]=LikeRangeStart([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%'), [Expr1018]=LikeRangeEnd([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%'), [Expr1019]=LikeRangeInfo([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%')))
| | | |--Index Scan(OBJECT:([dbo].[SitePath_T].[IDX_SitePath_SiteID_<Path>] AS [sp]), ORDERED FORWARD)
| | |--Sort(ORDER BY:([s].[ID_i] ASC))
| | |--Clustered Index Seek(OBJECT:([dbo].[Site_T].[IDXC_Site_ParentID+ShortName+ID] AS [s]), SEEK:([s].[ParentID_i]=(1) AND [s].[ShortName_v] >= '9þþþþþ' AND [s].[ShortName_v] < 'Z'), WHERE:([dbo].[Site_T].[ShortName_v] as [s].[ShortName_v] like '[a-y]%') ORDERED FORWARD)
| |--Nested Loops(Inner Join, OUTER REFERENCES:([usp].[SiteID_i], [Expr1020]) WITH UNORDERED PREFETCH)
| |--Clustered Index Scan(OBJECT:([dbo].[SitePath_T].[IDXC_SitePath_Path+SiteID] AS [usp]), WHERE:([dbo].[SitePath_T].[Path_v] as [usp].[Path_v] like [Expr1016]))
| |--Index Seek(OBJECT:([dbo].[UserSiteRight_T].[IDX_UserSiteRight_UserID+SiteID] AS [usr]), SEEK:([usr].[UserID_i]=(1) AND [usr].[SiteID_i]=[dbo].[SitePath_T].[SiteID_i] as [usp].[SiteID_i]) ORDERED FORWARD)
|--Compute Scalar(DEFINE:([Expr1014]=(1)))
|--Top(TOP EXPRESSION:((1)))
|--Nested Loops(Inner Join, OUTER REFERENCES:([usp].[SiteID_i], [Expr1021]) WITH UNORDERED PREFETCH)
|--Clustered Index Scan(OBJECT:([dbo].[SitePath_T].[IDXC_SitePath_Path+SiteID] AS [usp]), WHERE:([dbo].[SitePath_T].[Path_v] as [usp].[Path_v] like [dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_'))
|--Index Seek(OBJECT:([dbo].[UserSiteRight_T].[IDX_UserSiteRight_UserID+SiteID] AS [usr]), SEEK:([usr].[UserID_i]=(1) AND [usr].[SiteID_i]=[dbo].[SitePath_T].[SiteID_i] as [usp].[SiteID_i]) ORDERED FORWARD)
But if I enforce the indexes, the following query :
SELECT s.ID_i
, s.ShortName_v
, sp.Path_v
, ( SELECT TOP 1 1 -- has also user access on subsites ?
FROM SitePath_T usp WITH ( INDEX ( [IDX_SitePath_Path+SiteID] ) )
-- same performance when using WITH ( INDEX ( [IDX_SitePath_Path_INC<SiteID>] ) )
, UserSiteRight_t usr WITH ( INDEX ( [IDX_UserSiteRight_UserID+SiteID] ) )
WHERE usr.SiteID_i = usp.SiteID_i
AND usp.Path_v LIKE sp.Path_v + '%_'
AND usr.UserID_i = 1)
FROM Site_T s
, SitePath_T sp WITH ( INDEX ( [IDX_SitePath_SiteID+Path] ) )
-- same performance when using WITH ( INDEX ( [IDX_SitePath_SiteID_INC<Path>] ) )
WHERE sp.SiteID_i = s.ID_i
AND s.ShortName_v LIKE '[a-y]%'
AND s.ParentID_i = 1
AND EXISTS ( SELECT *
FROM SitePath_T usp WITH ( INDEX ( [IDX_SitePath_Path+SiteID] ) )
-- same performance when using WITH ( INDEX ( [IDX_SitePath_Path_INC<SiteID>] ) )
, UserSiteRight_t usr WITH ( INDEX ( [IDX_UserSiteRight_UserID+SiteID] ) )
WHERE usr.SiteID_i = usp.SiteID_i
AND usp.Path_v LIKE sp.Path_v + '%'
AND usr.UserID_i = 1 )
will run in :
CPU Reads Writes Duration
50 11237 0 55
the duration will drop to 55 milliseconds ( from more than 2 sec ) !!!!
And I’m happy with this result !
Execution plan :
|--Compute Scalar(DEFINE:([Expr1014]=[Expr1014]))
|--Nested Loops(Left Outer Join, OUTER REFERENCES:([sp].[Path_v]))
|--Nested Loops(Left Semi Join, OUTER REFERENCES:([Expr1016], [Expr1017], [Expr1018], [Expr1019]))
| |--Merge Join(Inner Join, MERGE:([sp].[SiteID_i])=([s].[ID_i]), RESIDUAL:([dbo].[SitePath_T].[SiteID_i] as [sp].[SiteID_i]=[dbo].[Site_T].[ID_i] as [s].[ID_i]))
| | |--Compute Scalar(DEFINE:([Expr1016]=[dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%', [Expr1017]=LikeRangeStart([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%'), [Expr1018]=LikeRangeEnd([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%'), [Expr1019]=LikeRangeInfo([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%')))
| | | |--Index Scan(OBJECT:([dbo].[SitePath_T].[IDX_SitePath_SiteID_<Path>] AS [sp]), ORDERED FORWARD)
| | |--Sort(ORDER BY:([s].[ID_i] ASC))
| | |--Clustered Index Seek(OBJECT:([dbo].[Site_T].[IDXC_Site_ParentID+ShortName+ID] AS [s]), SEEK:([s].[ParentID_i]=(1) AND [s].[ShortName_v] >= '9þþþþþ' AND [s].[ShortName_v] < 'Z'), WHERE:([dbo].[Site_T].[ShortName_v] as [s].[ShortName_v] like '[a-y]%') ORDERED FORWARD)
| |--Nested Loops(Inner Join, OUTER REFERENCES:([usp].[SiteID_i], [Expr1023]) WITH UNORDERED PREFETCH)
| |--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1017], [Expr1018], [Expr1019]))
| | |--Compute Scalar(DEFINE:([Expr1017]=[Expr1017], [Expr1018]=[Expr1018], [Expr1019]=[Expr1019]))
| | | |--Constant Scan
| | |--Index Seek(OBJECT:([dbo].[SitePath_T].[IDX_SitePath_Path+SiteID] AS [usp]), SEEK:([usp].[Path_v] > [Expr1017] AND [usp].[Path_v] < [Expr1018]), WHERE:([dbo].[SitePath_T].[Path_v] as [usp].[Path_v] like [Expr1016]) ORDERED FORWARD)
| |--Index Seek(OBJECT:([dbo].[UserSiteRight_T].[IDX_UserSiteRight_UserID+SiteID] AS [usr]), SEEK:([usr].[UserID_i]=(1) AND [usr].[SiteID_i]=[dbo].[SitePath_T].[SiteID_i] as [usp].[SiteID_i]) ORDERED FORWARD)
|--Compute Scalar(DEFINE:([Expr1014]=(1)))
|--Top(TOP EXPRESSION:((1)))
|--Nested Loops(Inner Join, OUTER REFERENCES:([usp].[SiteID_i], [Expr1027]) WITH UNORDERED PREFETCH)
|--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1024], [Expr1025], [Expr1026]))
| |--Compute Scalar(DEFINE:([Expr1024]=LikeRangeStart([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_'), [Expr1025]=LikeRangeEnd([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_'), [Expr1026]=LikeRangeInfo([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_')))
| | |--Constant Scan
| |--Index Seek(OBJECT:([dbo].[SitePath_T].[IDX_SitePath_Path+SiteID] AS [usp]), SEEK:([usp].[Path_v] > [Expr1024] AND [usp].[Path_v] < [Expr1025]), WHERE:([dbo].[SitePath_T].[Path_v] as [usp].[Path_v] like [dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_') ORDERED FORWARD)
|--Index Seek(OBJECT:([dbo].[UserSiteRight_T].[IDX_UserSiteRight_UserID+SiteID] AS [usr]), SEEK:([usr].[UserID_i]=(1) AND [usr].[SiteID_i]=[dbo].[SitePath_T].[SiteID_i] as [usp].[SiteID_i]) ORDERED FORWARD)
The next step is to run it for different users, thus I will declare UserID_i as a variable :
DECLARE @UserID_i INT
SELECT @UserID_i = 1
BUT NOW THE BELOW QUERY BECOMES CRAZY SLOW !!!
SELECT s.ID_i
, s.ShortName_v
, sp.Path_v
, ( SELECT TOP 1 1 -- has also user access on subsites ?
FROM SitePath_T usp WITH ( INDEX ( [IDX_SitePath_Path+SiteID] ) )
, UserSiteRight_t usr WITH ( INDEX ( [IDX_UserSiteRight_UserID+SiteID] ) )
WHERE usr.SiteID_i = usp.SiteID_i
AND usp.Path_v LIKE sp.Path_v + '%_'
AND usr.UserID_i = @UserID_i)
FROM Site_T s
, SitePath_T sp WITH ( INDEX ( [IDX_SitePath_SiteID+Path] ) )
WHERE sp.SiteID_i = s.ID_i
AND s.ShortName_v LIKE '[a-y]%'
AND s.ParentID_i = 1
AND EXISTS ( SELECT *
FROM SitePath_T usp WITH ( INDEX ( [IDX_SitePath_Path+SiteID] ) )
, UserSiteRight_t usr WITH ( INDEX ( [IDX_UserSiteRight_UserID+SiteID] ) )
WHERE usr.SiteID_i = usp.SiteID_i
AND usp.Path_v LIKE sp.Path_v + '%'
AND usr.UserID_i = @UserID_i )
The duration is now over 7 seconds !!!
CPU Reads Writes Duration
7421 149984 35 7625
And the execution plan :
|--Compute Scalar(DEFINE:([Expr1014]=[Expr1014]))
|--Nested Loops(Left Outer Join, OUTER REFERENCES:([sp].[Path_v]))
|--Nested Loops(Left Semi Join, WHERE:([dbo].[SitePath_T].[Path_v] as [usp].[Path_v] like [Expr1016]))
| |--Merge Join(Inner Join, MERGE:([sp].[SiteID_i])=([s].[ID_i]), RESIDUAL:([dbo].[SitePath_T].[SiteID_i] as [sp].[SiteID_i]=[dbo].[Site_T].[ID_i] as [s].[ID_i]))
| | |--Compute Scalar(DEFINE:([Expr1016]=[dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%', [Expr1017]=LikeRangeStart([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%'), [Expr1018]=LikeRangeEnd([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%'), [Expr1019]=LikeRangeInfo([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%')))
| | | |--Index Scan(OBJECT:([dbo].[SitePath_T].[IDX_SitePath_SiteID+Path] AS [sp]), ORDERED FORWARD)
| | |--Sort(ORDER BY:([s].[ID_i] ASC))
| | |--Clustered Index Seek(OBJECT:([dbo].[Site_T].[IDXC_Site_ParentID+ShortName+ID] AS [s]), SEEK:([s].[ParentID_i]=(1) AND [s].[ShortName_v] >= '9þþþþþ' AND [s].[ShortName_v] < 'Z'), WHERE:([dbo].[Site_T].[ShortName_v] as [s].[ShortName_v] like '[a-y]%') ORDERED FORWARD)
| |--Table Spool
| |--Hash Match(Inner Join, HASH:([usr].[SiteID_i])=([usp].[SiteID_i]))
| |--Index Seek(OBJECT:([dbo].[UserSiteRight_T].[IDX_UserSiteRight_UserID+SiteID] AS [usr]), SEEK:([usr].[UserID_i]=[@UserID_i]) ORDERED FORWARD)
| |--Index Scan(OBJECT:([dbo].[SitePath_T].[IDX_SitePath_Path+SiteID] AS [usp]))
|--Compute Scalar(DEFINE:([Expr1014]=(1)))
|--Top(TOP EXPRESSION:((1)))
|--Nested Loops(Inner Join, WHERE:([dbo].[UserSiteRight_T].[SiteID_i] as [usr].[SiteID_i]=[dbo].[SitePath_T].[SiteID_i] as [usp].[SiteID_i]))
|--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1020], [Expr1021], [Expr1022]))
| |--Compute Scalar(DEFINE:([Expr1020]=LikeRangeStart([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_'), [Expr1021]=LikeRangeEnd([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_'), [Expr1022]=LikeRangeInfo([dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_')))
| | |--Constant Scan
| |--Index Seek(OBJECT:([dbo].[SitePath_T].[IDX_SitePath_Path+SiteID] AS [usp]), SEEK:([usp].[Path_v] > [Expr1020] AND [usp].[Path_v] < [Expr1021]), WHERE:([dbo].[SitePath_T].[Path_v] as [usp].[Path_v] like [dbo].[SitePath_T].[Path_v] as [sp].[Path_v]+'%_') ORDERED FORWARD)
|--Table Spool
|--Index Seek(OBJECT:([dbo].[UserSiteRight_T].[IDX_UserSiteRight_UserID+SiteID] AS [usr]), SEEK:([usr].[UserID_i]=[@UserID_i]) ORDERED FORWARD)
The execution plan is changing completely when I’m using a variable instead of hard coding the UserID_i value !
Why the query optimizer is behaving like this ?
How can I enforce the execution plan to be the same as the second fast query ?
Thank you.
UPDATE 1
Deleted ( irrelevant )
UPDATE 2
It seems that I am not the only one having this problem.
Please check the following topics :
Why does the SqlServer optimizer get so confused with parameters?
Known issue?: SQL Server 2005 stored procedure fails to complete with a parameter
UPDATE 3
An excellent article from SQL Server Query Optimization Team covering parameter sniffing : I Smell a Parameter !
After reading the above articles ( provided in Update 2 and Update 3 ) I finally understood more about how Sql Server is treating / caching execution plans.
Adding
OPTION ( RECOMPILE )at the end of mySELECTstatements will force Sql Server to recalculate the execution plan ( and not using the cached one ) every time the query will be run, thus choosing the best plan matching the variable.