Question

I asked the below query earlier, but it was downvoted as the query plan was not provided which I was not aware of. This time around I have provided the query plan and hope some suggestions/comments on how to improve the following query:

The following query takes 5 min to return about 68000 records. But If I remove, the LKP_PrivateSource (i.e. The last Left Join) from the query, then it takes 1 SECOND to return 68000 records. BTY, there are no records in LKP_PrivateSource table. Any idea about what’s causing the issue?

Select 
    Case IsNull(LNK.StockID,0)
        When 0 Then AE.StockID
        Else LNK.StockID
    End StockID,
    IsNull(AE.LinkID,0) LinkID,
    IsNull(LNK.CapitalID,0) CapitalID,
    DE.SourceName,
    AE.SourceDate 'Date',
    IsNull(AE.Formula,'') Formula
From 
    AE_RevenuData AE
Left Join 
    Linking LNK With(NoLock) ON LNK.LinkID = AE.LinkID
Inner Join 
    DE_DataEntities DE ON DE.EntityID = AE.EntityID
Inner Join 
    DataEntityIDs TE ON TE.EntityID = DE.EntityID
Inner Join 
    STG_LockedEntityData STG ON STG.StockID = IsNull(LNK.StockID, AE.StockID) AND STG.CapitalID = IsNull(LNK.CapitalID, 0) 
Left Join 
    LKP_PrivateSource PS ON PS.PSourceId = AE.PSourceID
Where 
    AE.ProjectID IN (13)
    AND AE.LinkID IS NOT NULL

Please see below the plan:

  |--Compute Scalar(DEFINE:([Expr1017]=CASE WHEN [Expr1026]=(0) THEN [MYDBNAME].[dbo].[AE_RevenueData].[StockID] as [AE].[StockID] ELSE [MYDBNAME].[dbo].[INV_InvestorFundLinking].[StockID] as [LNK].[StockID] END))
   |--Nested Loops(Left Outer Join, OUTER REFERENCES:([Expr1027]))
        |--Filter(WHERE:([MYDBNAME].[dbo].[STG_LockedEntityData].[StockID] as [STG].[StockID]=isnull([MYDBNAME].[dbo].[INV_InvestorFundLinking].[StockID] as [LNK].[StockID],[MYDBNAME].[dbo].[AE_RevenueData].[StockID] as [AE].[StockID]) AND [MYDBNAME].[dbo].[STG_LockedEntityData].[CapitalID] as [STG].[CapitalID]=[Expr1019]))
        |    |--Compute Scalar(DEFINE:([Expr1019]=isnull([MYDBNAME].[dbo].[INV_InvestorFundLinking].[CapitalID] as [LNK].[CapitalID],(0)), [Expr1026]=isnull([MYDBNAME].[dbo].[INV_InvestorFundLinking].[StockID] as [LNK].[StockID],(0))))
        |         |--Nested Loops(Left Outer Join, OUTER REFERENCES:([AE].[LinkID]))
        |              |--Nested Loops(Inner Join, WHERE:([MYDBNAME].[dbo].[AE_RevenueData].[LinkID] as [AE].[LinkID]=[MYDBNAME].[dbo].[STG_LockedEntityData].[LinkID] as [STG].[LinkID]))
        |              |    |--Sort(ORDER BY:([Expr1027] ASC))
        |              |    |    |--Hash Match(Inner Join, HASH:([TE].[EntityID])=([AE].[EntityID]))
        |              |    |         |--Nested Loops(Inner Join, OUTER REFERENCES:([Uniq1008], [DE].[ListingID]))
        |              |    |         |    |--Nested Loops(Inner Join, OUTER REFERENCES:([TE].[EntityID]))
        |              |    |         |    |    |--Table Scan(OBJECT:([MYDBNAME].[dbo].[DataEntityIDs] AS [TE]))
        |              |    |         |    |    |--Index Seek(OBJECT:([MYDBNAME].[dbo].[DE_DataEntities].[PK_DE_DataSources] AS [DE]), SEEK:([DE].[EntityID]=[MYDBNAME].[dbo].[DataEntityIDs].[EntityID] as [TE].[EntityID]) ORDERED FORWARD)
        |              |    |         |    |--Clustered Index Seek(OBJECT:([MYDBNAME].[dbo].[DE_DataEntities].[IX_DE_DataSources] AS [DE]), SEEK:([DE].[ListingID]=[MYDBNAME].[dbo].[DE_DataEntities].[ListingID] as [DE].[ListingID] AND [Uniq1008]=[Uniq1008]) LOOKUP ORDERED FORWARD)
        |              |    |         |--Compute Scalar(DEFINE:([Expr1018]=[MYDBNAME].[dbo].[AE_RevenueData].[LinkID] as [AE].[LinkID], [Expr1020]=isnull([MYDBNAME].[dbo].[AE_RevenueData].[Formula] as [AE].[Formula],''), [Expr1021]=CONVERT(bit,[MYDBNAME].[dbo].[AE_RevenueData].[IsSumOfFunds] as [AE].[IsSumOfFunds],0), [Expr1022]=isnull([MYDBNAME].[dbo].[AE_RevenueData].[ClientSpecificSource] as [AE].[ClientSpecificSource],N''), [Expr1023]=isnull([MYDBNAME].[dbo].[AE_RevenueData].[GenericSource] as [AE].[GenericSource],N''), [Expr1027]=isnull([MYDBNAME].[dbo].[AE_RevenueData].[PSourceID] as [AE].[PSourceID],(0))))
        |              |    |              |--Clustered Index Seek(OBJECT:([MYDBNAME].[dbo].[AE_RevenueData].[IX_AE_RevenueData] AS [AE]), SEEK:([AE].[ProjectID]=(13)),  WHERE:([MYDBNAME].[dbo].[AE_RevenueData].[LinkID] as [AE].[LinkID] IS NOT NULL) ORDERED FORWARD)
        |              |    |--Table Scan(OBJECT:([MYDBNAME].[dbo].[STG_LockedEntityData] AS [STG]))
        |              |--Index Seek(OBJECT:([MYDBNAME].[dbo].[INV_InvestorFundLinking].[PK_Linking_1] AS [LNK]), SEEK:([LNK].[LinkID]=[MYDBNAME].[dbo].[AE_RevenueData].[LinkID] as [AE].[LinkID]) ORDERED FORWARD)
        |--Clustered Index Seek(OBJECT:([MYDBNAME].[dbo].[LKP_PrivateSource].[PK_LKP_FactsetSource] AS [PS]), SEEK:([PS].[PSourceID]=[Expr1027]) ORDERED FORWARD)

Please see below the plan without Join on LKP_PrivateSource

  |--Compute Scalar(DEFINE:([Expr1015]=CASE WHEN [Expr1024]=(0) THEN [MYDBNAME].[dbo].[AE_RevenueData].[StockID] as [AE].[StockID] ELSE [MYDBNAME].[dbo].[Linking].[StockID] as [LNK].[StockID] END))
   |--Nested Loops(Left Outer Join, OUTER REFERENCES:([Expr1027]))
       |--Filter(WHERE:([MYDBNAME].[dbo].[STG_LockedEntityData].[StockID] as [STG].[StockID]=isnull([MYDBNAME].[dbo].[Linking].[StockID] as [LNK].[StockID],[MYDBNAME].[dbo].[AE_RevenueData].[StockID] as [AE].[StockID]) AND [MYDBNAME].[dbo].[STG_LockedEntityData].[CapitalID] as [STG].[CapitalID]=[Expr1017]))
            |--Compute Scalar(DEFINE:([Expr1017]=isnull([MYDBNAME].[dbo].[Linking].[CapitalID] as [LNK].[CapitalID],(0)), [Expr1024]=isnull([MYDBNAME].[dbo].[Linking].[StockID] as [LNK].[StockID],(0))))
                 |--Nested Loops(Left Outer Join, OUTER REFERENCES:([AE].[LinkID]))
                      |--Hash Match(Inner Join, HASH:([AE].[LinkID])=([STG].[LinkID]))
                      |    |--Hash Match(Inner Join, HASH:([TE].[EntityID])=([AE].[EntityID]))
                      |    |    |--Nested Loops(Inner Join, OUTER REFERENCES:([Uniq1008], [DE].[ListingID]))
                      |    |    |    |--Nested Loops(Inner Join, OUTER REFERENCES:([TE].[EntityID]))
                      |    |    |    |    |--Table Scan(OBJECT:([MYDBNAME].[dbo].[DataEntityIDs] AS [TE]))
                      |    |    |    |    |--Index Seek(OBJECT:([MYDBNAME].[dbo].[DE_DataEntities].[PK_DE_DataSources] AS [DE]), SEEK:([DE].[EntityID]=[MYDBNAME].[dbo].[DataEntityIDs].[EntityID] as [TE].[EntityID]) ORDERED FORWARD)
                      |    |    |    |--Clustered Index Seek(OBJECT:([MYDBNAME].[dbo].[DE_DataEntities].[IX_DE_DataSources] AS [DE]), SEEK:([DE].[ListingID]=[MYDBNAME].[dbo].[DE_DataEntities].[ListingID] as [DE].[ListingID] AND [Uniq1008]=[Uniq1008]) LOOKUP ORDERED FORWARD)
                      |    |    |--Compute Scalar(DEFINE:([Expr1016]=[MYDBNAME].[dbo].[AE_RevenueData].[LinkID] as [AE].[LinkID], [Expr1018]=isnull([MYDBNAME].[dbo].[AE_RevenueData].[Formula] as [AE].[Formula],''), [Expr1019]=CONVERT(bit,[MYDBNAME].[dbo].[AE_RevenueData].[IsSumOfFunds] as [AE].[IsSumOfFunds],0), [Expr1020]=isnull([MYDBNAME].[dbo].[AE_RevenueData].[ClientSpecificSource] as [AE].[ClientSpecificSource],N''), [Expr1021]=isnull([MYDBNAME].[dbo].[AE_RevenueData].[GenericSource] as [AE].[GenericSource],N'')))
                      |    |         |--Clustered Index Seek(OBJECT:([MYDBNAME].[dbo].[AE_RevenueData].[IX_AE_RevenueData] AS [AE]), SEEK:([AE].[ProjectID]=(13)),  WHERE:([MYDBNAME].[dbo].[AE_RevenueData].[LinkID] as [AE].[LinkID] IS NOT NULL) ORDERED FORWARD)
                      |    |--Table Scan(OBJECT:([MYDBNAME].[dbo].[STG_LockedEntityData] AS [STG]))
                      |--Index Seek(OBJECT:([MYDBNAME].[dbo].[Linking].[PK_Linking_1] AS [LNK]), SEEK:([LNK].[LinkID]=[MYDBNAME].[dbo].[AE_RevenueData].[LinkID] as [AE].[LinkID]) ORDERED FORWARD)

Answer 1

For some reason, adding a LEFT JOIN to LKP_PrivateSource makes optimizer use NESTED LOOPS instead of a HASH JOIN to join STG_LockedEntityData (which goes earlier).

It’s hard to tell why it is, but to improve the nested loops, create an index on STG_LockedEntityData (stock_id, capital_id)