Question

I have a piece of code in an ASP.NET application that builds a SQL query from a list of parameters. The number of parameters can vary, so various criteria can be added to this query. The database is Microsoft SQL Server 2008.

All the AND and OR’s are generated programmatically.

The query took more than 3 seconds to execute, but after some profiling and indexes it runs less than a second. Still I think the query itself can be optimized. I’ve looked at execution plans, but that doesn’t mean much to me – not being a SQL guru.

I wonder if the query can be done in a more intelligent fashion – I haven’t been able to figure it out. Here’s an example of the query:

 SELECT [id], [WorkTitle], [CreateDate], [UpdateDate], [Writer], [ValidFrom], 

[ValidTo], [Text] 
 FROM dbo.Texts T  
 WHERE Category_id = 3 AND '2012-11-06' BETWEEN ValidFrom AND ValidTo  
 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 1 AND CL.Value = '95068')       
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 1))  

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 2 AND CL.Value = 'C')       
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 2))  

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 3 AND CL.Value = 'HEL')       
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 3))  

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 4 AND CL.Value = 'CC')      
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 4))  

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 5 AND CL.Value = NULL)     
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 5))  

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 7 AND CL.Value = '321')      
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 7))  

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 9 AND CL.Value = 'DK7778')    
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 9))  

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 10 AND CL.Value = 'TFS')   
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 10)) 

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 11 AND CL.Value = 'TMP')   
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 11)) 

 AND (EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id   AND CL.Criteria_id = 13 AND CL.Value = 'OY-VKB')   
    OR NOT EXISTS (SELECT 'X' FROM dbo.Criteria_List CL WHERE T.id = CL.Text_id AND CL.Criteria_id = 13)) 

Any tips and tricks are appreciated.

Cheers
Jens

Answer 1

• For table Texts I’d add an index on (Category_id, ValidFrom) INCLUDE (ValidTo), if there isn’t one. (If you already have indexes on either (Category_id, ValidFrom, ValidTo) or on (Category_id, ValidFrom), they may be quite good too.

• For the Criteria_List table an index on (Text_id, Criteria_id, Value) would probably be enough for the optimizer to produce a good execution plan.

  The second option (or maybe even better, you’d have to test the execution plans and running times with your tables sizes and distributions) would be two indexes, one on (Criteria_id, Text_id) and one on (Criteria_id, Value, Text_id).

---

You could rewrite the 10 conditions like this – but the indexing should be taken care anyway:

 WHERE Category_id = 3 AND '2012-11-06' BETWEEN ValidFrom AND ValidTo  
 AND NOT EXISTS ( SELECT 'X' FROM dbo.Criteria_List CL 
                  WHERE T.id = CL.Text_id AND CL.Criteria_id = 1 
                    AND (CL.Value <> '95068' OR CL.Value IS NULL)
                ) 
 AND NOT EXISTS ( SELECT 'X' FROM dbo.Criteria_List CL 
                  WHERE T.id = CL.Text_id AND CL.Criteria_id = 2 
                    AND (CL.Value <> 'C' OR CL.Value IS NULL)
                ) 
 ...
 AND NOT EXISTS ( SELECT 'X' FROM dbo.Criteria_List CL 
                  WHERE T.id = CL.Text_id AND CL.Criteria_id = 5 
                    AND (CL.Value IS NOT NULL)
                ) 
 ...