Question

My understanding of the WITH statement (CTE) is that it executes once per query. With a query like this:

WITH Query1 AS ( ... )
SELECT *
FROM
    SomeTable t1
    LEFT JOIN Query1 t2 ON ...

If this results in 100 rows, I expect that Query1 was executed only once – not 100 times. If that assumption is correct, the time taken to run the entire query is roughly equal to the time taken to: run Query1 + select from SomeTable + join SomeTable to Query1.

I am in a situation where:

• Query1 when run alone takes ~5 seconds (400k rows).
• The remainder of the query, after removing the WITH statement and the LEFT JOIN takes ~15 seconds (400k rows).

So, when running the entire query with the WITH statement and the LEFT JOIN in place, I would have expected the query to complete in a timely manner, instead I’ve let it run for over an hour and once stopped it only got as far as 11k rows.

I am clearly wrong, but why?

Answer 1

Example:

SET NOCOUNT ON;
SET IMPLICIT_TRANSACTIONS ON;

CREATE TABLE MyTable (MyID INT  PRIMARY KEY);
GO
INSERT  MyTable (MyID)
VALUES  (11), (22), (33), (44), (55);

PRINT 'Test MyCTE:';
WITH MyCTE
AS (
    SELECT  *, ROW_NUMBER()OVER(ORDER BY MyID) AS RowNum
    FROM    MyTable
)
SELECT  *
FROM    MyCTE crt
LEFT JOIN MyCTE prev ON crt.RowNum=prev.RowNum+1;

ROLLBACK;

If you run previous script in SSMS (press Ctrl+M -> Actual Execution Plan) then you will get this execution plan for the last query:

In this case, the CTE is executed one time for crt alias and five (!) times for prev alias, once for every row from crt.

So, the answer for this question

Does WITH statement execute once per query or once per row?

is both: once per query (crt) and once per row (prev: once for every for from crt).

To optimize this query, for the start,
1) You can try to store the results from CTE (MyCTE or Query) into a table variable or a temp table

2) Define the primary key of this table as been the join colum(s),

3) Rewrite the final query to use this table variable or temp table.

Off course, you can try to rewrite the final query without this self join between CTE.