Question

I have a SourceTable and a table variable @TQueries containing various T-SQL predicates that target SourceTable.

The expected result is to dynamically generate SELECT statements that return a list of Id’s as specified by the predicates in @TQueries. Each dynamically generated SELECT statement also needs to execute in a particular order, and the final set of values needs to be unique and the ordering must be preserved.

Fortunately, there’s a limit to how many values need to be retrieved and how many dynamic queries need to be generated. The Id list should contain at most 10 Ids, and we don’t expect more than 7 queries.

The following is a sample of this setup, not the actual data/database:

-- Set up some test data, this is quick and dirty just to provide some data to test against
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SourceTable]') AND type in (N'U'))
BEGIN
    -- Create a numbers table, sorta
    SELECT TOP 20 
        IDENTITY(INT,1,1) AS Id,
        ABS(CHECKSUM(NewId())) % 100 AS [SomeValue]
    INTO [SourceTable]
    FROM sysobjects a
END


DECLARE @TQueries TABLE (
    [Ordinal] INT,
    [WherePredicate] NVARCHAR(MAX),
    [OrderByPredicate] NVARCHAR(MAX)
);

-- Simulate SELECTs with different order by that get different data due to varying WHERE clauses and ORDER conditions
INSERT INTO @TQueries VALUES ( 1, N'[Id] IN (6,11,13,7,10,3,15)',  '[SomeValue] ASC' ) -- Sort Asc
INSERT INTO @TQueries VALUES ( 2, N'[Id] IN (9,15,14,20,17)', '[SomeValue] DESC' ) -- Sort Desc
INSERT INTO @TQueries VALUES ( 3, N'[Id] IN (20,10,1,16,11,19,9,15,17,6,2,3,13)', 'NEWID()' ) -- Sort Random

My main issue has been avoiding the use of a CURSOR or iterating through the rows one by one. The closest I’ve come to a set operation that meets this criteria is using a table variable to store the results of each query or a massive CTE.

Suggestions and comments are welcome.

Answer 1

Here’s a solution that builds a single statement both to run all the queries and to return the results.

It uses a similar approach as in your answer when iterating over the @TQueries table, i.e. it also uses {...} tokens where column values from @TQuery should go, and it puts the values there with nested REPLACE() calls.

Other than that, it heavily depends on ranking functions, and I’m not sure if doesn’t really abuse them. You’d need to test this method before deciding if it’s better or worse than the one you’ve got so far.

DECLARE @QueryTemplate nvarchar(max), @FinalSQL nvarchar(max);

SET @QueryTemplate =
N'SELECT
  [Id],
  QueryRank = {Ordinal},
  RowRank = ROW_NUMBER() OVER (ORDER BY {OrderByPredicate})
FROM [dbo].[SourceTable]
WHERE {WherePredicate}
';

SET @FinalSQL =
N'WITH AllData AS (
' +
SUBSTRING(
  (
    SELECT
      'UNION ALL ' +
      REPLACE(REPLACE(REPLACE(@QueryTemplate,
        '{Ordinal}'         , [Ordinal]         ),
        '{OrderByPredicate}', [OrderByPredicate]),
        '{WherePredicate}'  , [WherePredicate]  )
    FROM @TQueries
    ORDER BY [Ordinal]
    FOR XML PATH (''), TYPE
  ).value('.', 'nvarchar(max)'),
  11,                      -- starting just after the first 'UNION ALL '
  CAST(0x7FFFFFFF AS int)  -- max int; no need to specify the exact length
) +
'),
RankedData AS (
  SELECT
    [Id],
    QueryRank,
    RowRank,
    ValueRank = ROW_NUMBER() OVER (PARTITION BY [Id] ORDER BY QueryRank)
  FROM AllData
)SELECT TOP (@top)
  [Id]
FROM RankedData
WHERE ValueRank = 1
ORDER BY
  QueryRank,
  RowRank
';

PRINT @FinalSQL;
EXECUTE sp_executesql @FinalSQL, N'@top int', 10;

Basically, every subquery gets these auxiliary columns:

• QueryRank – a constant value (within the subquery’s result set) derived from [Ordinal];

• RowRank – a ranking assigned to a row based on the [OrderByPredicate].

The result sets are UNIONed and then every entry of every unique value is again ranked (ValueRank) based on the query ranking.

When pulling the final result set, duplicates are suppressed (by the condition ValueRank = 1), and QueryRank and RowRank are used in the ORDER BY clause to preserve the original row order.

I used EXECUTE sp_executesql @query instead of EXECUTE (@query), because the former allows you to add parameters to the query. In particular, I parametrised the number of results to return (the argument of TOP). But you could certainly concatenate that value into the dynamic script directly, just like other things, if you prefer EXECUTE () over EXECUTE sq_executesql.

If you like, you can try this query at SQL Fiddle. (Note: the SQL Fiddle version replaces the @TQueries table variable with the TQueries table.)