Question

I need to query large amount of data from a database via C# and ADO.NET (IDbDataReader and IDbCommand API).
I’ve created a query like the follwing:

WITH v as 

(SELECT myFields, Datefield 

ROW_NUMBER() OVER (ORDER BY Datefield ASC) AS CurrentRow 

FROM dbTable 

WHERE /**/ 

AND Datefield BETWEEN @pStart AND @pEnd 

// ... )


SELECT myFields, Datefield from v where CurrentRow 
BETWEEN @pRowStart AND @pRowEnd

From the results I have to use an C# API which will transform and generate new data,
thats why an SqlServer – only solution can’t be used.

I want to query against the database with a pagesize of 10000 until there is no more data.
Something like

while (true)
{
  // ... execute reader
  if (reader.HasRows) 
     break;
}

will not work cause I have to use the IDbDataReader interface.

What can I do in my situation?

EDIT + Solution
I iterate over each block in a while loop and check HasRows property of the datareader, cause i can use the specialized type.

Answer 1

On SQL Server 2012 you can use the OFFSET and FETCH clauses for simple and efficient paging, as shown in the ORDER BY clause examples:

SELECT DepartmentID, Name, GroupName
FROM HumanResources.Department
ORDER BY DepartmentID ASC 
    OFFSET @StartingRowNumber - 1 ROWS 
    FETCH NEXT @RowCountPerPage ROWS ONLY;

On previous versions you can use a CTE and ROW_NUMBER() to calculate a number for each row and limit the results:

WITH OrdersRN AS
(
    SELECT ROW_NUMBER() OVER(ORDER BY OrderDate, OrderID) AS RowNum
      ,OrderID
      ,OrderDate
      ,CustomerID
      ,EmployeeID
    FROM dbo.Orders
)

SELECT * 
  FROM OrdersRN
  WHERE RowNum BETWEEN (@PageNum - 1) * @PageSize + 1 
              AND @PageNum * @PageSize
  ORDER BY OrderDate ,OrderID;