Question

I’ve been working for the last week on trying to speed up the search results on a site. What we’ve been working on is moving the paging on the site to SQL. This has sped up the paging drastically, however I still need to query the entire table again to get the total count of the records in that table.

I am caching the totals and only re-running this part of the query when the user alters search parameters to speed up the paging and this works nicely. The issue we are having now is that the load on the CPU of the SQL server is increasing drasticlly and as such the performance of the paging fluctuates drastically (between 100 milliseconds and 2 seconds).

I am just wondering if it wouldn’t be better to cache the entire results table on the web server and use either DataTable.Select or a Linq Statement to query the memory table / list? I realise this will add a large memory load to the Web Server, but we are trying to improve speed and as such it may be worth while to upgrade the Web Servers as they are also load balanced, whereas the SQL box is not.

Answer 1

I found this solution works really well in situations where you have paging, and want to return the total number of rows without running the query twice…

DECLARE @startRow INT ; SET @startrow = 50
;WITH cols
AS
(
    SELECT table_name, column_name, 
        ROW_NUMBER() OVER(ORDER BY table_name, column_name) AS seq, 
        ROW_NUMBER() OVER(ORDER BY table_name DESC, column_name desc) AS totrows
    FROM [INFORMATION_SCHEMA].columns
)
SELECT table_name, column_name, totrows + seq -1 as TotRows
FROM cols
WHERE seq BETWEEN @startRow AND @startRow + 49
ORDER BY seq

Taken from here: SQL Server Paging – The Holy Grail

The total number of rows exists as an extra column in the resultset, but I think that’s a fair tradeoff.

One modification I had to make to the solution in the article was to ensure a unique column was included in the OVER(ORDER BY) column lists.