Question

I have a system that needs to suck up an entire MS SQL database. Currently it does so with something like:

select top 1000 from table where id > 0 order by id;

Then, for the next chunk:

select top 1000 from table where id > 1000 order by id;

And then:

select top 1000 from table where id > 2000 order by id;

And so forth.

In MySQL, I’ve learned that doing LIMIT and OFFSET queries is brutally slow because the database has to first sort the results, then scan over the OFFSET count. When that count gets big, life starts to suck as the read count skyrockets.

My question is this: does the same problem apply to TOP? Put another way, can I expect a really high read count when I run these queries on a database with, say 10,000,000 records, at the time when id > 9,999,000? If so, are there any ways to handle this better?

Answer 1

It will be very fast if ID is indexed. If that column is not indexed then it would case a full table scan.

I would suggest the following in addition:

  select * from table where id > 0 and id <= 1000 order by id ;

This way if you don’t have all records you don’t have bugs (duplicates).