Question

I have a huge table of > 10 million rows. I need to efficiently grab a random sampling of 5000 from it. I have some constriants that reduces the total rows I am looking for to like 9 millon.

I tried using order by NEWID(), but that query will take too long as it has to do a table scan of all rows.

Is there a faster way to do this?

Answer 1

If you can use a pseudo-random sampling and you’re on SQL Server 2005/2008, then take a look at TABLESAMPLE. For instance, an example from SQL Server 2008 / AdventureWorks 2008 which works based on rows:

USE AdventureWorks2008;  GO    SELECT FirstName, LastName FROM Person.Person  TABLESAMPLE (100 ROWS) WHERE EmailPromotion = 2; 

The catch is that TABLESAMPLE isn’t exactly random as it generates a given number of rows from each physical page. You may not get back exactly 5000 rows unless you limit with TOP as well. If you’re on SQL Server 2000, you’re going to have to either generate a temporary table which match the primary key or you’re going to have to do it using a method using NEWID().