Question

I need to query the MYSQL with some condition, and get five random different rows from the result.

Say, I have a table named ‘user’, and a field named ‘cash’. I can compose a SQL like:

SELECT * FROM user where cash < 1000 order by RAND() LIMIT 5.

The result is good, totally random, unsorted and different from each other, exact what I want.

But I got from google that the efficiency is bad when the table get large because MySQL creates a temporary table with all the result rows and assigns each one of them a random sorting index. The results are then sorted and returned.

Then I go on searching and got a solution like:

SELECT * FROM `user` AS t1 JOIN (SELECT ROUND(RAND() * ((SELECT MAX(id) FROM `user`)- (SELECT MIN(id) FROM `user`))+(SELECT MIN(id) FROM `user`)) AS id) AS t2 WHERE t1.id >= t2.id AND cash < 1000 ORDER BY t1.id LIMIT 5;      

This method uses JOIN and MAX(id). And the efficiency is better than the first one according to my testing. However, there is a problem. Since I also needs a condition “cash<1000” and if the the RAND() is so big that no row behind it has the cash<1000, then no result will return.

Anyone has good idea of how to compose the SQL that has have the same effect as the first one but has better efficiency?

Or, shall I just do simple query in MYSQL and let PHP randomly pick 5 different rows from the query result?

Your help is appreciated.

Answer 1

To make first query faster, just SELECT id – that will make the temporary table rather small (it will contain only IDs and not all fields of each row) and maybe it will fit in memory (temp table with text/blob are always created on-disk for example). Then when you get a result, run another query SELECT * FROM xy WHERE id IN (a,b,c,d,...). As you mentioned this approach is not very efficient, but as a quick fix this modification will make it several times faster.

One of the best approaches seems to be getting the total number of rows, choosing random numbers and for each result run a new query SELECT * FROM xy WHERE abc LIMIT $random,1. It should be quite efficient for random 3-5, but not good if you want 100 random rows each time 🙂

Also consider caching your results. Often you don’t need different random rows to be displayed on each page load. Generate your random rows only once per minute. If you will generate the data for example via cron, you can live also with query which takes several seconds, as users will see the old data while new data are being generated.

Here are some of my bookmarks for this problem for reference:

http://jan.kneschke.de/projects/mysql/order-by-rand/

http://www.titov.net/2005/09/21/do-not-use-order-by-rand-or-how-to-get-random-rows-from-table/