Question

For fetching records from a table I use this mysql query:

SELECT 
    a.id as aid, a.data1 as adata1, a.data2 as adata2
    b.id as bid, b.data1 as bdata1, b.data2 as bdata2
FROM table AS a
JOIN table AS b ON ( a.id <> b.id ) 
WHERE (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100
ORDER BY RAND() 
LIMIT 1

This query fetching exactly the records which I need, but unfortunately because of RAND() is this query quite slow.

I’ve found some ways, how to avoid using RAND() function, for example here. But my problem is, that I still cannot find a way, how to replace RAND() function in this query.
In some simple query is not problem to replace RAND(), but I don’t know, how to do that in the example above… because of more conditions in the WHERE clause.

Answer 1

Since you are using MySQL you can try with the following SQL queries that first gets a count from the table, then selects a random offset based on that count. It then prepares a statement so the calculated offset can be used and executes the statement.

SELECT @count := COUNT(*) FROM table AS a JOIN table AS b ON ( a.id <> b.id ) WHERE (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100;
SET @offset = CONVERT(FLOOR(RAND() * @count), SIGNED);
PREPARE mystatement FROM "SELECT 
                          a.id as aid, a.data1 as adata1, a.data2 as adata2
                          b.id as bid, b.data1 as bdata1, b.data2 as bdata2
                          FROM table AS a
                          JOIN table AS b ON ( a.id <> b.id ) 
                          WHERE (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100 LIMIT ?, 1";
EXECUTE mystatement USING @offset;
DEALLOCATE PREPARE mystatement;

On a large dataset should perform faster than ORDER BY RAND(), try and let me know … 😉

EDIT

The queries will not work used on phpmyadmin, so run them using the MySQL console or write a php script in which you have two option, the first one is let mysql do the work :

mysql_query('SELECT @count := COUNT(*) FROM table AS a JOIN table AS b ON ( a.id <> b.id ) WHERE (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100');
mysql_query('SET @offset = CONVERT(FLOOR(RAND() * @count), SIGNED)');
mysql_query('PREPARE mystatement FROM "SELECT 
                          a.id as aid, a.data1 as adata1, a.data2 as adata2
                          b.id as bid, b.data1 as bdata1, b.data2 as bdata2
                          FROM table AS a
                          JOIN table AS b ON ( a.id <> b.id ) 
                          WHERE (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100 LIMIT ?, 1"');
$res = mysql_query('EXECUTE mystatement USING @offset');
$row = mysql_fetch_assoc($res);
print_r($row);

The second option that could be even more faster consist of doing a part of the work with MySQL and the other part with the programming language (in our case PHP) :

$res = mysql_query("SELECT COUNT(*) FROM table AS a JOIN table AS b ON ( a.id <> b.id ) WHERE (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100')");
$row = mysql_fetch_array($res);
$offset = rand(0, $row[0]-1);

$res = mysql_query("SELECT 
                              a.id as aid, a.data1 as adata1, a.data2 as adata2
                              b.id as bid, b.data1 as bdata1, b.data2 as bdata2
                              FROM table AS a
                              JOIN table AS b ON ( a.id <> b.id ) 
                              WHERE (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100 LIMIT $offset, 1");
$row = mysql_fetch_assoc($res);

Another alternative way to speed up the ORDER BY RAND() that I’ve found consist in a query like the following :

SELECT 
    a.id as aid, a.data1 as adata1, a.data2 as adata2
    b.id as bid, b.data1 as bdata1, b.data2 as bdata2
FROM table AS a
JOIN table AS b ON ( a.id <> b.id ) 
WHERE (RAND() < (SELECT ((1/COUNT(*))*10) FROM table AS a JOIN table AS b ON ( a.id <> b.id ) ) )
 AND (a.data1=1 AND b.data1=1) AND ABS( a.rating - b.rating ) <100
ORDER BY RAND() 
LIMIT 1

Don’t forget to update me about the result you get 😉 .