Question

I’ve got a serious problem with a nested query, which I suspect MySQL is interpreting as a correlated subquery when in fact it should be uncorrelated. The query spans two tables, one being a list of products and the other being their price at various points in time. My aim is to return each price record for products that have a price range above a certain value for the whole time. My query looks like this:

SELECT oP.id, oP.title, oCR.price, oC.timestamp
FROM Crawl_Results AS oCR
        JOIN Products AS oP 
            ON oCR.product = oP.id
        JOIN Crawls AS oC 
            ON oCR.crawl = oC.id
WHERE oP.id
IN (
    SELECT iP.id
    FROM Products AS iP
            JOIN Crawl_Results AS iCR 
               ON iP.id = iCR.product
    WHERE iP.category =2
    GROUP BY iP.id
    HAVING (
              MAX( iCR.price ) - MIN( iCR.price )
           ) >1
)
ORDER BY oP.id ASC

Taken alone, the inner query executes fine and returns a list of the id’s of the products with a price range above the criterion. The outer query also works fine if I provide a simple list of ids in the IN clause. When I run them together however, the query takes ~3min to return ~1500 rows, so I think it’s executing the inner query for every row of the outer, which is not ideal. I did have the columns aliased the same in the inner and outer queries, so I thought that aliasing them differently in the inner and outer as above would fix it, but it didn’t.

Any ideas as to what’s going on here?

Answer 1

MySQL might think it could use indexes to execute the query faster by running it once for every OP.id. The first thing to check is if your statistics are up to date.

You could rewrite the where ... in as a filtering inner join. This is less likely to be “optimized” for seeks:

SELECT  *
FROM    Crawl_Results AS oCR
JOIN    Products AS oP 
ON      oCR.product = oP.id
JOIN    Crawls AS oC 
ON      oCR.crawl = oC.id
JOIN    (
        SELECT  iP.id
        FROM    Products AS iP
        JOIN    Crawl_Results AS iCR 
        ON      iP.id = iCR.product
        WHERE   iP.category =2
        GROUP BY 
                iP.id
        HAVING  (MAX(iCR.price) - MIN(iCR.price)) > 1
        ) filter
ON      OP.id = filter.id

Another option is to use a temporary table. You store the result of the subquery in a temporary table and join on that. That really forces MySQL not to execute the subquery as a correlated query.