Question

I have a very complex MySQL query that includes use of the same subquery three times. Will MySQL actually run the subquery three times? (It’s an expensive one.) If so, is there a way for me to tell MySQL to save or cache the results so it won’t do that? I could save the data in a large array then re-feed it to MySQL, but I’d rather not move it out and back into the database like that.

This is the subquery that appears three times:

SELECT id FROM programs  WHERE submitter_id=32 AND id in (     SELECT id FROM programs      WHERE feed_id=2478 AND id in (         SELECT program_id FROM playlist_program_map          WHERE playlist_id=181))) 

And here’s an example of the full query in which the query appears:

SELECT object_id, programs.created AS created,  MATCH(text) AGAINST ('excellent ' IN BOOLEAN MODE) AS relevance  FROM comments_programs USE INDEX (text)  LEFT JOIN programs ON programs.id=object_id  WHERE object_id IN (     SELECT id FROM programs      WHERE 1 AND id IN (         SELECT id FROM programs          WHERE submitter_id=32 AND id in (             SELECT id FROM programs              WHERE feed_id=2478 AND id in (                 SELECT program_id FROM playlist_program_map                  WHERE playlist_id=181))))  AND MATCH(text) AGAINST ('excellent ' IN BOOLEAN MODE)>0)  UNION (  SELECT object_id, programs.created AS created,  MATCH(text) AGAINST ('excellent ' IN BOOLEAN MODE) AS relevance  FROM descriptions_programs USE INDEX (text)  LEFT JOIN programs ON programs.id=object_id  WHERE object_id IN (     SELECT id FROM programs      WHERE 1 AND id IN (         SELECT id FROM programs          WHERE submitter_id=32 AND id in (             SELECT id FROM programs              WHERE feed_id=2478 AND id in (                 SELECT program_id FROM playlist_program_map                  WHERE playlist_id=181))))  AND MATCH(text) AGAINST ('excellent ' IN BOOLEAN MODE)>0 AND current=1 )   UNION (  SELECT object_id, programs.created AS created,  MATCH(text) AGAINST ('excellent ' IN BOOLEAN MODE) AS relevance  FROM titles_programs USE INDEX (text)  LEFT JOIN programs ON programs.id=object_id  WHERE object_id IN (     SELECT id FROM programs      WHERE 1 AND id IN (         SELECT id FROM programs          WHERE submitter_id=32 AND id in (             SELECT id FROM programs              WHERE feed_id=2478 AND id in (                 SELECT program_id FROM playlist_program_map                  WHERE playlist_id=181))))  AND MATCH(text) AGAINST ('excellent ' IN BOOLEAN MODE)>0 AND current=1; 

Answer 1

See what EXPLAIN EXTENDED says.

If it says DEPENDENT SUBQUERY or UNCACHEABLE SUBQUERY, then it will be reevaluated each time it’s used.

This happens if the subquery uses session variables or is a correlated subquery.

If it doesn’t, it most probably will be cached.

If your case the subquery will not be cached, it will be reevaluated in each UNION‘ed set.

You subquery, though, seems to be too complicated. Why don’t you just use:

SELECT id FROM   playlist_program_map ppm, programs p WHERE  ppm.playlist_id = 181        AND p.id = ppm.program_id        AND submitter_id = 32        AND feed_id = 2478 

If you have an index on playlist_program_map (playlist_id), this query should work like a charm.

Could you please tell me two more things:

1. How many rows are there in playlist_program_map and how many DISTINCT playlist_id values are there?
   • How many rows are there in programs and how many DISTINCT submitter_id, feed_id pairs are there?

From your comment I can conclude that there are 10 programs per playlist in average, and 200 programs per (submitter, feed) pair. This means your index on playlist_program_map is more selective than the one on (submitter, feed), and playlist_program_map must be leading in the join.

The fulltext index in your case also doesn’t seem to be very selective, given that you need to join 10 programs out of 2,000,000.

You may better try the following:

SELECT object_id, programs.created AS created FROM   playlist_program_map ppm, programs p, comments_programs cp WHERE  ppm.playlist_id = 181        AND p.id = ppm.program_id        AND p.submitter_id = 32        AND p.feed_id = 2478        AND cp.object_id = p.id        AND cp.text REGEXP 'excellent' 

, and repeat this for all three tables.