Question

I am trying to get all the entries in a session log table where a session has more than 10 entries (i.e. the count of the session_id is greater than 10). What I have right now are two select statements:

select * from log_metrics where session_id in 
( select session_id from log_metrics 
  group by session_id having count(*) > 10
)

The log_metrics table is quite large, aprox. 7,700,000 rows. The inner select takes 12.88 seconds and finds 178,000 session id’s. The whole query doesn’t finish running written like this, but when adding limit 10 to the end of the outer select it completes in 18 seconds, limit 100 completes in 3 min 35 sec. I tried adding the limit to the inner select but got the following error:

ERROR 1235 (42000): This version of MySQL doesn’t yet support ‘LIMIT & IN/ALL/ANY/SOME subquery’

Is there a way to rewrite this query to speed things up? I only need to get about 5,000 rows from log_metrics returned, not the total 178,000 of the session id’s.

Thanks for any help you might be able to give. I am new to mysql and posting so pardon any etiquette mis-steps.

Answer 1

I have no idea if this will work (I don’t know what version of mySQL you have, and I don’t have an instance regardless), but would using a JOIN work as you want?

SELECT *
FROM log_metrics a
JOIN (SELECT session_id
      FROM log_metrics
      GROUP BY session_id
      HAVING COUNT(session_id) > 10
      LIMIT 5000) b
ON b.session_id = a.session_id

You didn’t mention this, but for future questioners, the reason he needs the LIMIT statement inside the inner query is because he wants (a maximum of) 5000 session_ids, not total rows from the log (which could be 50,000 rows or more returned).