Question

Some background. I’m using Rails 3 for a carpooling application. I’m using fullcalendar as the jquery calendar library. It doesn’t work too nicely when there are lots of events for a particular day. So, what I want to do, is to have two “eventsources”. In one, I want to return all the events, grouped by date and category, that have a count less than some value.
For other, I want to return all the records which have a count more than the value.

Apologies if this isn’t clear, some sample data might make it more obvious

Basic Record Structure:

|    id    |  date    |  category_id |
|    1     | 1-Aug    |       1      |
|    2     | 1-Aug    |       1      |
|    3     | 1-Aug    |       1      |
|    4     | 1-Aug    |       2      |
|    5     | 1-Aug    |       2      |
|    6     | 1-Aug    |       3      |

Assuming 3 is the magic count number (3 or higher should be grouped) I’d like one select to return

|    id    |  date    |  category_id |   count  |
|    1     | 1-Aug    |       1      |     3    |

(I don’t actually care that the ID could be combined (with MYSQL) using a group_concat), in my use case it’s not important – what I want to know is that on 1st August for category 1 there are 3 entries

The second select should return everything else

|    id    |  date    |  category_id |   count  |
|    4     | 1-Aug    |       2      |     2    |
|    5     | 1-Aug    |       2      |     2    |
|    6     | 1-Aug    |       3      |     1    |

I don’t really need the count returned, but the point is that there are individual rows for every result which doesn’t have a count >= 3.

Currently my SQL looks like this:

SELECT `pools`.*
FROM       `pools`
INNER JOIN `fields`    ON `fields`.`id`    = `pools`.`field_id` 
INNER JOIN `regions`   ON `regions`.`id`   = `fields`.`region_id` 
INNER JOIN `countries` ON `countries`.`id` = `regions`.`country_id`
WHERE `regions`.`country_id` = 1
  AND `pools`.`confirmed`    = 1
  AND (leaving_date >= '2012-04-30 00:00:00')
  AND (leaving_date <= '2012-06-04 00:00:00')
GROUP BY field_id, leaving_date
HAVING   count(*) >= 3
ORDER BY leaving_date

I could iterate through the arrays returned – but I’d rather do it SQL side if it’s possible. Would also like to do it in the least number of database trips.. A general pointer would be really appreciated!

Answer 1

Maybe this is useful for starting:

CREATE TEMPORARY TABLE grouped_rows AS
  SELECT `pools`.*
  FROM       `pools`
  INNER JOIN `fields`     ON `fields`.`id`    = `pools`.`field_id` 
  INNER JOIN `regions`    ON `regions`.`id`   = `fields`.`region_id` 
  INNER JOIN `countries`  ON `countries`.`id` = `regions`.`country_id`
  WHERE `regions`.`country_id` = 1
    AND `pools`.`confirmed`    = 1
    AND (leaving_date >= '2012-04-30 00:00:00')
    AND (leaving_date <= '2012-06-04 00:00:00')
  GROUP BY field_id, leaving_date;
  HAVING   count(*) >= 3;


SELECT `pools`.*
FROM       `pools`
INNER JOIN `fields`     ON `fields`.`id`    = `pools`.`field_id` 
INNER JOIN `regions`    ON `regions`.`id`   = `fields`.`region_id` 
INNER JOIN `countries`  ON `countries`.`id` = `regions`.`country_id`
LEFT  JOIN grouped_rows ON grouped_rows.field_id     = pools.field_id
                       AND grouped_rows.leaving_date = pools.leaving_date
WHERE `regions`.`country_id` = 1
  AND `pools`.`confirmed`    = 1
  AND (leaving_date >= '2012-04-30 00:00:00')
  AND (leaving_date <= '2012-06-04 00:00:00')
  AND grouped_rows.some_not_null_field IS NULL

UNION ALL

SELECT * FROM grouped_rows

ORDER BY leaving_date;


DROP TEMPORARY TABLE grouped_rows;

I don’t know if you can issue all three statements and get the results of the UNION in one roundtrip.
Maybe you could embed the temptable’s select into both halves of the UNION, but it may give some more work to the MySQL server (or not).

Update:

I found a one-query solution that you can use. This is just the basic skeleton structure, but I think you can convert your query according to it:

http://sqlfiddle.com/#!2/f67e3/18

SELECT
  IF( grouped.gcat, NULL, item.id ) AS out_id
 ,item.cat AS out_cat
 ,COALESCE( grouped.gcount, 1 ) out_count

FROM item

LEFT JOIN
(
  SELECT   count(*) AS gcount, cat AS gcat
  FROM     item
  GROUP BY gcat
  HAVING   count(*) >= 3
) AS grouped
  ON item.cat = grouped.gcat

GROUP BY out_id