Question

I have to make some statistics for my application, so I need an algorithm with a performance as best as possible. I have some several question.

I have a data structure like this in the mysql database:

user_id    group_id     date
1          5            2012-11-20
1          2            2012-11-01
1          4            2012-11-01
1          3            2012-10-15
1          9            2013-01-18
...

So I need to find the group of some user at a specific date. For example, the group of the user 1 at date 2012-11-15 (15 november 2012) should return the most recent group, which is 2 and 4 (many group at the same time) at date 2012-11-01 (the closest and smaller date).

Normally, I could do a Select where date <= chosen date order by date desc, etc… but that’s not the point because if I have 1000 users, it will need 1000 requests to have all the result.

So here are some question:

1. I have already using the php method to loop through the array to avoid the high number of mysql request, but it’s still not good because the array size may be 10000+. Using a foreach (or for?) is quite costly.
   So my question is if given an array, ordered by date (desc or asc), what’s the fastest way to find the closest index of the element which contain a date smaller (or greater) than a given date; beside using a for or foreach loop to loop through each element.
2. If there is no solution for the first question, then what kind of data structure would you suggest for this kind of problem.

Note: the date is in mysql format, it’s not converted in timestamp when you stored it in an array

EDIT: this is a sql fiddle http://sqlfiddle.com/#!2/dc28d/1
For dos_id = 6, t=”2012-11-01″ it should returns only 2 and 5 at date “2010-12-10 13:16:58”

Answer 1

This would give you a list of all users and their groups (1 row per group) for the latest date that is smaller than the one you specify (2012-11-15 below).

SELECT user_id, group_id, date FROM table WHERE date <= '2012-11-15' AND NOT EXISTS (SELECT 1 FROM table test WHERE test.user_id = table.user_id AND test.date > table.date and test.date <= '2012-11-15')