Question

I have a site with about 30,000 members to which I’m adding a functionality that involves sending a random message from a pool of 40 possible messages. Members can never receive the same message twice.

One table contains the 40 messages and another table maps the many-to-many relationship between messages and members.

A cron script runs daily, selects a member from the 30,000, selects a message from the 40 and then checks to see if this message has been sent to this user before. If not, it sends the message. If yes, it runs the query again until it finds a message that has not yet been received by this member.

What I’m worried about now is that this m-m table will become very big: at 30,000 members and 40 messages we already have 1.2 million rows through which we have to search to find a message that has not yet been sent.

Is this a case for denormalisation? In the members table I could add 40 columns (message_1, message_2 … message_40) in which a 1 flag is added each time a message is sent. If I’m not mistaken, this would make the queries in the cron script run much faster

?

Answer 1

I know that doesn’t answer your original question, but wouldn’t it be way faster if you selected all the messages that weren’t yet sent to a user and then select one of those randomly?

See this pseudo-mysql here:

SELECT 
    CONCAT_WS(',', messages.ids) unsent_messages, 
    user.id user
FROM
    messages,
    user
WHERE
    messages.id NOT IN (
        SELECT 
            id 
        FROM 
            sent_messages 
        WHERE 
            user.id = sent_messages.user
    )
GROUP BY ids