Question

I made a email queue in a database table(MySQL). Each row is an email message with recipient, sender, etc. And a script is automatically called every minute and it retrieves unsent messages and send them and set the ‘sent’ column of each message to true.

I need to ensure that each message is sent only once. But if the script runs long, another script might retrieve the messages and send them without knowing that they will be sent by another script.

Currently, I limit the number of rows to 50 at a time in order to keep the script running time short. But this doesn’t guarantee. I am sure there’s a better way.

Thanks.

Sam

Answer 1

Building a reliable job queue is not easy. You definitively need to store more information than “sent/not sent”. The bare minimum I can think of is “Date taken” and “Date sent”. The delivery process should:

1. Pick some available messages:

   START TRANSACTION;
   
   SELECT message_id
   FROM message
   WHERE date_taken IS NULL AND date_sent IS NULL
   ORDER BY message_id
   LIMIT 5
   FOR UPDATE;
   

2. Tag them as processing:

   UPDATE message
   SET date_taken=NOW()
   WHERE message_id IN (101, 102, 103, 104, 105);
   
   COMMIT;
   

3. Once done, tag them as sent:

   UPDATE message
   SET date_sent=NOW()
   WHERE message_id IN (101, 102, 103, 104, 105);
   

You’d also need to release messages that have been taken for too long (since the process that took them probably crashed).

It this is too much of a hassle, there’re some third-party mail queries like PEAR’s Mail_Queue.