Question

I have a django site. Certain actions by the end user send email to the rest of users in a group.

When the number of users gets to be > 20 it can add 1-3 seconds to the request cycle, which I don’t like. I’d like to be able to send the email from a non-blocking function.

I know RabbitMQ and Celery in conjunction can solve this, but with 200 users that seems like over-engineering and it adds two more applications I have to install, understand, and babysit.

I’ve done some research, and it appears that both threading.Thread and subprocess would be ways to wrap a non-blocking call. Am I missing an obvious way to do this? Are there downsides to using either the threading.thread or subprocess approach?

Thanks,
Ted

Answer 1

Offloading the work to some other external process is really the right thing to do, and once you’ve done it, it’s not likely to be the last time you do it. Celery/RabbitMQ is a decent solution, and the nice thing is they’re already there. Recent RabbitMQ releases have a decent web-based management app and a decent management API that’ll make babysitting pretty easy, and celery works pretty well in Django apps.

You can do this with subprocess or threading, but to be honest, I think that’s a bad habit to get into. Unfortunately, they’re the most straightforward ways to do what you want to do if you don’t want to offload things.

If you wanted to go completely ‘ghetto async email’ you could have your app just dump emails to files in a directory and have a cron job check the directory for files in that directory every minute, and send them off as emails, but really that’s a lot more work than rabbit/celery.

I say just go with rabbit/celery. It’s not as much work as it seems, and it’s worth it going forward.