Question

In my Django project I have to do repeatedly some processing in the background.
This processing needs access to Django stuff, so I put it into Django’s commands and run it as cronjob.
Right now I realize, that I have to do some of them more frequently (cronjob has limitation to invoke command at most every 1 minute). Another problem is that I don’t have enough control, to protect running the same command in one time. It’s happen when one processing takes longer than one minute.
I think that I should run them like daemons, but I am looking for pure way to do it with Django.
Have you ever faced with this problem or know any clean solution for it?

Answer 1

We do a lot of background processing for django using Celery http://celeryproject.org/. It requires some effort to set up and there is a bit of a learning curve, but once it’s up and running it’s just awesome.