Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I have a series of maintenance tasks for a python WSGI application that are a bit too complex for a crontab (jobs need to be run at frequencies derived from the size of the job queue, manage a connection pool to a group of EC2 instances, etc).
How should I implement a long-running, event-driven python program? I’ve never needed this functionality before, so I’m not even sure what to google.
Most of the large, modern python sites are using Celery for this type of work. It is a distributed task queue that supports scheduling of tasks as well.
Though probably a bit heavyweight for a small site, it’ll grow with you. I’m looking to implement it myself (sans Rabbit) shortly.
I recently found another choice for django users, django-tasks which is focused on fewer, longer, batch processing type jobs. There is also django-ztask using zeromq.
Addendum: Just came across gearman which has python bindings.