Question

Question is relevant to this and this;

the difference is, I’d prefer something with possibly more precision and low load (per-minute cron job isn’t preferable for those) and with minimal overhead (i.e. installing celery with rabbitmq seems like a big overkill).

An example task for such is personal reminders server (with reminders that could be edited over web and sent out through e-mail or XMPP).

I’m probably looking for something more like node.js’s setTimeout but for django (and though I might prefer to implement reminders in node.js anyway, it’s still a possibly interesting question).

For example, it’s possible to start new threads in django app (with functions consisting of sleep() and send()); in what ways this can be bad?

Answer 1

The problem with using threads for this solution are the typical problems with Python threads that always drive people towards multi-process solutions instead. The problem is compounded here by the fact your thread isn’t driven by the normal request-response cycle. This is summarized nicely by Malcolm Tredinnick here:

Have to disagree. Threads are not a good solution to this problem. The
issue is process management. As written, your threads will never be
rejoined. Webserver processes have a lifecycle uncontrollable by you
(the MaxRequestsPerChild Apache parameter and similar things in other
servers) and you are messing with that by using threads.

If you need a process with a lifecycle that is not matched by the
request-response path — something long running and independent of the
response — a completely separate process is definitely the right model
to use. Using a thread is tying it to the response lifecycle, which
wil have unintended side-effects.

A possible solution for you might be to have a long running process performing your tasks which gets a wake-up signal from a light cron process.

Another possibility would be build something using 0mq, which is much lighter than AMQP style queues (at the cost of some features of course). Tarek Ziade is working on a Mozilla project called powerhose that uses 0mq, looks super simple, and has a heartbeat capability with resolution to the second.