Question

how does one add deferreds after a reactor is started?

For example how do I realise the following workflow?

1. get a list of webpages & extract data
2. based on this data, get an unknown number of other websites

At present I do the following:

1. define list & functions for point 1 above and create a deferreds
2. run reactor
3. in the functions processing the responses from 1, create other deferreds for point 2 above. Unfortunately these never get called…

Many thanks.

Answer 1

The language “add Deferreds” suggests you misunderstand the purpose of a Deferred. A Deferred is just a convenient API for associating callbacks with long-running asynchronous operations. Deferreds are not gathered in any one collection or tracked by the reactor. They’re a common object tying a piece of code providing some result to another piece of code consuming that result.

That said, it sounds like you’re doing roughly the correct thing. There is no difference whatsoever in how Deferreds work when the reactor is running compared to when it is not running. This is because the reactor plays no part in the operation of Deferreds.

If you have callbacks that aren’t being called, it’s because the Deferreds they are attached to aren’t getting results. There’s no way to say why that’s happening without seeing at least some of your code (and preferably a short, self-contained, correct example).