Question

Various connections – e.g. those created with twisted.web.client.getPage() seem to leak – they hang around indefinitely, since the OS time-out is measured in hours – if the server doesn’t respond timely. And putting a time-out on the deferred you get back is deprecated.

How can you track the requests you have open, and close them forcefully in your twisted program?

(Forcefully closing connections that have timed-out in application logic is important to making a twisted server that scales; various reactors have different limits on the number of open file descriptors they allow – select being as low as 1024! So please help twisted users keep the open connections count nice and trimmed.)

Answer 1

getPage accepts a timeout parameter. If you pass a value for it and the response is not fully received within that number of seconds, the connection will be closed and the Deferred returned by getPage will errback.