I am implementing a library module to serve as client for couchdb change notifications. I want to have the library in “continuous” mode, that is, the connection must remain open forever, or at least it must reconnect if the connection has been closed, so that couchdb has a channel to notify of any new changes happening in the database. I will then process those notifications to generate certain events (this is not yet implemented).
The approach that I have chosen is to use the ReconnectingClientFactory (which does automatic re-connection as per an elaborated algorithm) as basis for my Protocol Factory. Whenever the connection has been established, the buildProtocol method is called. In this method I create the protocol instance, and fire a callLater (immediately) to signal that the connection is ready. In the cdConnected function, I send the request and add a callback to process the received data (cbReceived).
The code does reconnection as expected, but I am having two different problems:
- the request is failing (no data is sent over the tcp connection), but I do not know why.
- an error is generated, even though the connection is closed cleanly.
Maybe somebody has an idea of what I am doing wrong?
Thanks!
(edit: the “Connection was closed cleanly.” error is being printed by myself, so it can be ignored)
Here is the code:
from twisted.internet import defer
from twisted.internet.protocol import ReconnectingClientFactory
from twisted.web._newclient import HTTP11ClientProtocol
from twisted.web._newclient import Request
from twisted.web.client import _parse
class MyReconnectingClientFactory(ReconnectingClientFactory):
def __init__(self, reactor, cbConnected):
self.reactor = reactor
self.cbConnected = cbConnected
def startedConnecting(self, connector):
print 'Started to connect ...'
def buildProtocol(self, addr):
print 'Resetting reconnection delay'
self.resetDelay()
proto = HTTP11ClientProtocol()
self.reactor.callLater(0, self.cbConnected, proto)
return proto
def clientConnectionLost(self, connector, reason):
print 'Lost connection. Reason:', reason
ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
def clientConnectionFailed(self, connector, reason):
print 'Connection failed. Reason:', reason
ReconnectingClientFactory.clientConnectionFailed(self, connector, reason)
def cbReceived(response):
print response
def printError(failure):
print "printError > %s" % (str(failure))
def cbConnected(proto):
print "Sending request ..."
req = Request(method, path, headers, bodyProducer)
d = proto.request(req)
d.addCallback(cbReceived).addErrback(printError)
return d
from twisted.internet import reactor
uri='http://localhost:5984/cn/_changes?feed=continuous'
method='GET'
headers=None
bodyProducer=None
scheme, host, port, path = _parse(uri)
factory = MyReconnectingClientFactory(reactor, cbConnected)
reactor.connectTCP(host, port, factory)
reactor.run()
And here is the output:
Started to connect ...
Resetting reconnection delay
Sending request ...
printError > [Failure instance: Traceback (failure with no frames): <class 'twisted.web._newclient.RequestGenerationFailed'>: [<twisted.python.failure.Failure <type 'exceptions.AttributeError'>>]
]
Lost connection. Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.
]
Started to connect ...
Resetting reconnection delay
Sending request ...
printError > [Failure instance: Traceback (failure with no frames): <class 'twisted.web._newclient.RequestGenerationFailed'>: [<twisted.python.failure.Failure <type 'exceptions.AttributeError'>>]
]
Lost connection. Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.
]
You should take a look at the failures generated. _newclient has a habit of throwing compound failures; these contain further failures.
I adapted your code a little:
You were doing a str() on failure and that’s not a good way to get information about an exception. Let repr take care of it, that’s its job.
Using %r, I saw that it was actually giving me a RequestGenerationFailed. That’s a more interesting failure. The failure value has reasons.
With my modification, the script gave this:
That should give you a good clue on where to look for your actual problem.
Incidentally, take a look at Paisley, a Twisted client for CouchDB: paisley
There are a few branches, and in particular my changes branch has some change notification stuff you may be interested in. I made a desktop applet that showed me tasks added to my CouchDB-based todo system.
It sounds like your changes are a) already in there or b) should go in there; and c) you should consider working with and contributing to Paisley.
Good luck!