Question

I am trying to understand a simple python proxy example using Twisted located here. The proxy instantiates a Server Class, which in turn instantiates a client class. defer.DeferredQueue() is used to pass data from client class to server class.

I am now trying to understand how defer.DeferredQueue() works in this example. For example what is the significance of this statement:

self.srv_queue.get().addCallback(self.clientDataReceived)

and it’s analogous

self.cli_queue.get().addCallback(self.serverDataReceived)

statement.

What happens when self.cli_queue.put(False) or self.cli_queue = None is executed?

Just trying to get into grips with Twisted now, so things seems pretty daunting. A small explanation of how things are connected would make it far more easy to get into grips with this.

Answer 1

According to the documentation, DeferredQueue has a normal put method to add object to queue and a deferred get method.

The get method returns a Deferred object. You add a callback method (e.g serverDataReceived) to the object. Whenever the object available in the queue, the Deferred object will invoke the callback method. The object will be passed as argument to the method. In case the queue is empty or the serverDataReceived method hasn’t finished executing, your program still continues to execute next statements. When new object available in the queue, the callback method will be called regardless of the point of execution of your program.

In other words, it is an asynchronous flow, in contrary to a synchronous flow model, in which, you might have a BlockingQueue, i.e, your program will wait until the next object available in the queue for it to continue executing.

In your example program self.cli_queue.put(False) add a False object to the queue. It is a sort of flag to tell the ProxyClient thread that there won’t be anymore data added to the queue. So that it should disconnect the remote connection. You can refer to this portion of code:

def serverDataReceived(self, chunk):
        if chunk is False:
            self.cli_queue = None
            log.msg("Client: disconnecting from peer")
            self.factory.continueTrying = False
            self.transport.loseConnection()

Set the cli_queue = None is just to discard the queue after the connection is closed.