python 2.6
Windows 7
I am trying to put together an as simple as possible tutorial of how to write cooperative multitasking programs. As an example application I’ve written a chat server with python’s asyncore backend. I think this will be a valuable resource for the community. However, I have not gotten it to work yet, hence this post.
The structure is as follows. An instance of ChatServer runs on a remote computer. It’s socket listens on REMOTE_PORT. When it detects an incoming connection, it spawns an instance of ChatHandler to mediate communication with that connection. Now, who is that connection? On the user’s local machine we run an instance of ChatDaemon. This guy listens on LOCAL_PORT. When you connect to him like this
import socket
s = socket.socket()
s.connect(('localhost',LOCAL_PORT))
he detects the connection and spawns two things, a LocalListener and a Connection. The Connection connects to the server, answering our question from above. The LocalListener simply waits for data coming in from the user. If you send data
s.send("Hello, world!")
the LocalListener picks it up, and gives it to the Connection, which then sends it to the ChatServer. The server then puts the data in each ChatHandler’s buffer to be sent out to all connected clients. When the Connection receives that data, it passes it to the Daemon who prints it to the screen.
(The Daemon layer seems overly complex but without it you have to do other complicated things to prevent hot loops in asyncore’s select() loop whilst keeping latency for the user sending data low. I don’t want to go down that road.)
The problem is that the connection to the Daemon does not seem to be made. My exact steps are
In one python session
d = ChatDaemon('localhost')
d.start()
When I do this I see the message “Chat daemon binding to ‘localhost: 7668’ as expected.
In another python session
import socket
s = socket.socket()
s.connect(('localhost',7668))
When I do this, I do not see the “Got new local connection” line printed.
I have edited my etc/hosts file to map ‘localhost’ to 127.0.0.1, and I installed the Microsoft Loopback adapter.
EDIT: I have found and fixed the problem. The code below should now be acceptable as a very simple chat implementation using asyncore.
Here is the source
import asyncore
import socket
import sys
LOCAL_HOST = 'localhost'
LOCAL_PORT = 7668
REMOTE_HOST = 'localhost'
REMOTE_PORT = 7667
class LocalListener(asyncore.dispatcher):
"""Receive data from user, putting into cxn's buffer"""
def __init__(self, sock, cxn):
self.cxn = cxn
asyncore.dispatcher.__init__(self, sock)
def writable(self):
return False
def readable(self):
return True
def handle_read(self):
data = self.recv(4096)
if data:
self.cxn.buf = self.cxn.buf + data
class Connection(asyncore.dispatcher):
"""Mediates between user and server"""
def __init__(self, host, port, master):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.connect((host,port))
self.buf=""
def writable(self):
return len(self.buf) > 0
def readable(self):
return True
def handle_read(self):
data = self.recv(4096)
if data:
self.master.newMessage(data)
def handle_write(self):
sent = self.send(self.buf)
self.buf = self.buf[sent:]
class ChatDaemon(asyncore.dispatcher):
"""Listen for local connections and dispatch in/out data"""
ADDRESS_FAMILY = socket.AF_INET
SOCKET_TYPE = socket.SOCK_STREAM
def __init__(self, remoteHost, remotePort=REMOTE_PORT,
localHost=LOCAL_HOST, localPort=LOCAL_PORT):
self.remoteHost = remoteHost
self.remotePort = remotePort
self.localHost = localHost
self.localPort = localPort
self.buffer = ""
asyncore.dispatcher.__init__(self)
def writable(self):
return False
def readable(self):
return True
def newMessage(self, data):
print data
def start(self):
"""Listen for user connection on local port"""
self.create_socket(self.ADDRESS_FAMILY, self.SOCKET_TYPE)
print("Chat deamon binding to '%s': %s"%(self.localHost,self.localPort))
self.bind((self.localHost,self.localPort))
self.listen(1)
asyncore.loop()
def handle_accept(self):
"""Spawn local reader and remote reader/writer"""
print "Got new local connection"
(connSock, localAddress) = self.accept()
print("New connection address is %s"%localAddress)
#Make a server connection
cxn = Connection(self.remoteHost, self.remotePort, self)
#Connect to local user
LocalListener(connSock, cxn)
### SERVER ###
class ChatHandler(asyncore.dispatcher):
def __init__(self, sock, map, server):
self.server = server
self.buffer = ''
asyncore.dispatcher.__init__(self, sock, map)
def writable(self):
return len(self.buffer) > 0
def readable(self):
return True
def handle_read(self):
"""Notify server of any new incoming data"""
data = self.recv(4096)
if data:
self.server.newMessage(data, self)
def handle_write(self):
"""send some amount of buffer"""
sent = self.send(self.buffer)
self.buffer = self.buffer[sent:]
class ChatServer(asyncore.dispatcher):
"""Receive and forward chat messages
When a new connection is made we spawn a dispatcher for that
connection.
"""
ADDRESS_FAMILY = socket.AF_INET
SOCKET_TYPE = socket.SOCK_STREAM
def __init__(self, host=REMOTE_HOST, port=REMOTE_PORT):
self.map = {}
self.address = (host,port)
self.clients = []
asyncore.dispatcher.__init__(self, map=self.map)
def serve(self):
"""Bind to socket and start asynchronous loop"""
self.create_socket(self.ADDRESS_FAMILY, self.SOCKET_TYPE)
self.bind(self.address)
self.listen(1)
asyncore.loop(map=self.map)
def writable(self):
return False
def readable(self):
return True
def newMessage(self, data, fromWho):
"""Put data in all clients' buffers"""
for client in self.clients:
client.buf = client.buf + data
def handle_accept(self):
"""Deal with newly accepted connection"""
print 'got new connection'
(connSock, clientAddress) = self.accept()
self.clients.append(ChatHandler(connSock, self.map, self))
The problem was that in ChatDaemon I forget the “return” keywords in readable and writable