This is my first project using Tornado… building a message service
with TornadIO + ZMQ. I’m using pyzmq’s ioloop.
I have been running into this issue of a recursive loop between the
iostream and the websocket, and can’t yet pinpoint why. Seems a client
connects fine and sends plenty of messages and receives fine. Someone
else connects and is fine for a bit then crash. I dont know if I’m not
handling a disconnect properly, or if something in a message is
causing a read error in the tornado code. All that is really happening in my server is that a client connects and sends messages, which then just get broadcasted to everyone else that is also connected via the websockets.
Maybe someone can tell me whats really happening here and if its something I’m not handling
properly…
// Start of the traceback…
[E 110426 01:45:28 ioloop:295] Exception in I/O handler for fd 22
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/pyzmq-2.1.4-py2.6-linux-x86_64.egg/zmq/eventloop/ioloop.py", line 282, in start
self._handlers[fd](fd, events)
File "/usr/local/lib/python2.6/dist-packages/pyzmq-2.1.4-py2.6-linux-x86_64.egg/zmq/eventloop/stack_context.py", line 133, in wrapped
callback(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 199, in _handle_events
self._handle_read()
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 258, in _handle_read
if self._read_from_buffer():
// START: block that loops recursively
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 325, in _read_from_buffer
self._consume(loc + delimiter_len))
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 230, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/stack_context.py", line 173, in wrapped
callback(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/websocket.py", line 193, in _on_end_delimiter
self._receive_message()
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/websocket.py", line 178, in _receive_message
self.stream.read_bytes(1, self._on_frame_type)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 149, in read_bytes
if self._read_from_buffer():
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 314, in _read_from_buffer
self._run_callback(callback, self._consume(num_bytes))
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 230, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/stack_context.py", line 173, in wrapped
callback(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/websocket.py", line 183, in _on_frame_type
self.stream.read_until("\xff", self._on_end_delimiter)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/iostream.py", line 133, in read_until
if self._read_from_buffer():
// Loop back to start (line 325 in iostream)
Any help??????
Im using the pypi tornado
1.2.1, tornadIO 0.0.4, zeromq 2.1.4, and pypi pyzmq, on ubuntu linux
Edit:
It seems to originate from this snippet in my on_close() handler:
if self._client_sub and self._client_sub.stream:
try:
self._client_sub.stream.stop_on_recv()
self._client_sub.stream.stop_on_err()
except Exception, e:
logging.error("Failed to close stream and sub socket on client disconnect", exc_info=True)
and here is the traceback:
[E 110425 20:52:50 messgr:202] Failed to close stream and sub socket on client disconnect
Traceback (most recent call last):
File "messgr.py", line 198, in on_close
try:
AttributeError: 'NoneType' object has no attribute 'stream'
Traceback (most recent call last):
File "/usr/lib/python2.6/logging/init.py", line 776, in emit
msg = self.format(record)
File "/usr/lib/python2.6/logging/init.py", line 654, in format
return fmt.format(record)
File "/usr/local/lib/python2.6/dist-packages/tornado-1.2.1-py2.6.egg/tornado/options.py", line 362, in format
record.exc_text = self.formatException(record.exc_info)
File "/usr/lib/python2.6/logging/init.py", line 416, in formatException
traceback.print_exception(ei[0], ei[1], ei[2], None, sio)
File "/usr/lib/python2.6/traceback.py", line 125, in print_exception
print_tb(tb, limit, file)
File "/usr/lib/python2.6/traceback.py", line 69, in print_tb
line = linecache.getline(filename, lineno, f.f_globals)
File "/usr/lib/python2.6/linecache.py", line 14, in getline
lines = getlines(filename, module_globals)
File "/usr/lib/python2.6/linecache.py", line 40, in getlines
return updatecache(filename, module_globals)
What happens is that traceback happens one. Then it seems to stack, and happen twice, and then 3 times…endlessly.
Seems very similar to this issue