Question

I run some task using eventlet.GreenPool.spawn, then wait for all greanthreads to finish. I know there would be an exception raised – how can I catch that exception and throw it in a main thread? I am pretty sure it’s easy however I’m totally missing something.

Here is an example (which fails and I would like it to succeed)

import unittest
import eventlet


def broken_fetch(url):
    print " Raising exception "
    raise RuntimeError


class TestPool(unittest.TestCase):

    def test_error_is_bubbled_up(self):
        with self.assertRaises(RuntimeError):
            pool = eventlet.GreenPool(100)
            urls = ['http://google.com/', 'http://example.com/']
            for url in urls:
                pool.spawn(broken_fetch, url)
            pool.waitall()

if __name__ == '__main__':
    unittest.main()

and it’s output:

> python errors.py
Raising exception 
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in fire_timers
    timer()
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
File "/usr/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
File "errors.py", line 10, in broken_fetch
    raise RuntimeError
RuntimeError
Raising exception 
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in fire_timers
    timer()
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
File "/usr/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
File "errors.py", line 10, in broken_fetch
    raise RuntimeError
RuntimeError
F
======================================================================
FAIL: test_error_is_bubbled_up (__main__.TestPool)
----------------------------------------------------------------------
Traceback (most recent call last):
File "errors.py", line 21, in test_error_is_bubbled_up
    pool.waitall()
AssertionError: RuntimeError not raised

----------------------------------------------------------------------
Ran 1 test in 0.003s

FAILED (failures=1)

Answer 1

Eventlet’s event loop swallows all exceptions, barring KeyboardInterrupt and SystemExit. Check the wait() implementation of various eventlet hubs. The following excerpt is from select hub’s wait():

for listeners, events in ((readers, r), (writers, w)):
    for fileno in events:
        try:
            listeners.get(fileno, noop).cb(fileno)
        except self.SYSTEM_EXCEPTIONS:
            raise
        except:
            self.squelch_exception(fileno, sys.exc_info())
            clear_sys_exc_info()

To work around this, you can pass the exception info as value and deal with it later in the main thread.

import unittest2
import sys
import eventlet


def broken_fetch(url):
    print " Raising exception "
    try:
        raise RuntimeError
    except:
        return sys.exc_info()

class TestPool(unittest2.TestCase):

    def test_error_is_bubbled_up(self):
        with self.assertRaises(RuntimeError):
            pool = eventlet.GreenPool(100)
            urls = ['http://google.com/', 'http://example.com/']
            for exc_info in pool.imap(broken_fetch, urls):
                if exc_info is not None:
                    exc_class, value, tb = exc_info
                    raise exc_class, value, tb

if __name__ == '__main__':
    unittest2.main()

You may want to turn off eventlet’s DEBUG flag to prevent eventlet from printing the swallowed exception. When you want to handle the exception in main thread, you probably don’t want to get confused by the duplicated traceback print.

import eventlet.debug
eventlet.debug.hub_exceptions(False)