Question

So I created the following file (testlib.py) to automatically load all doctests (throughout my nested project directories) into the __tests__ dictionary of tests.py:

# ./testlib.py
import os, imp, re, inspect
from django.contrib.admin import site

def get_module_list(start):
    all_files = os.walk(start)
    file_list = [(i[0], (i[1], i[2])) for i in all_files]
    file_dict = dict(file_list)

    curr = start
    modules = []
    pathlist = []
    pathstack = [[start]]

    while pathstack is not None:

        current_level = pathstack[len(pathstack)-1]
        if len(current_level) == 0:
            pathstack.pop()

            if len(pathlist) == 0:
                break
            pathlist.pop()
            continue
        pathlist.append(current_level.pop())
        curr = os.sep.join(pathlist)

        local_files = []
        for f in file_dict[curr][1]:
            if f.endswith(".py") and os.path.basename(f) not in ('tests.py', 'models.py'):
                local_file = re.sub('\.py$', '', f)
                local_files.append(local_file)

        for f in local_files:
            # This is necessary because some of the imports are repopulating the registry, causing errors to be raised
            site._registry.clear()
            module = imp.load_module(f, *imp.find_module(f, [curr]))
            modules.append(module)

        pathstack.append([sub_dir for sub_dir in file_dict[curr][0] if sub_dir[0] != '.'])

    return modules

def get_doc_objs(module):
    ret_val = []
    for obj_name in dir(module):
        obj = getattr(module, obj_name)
        if callable(obj):
            ret_val.append(obj_name)
        if inspect.isclass(obj):
            ret_val.append(obj_name)

    return ret_val

def has_doctest(docstring):
    return ">>>" in docstring

def get_test_dict(package, locals):
    test_dict = {}
    for module in get_module_list(os.path.dirname(package.__file__)):
        for method in get_doc_objs(module):
            docstring = str(getattr(module, method).__doc__)
            if has_doctest(docstring):

                print "Found doctests(s) " + module.__name__ + '.' + method

                # import the method itself, so doctest can find it
                _temp = __import__(module.__name__, globals(), locals, [method])
                locals[method] = getattr(_temp, method)

                # Django looks in __test__ for doctests to run. Some extra information is
                # added to the dictionary key, because otherwise the info would be hidden.
                test_dict[method + "@" + module.__file__] = getattr(module, method)

    return test_dict

To give credit where credit is due, much of this came from here

In my tests.py file, I have the following code:

# ./project/tests.py
import testlib, project
__test__ = testlib.get_test_dict(project, locals())

All of this works quite well to load my doctests from all of my files and subdirectories. The problem is that when I import and invoke pdb.set_trace() anywhere, this is all I see:

(Pdb) l
(Pdb) args
(Pdb) n
(Pdb) n
(Pdb) l
(Pdb) cont

doctest is apparently capturing and mediating the output itself, and is using the output in assessing the tests. So, when the test run completes, I see everything that should have printed out when I was in the pdb shell within doctest’s failure report. This happens regardless of whether I invoke pdb.set_trace() inside a doctest line or inside the function or method being tested.

Obviously, this is a big drag. Doctests are great, but without an interactive pdb, I cannot debug any of the failures that they are detecting in order to fix them.

My thought process is to possibly redirect pdb’s output stream to something that circumvents doctest’s capture of the output, but I need some help figuring out the low-level io stuff that would be required to do that. Also, I don’t even know if it would be possible, and am too unfamiliar with doctest’s internals to know where to start. Anyone out there have any suggestions, or better, some code that could get this done?

Answer 1

I was able to get pdb by tweaking it. I just put the following code at the bottom of my testlib.py file:

import sys, pdb
class TestPdb(pdb.Pdb):
    def __init__(self, *args, **kwargs):
        self.__stdout_old = sys.stdout
        sys.stdout = sys.__stdout__
        pdb.Pdb.__init__(self, *args, **kwargs)

    def cmdloop(self, *args, **kwargs):
        sys.stdout = sys.__stdout__
        retval = pdb.Pdb.cmdloop(self, *args, **kwargs)
        sys.stdout = self.__stdout_old

def pdb_trace():
    debugger = TestPdb()
    debugger.set_trace(sys._getframe().f_back)

In order to use the debugger I just import testlib and call testlib.pdb_trace() and am dropped into a fully functional debugger.