Question

I’m having difficulties testing python functions that
return an iterable, like functions that are
yielding or functions that simply return an iterable, like return imap(f, some_iter) or return permutations([1,2,3]).

So with the permutations example, I expect the output of the function to be [(1, 2, 3), (1, 3, 2), ...]. So, I start testing my code.

def perm3():
  return permutations([1,2,3])

# Lets ignore test framework and such details
def test_perm3():
  assertEqual(perm3(), [(1, 2, 3), (1, 3, 2), ...])

This will not work, since perm3() is an iterable, not a
list. So we can fix this particular example.

def test_perm3():
  assertEqual(list(perm3()), [(1, 2, 3), (1, 3, 2), ...])

And this works fine. But what if I have nested iterables? That is
iterables yielding iterables? Like say the expressions
product(permutations([1, 2]), permutations([3, 4])). Now this is
probably not useful but it’s clear that it will be (once unrolling the
iterators) something like [((1, 2), (3, 4)), ((1, 2), (4, 3)), ...].
However, we can not just wrap list around our result, as that will only
turn iterable<blah> to [iterable<blah>, iterable<blah>, ...]. Well
of course I can do map(list, product(...)), but this only works for a
nesting level of 2.

So, does the python testing community have any solution for the
problems when testing iterables? Naturally some iterables can’t
be tested in this way, like if you want an infinite generator, but
still this issue should be common enough for somebody to have thought
about this.

Answer 1

I use KennyTM’s assertRecursiveEq:

import unittest
import collections
import itertools

class TestCase(unittest.TestCase):
    def assertRecursiveEq(self, first, second, *args, **kwargs):
        """
        https://stackoverflow.com/a/3124155/190597 (KennyTM)
        """
        if (isinstance(first, collections.Iterable)
            and isinstance(second, collections.Iterable)):
            for first_, second_ in itertools.izip_longest(
                    first, second, fillvalue = object()):
                self.assertRecursiveEq(first_, second_, *args, **kwargs)
        else:
            # If first = np.nan and second = np.nan, I want them to
            # compare equal. np.isnan raises TypeErrors on some inputs,
            # so I use `first != first` as a proxy. I avoid dependency on numpy
            # as a bonus.
            if not (first != first and second != second):
                self.assertAlmostEqual(first, second, *args, **kwargs)                

def perm3():
    return itertools.permutations([1,2,3])

class Test(TestCase):
    def test_perm3(self):
        self.assertRecursiveEq(perm3(),
            [(1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)])

if __name__ == '__main__':
    import sys
    sys.argv.insert(1, '--verbose')
    unittest.main(argv = sys.argv)