Question

I’m implementing unit tests for a family of functions that all share a number of invariants. For example, calling the function with two matrices produce a matrix of known shape.

I would like to write unit tests to test the entire family of functions for this property, without having to write an individual test case for each function (particularly since more functions might be added later).

One way to do this would be to iterate over a list of these functions:

import unittest import numpy  from somewhere import the_functions from somewhere.else import TheClass  class Test_the_functions(unittest.TestCase):   def setUp(self):     self.matrix1 = numpy.ones((5,10))     self.matrix2 = numpy.identity(5)    def testOutputShape(unittest.TestCase):      '''Output of functions be of a certain shape'''      for function in all_functions:        output = function(self.matrix1, self.matrix2)        fail_message = '%s produces output of the wrong shape' % str(function)        self.assertEqual(self.matrix1.shape, output.shape, fail_message)  if __name__ == '__main__':   unittest.main() 

I got the idea for this from Dive Into Python. There, it’s not a list of functions being tested but a list of known input-output pairs. The problem with this approach is that if any element of the list fails the test, the later elements don’t get tested.

I looked at subclassing unittest.TestCase and somehow providing the specific function to test as an argument, but as far as I can tell that prevents us from using unittest.main() because there would be no way to pass the argument to the testcase.

I also looked at dynamically attaching ‘testSomething’ functions to the testcase, by using setattr with a lamdba, but the testcase did not recognize them.

How can I rewrite this so it remains trivial to expand the list of tests, while still ensuring every test is run?

Answer 1

You could use a metaclass to dynamically insert the tests. This works fine for me:

import unittest  class UnderTest(object):      def f1(self, i):         return i + 1      def f2(self, i):         return i + 2  class TestMeta(type):      def __new__(cls, name, bases, attrs):         funcs = [t for t in dir(UnderTest) if t[0] == 'f']          def doTest(t):             def f(slf):                 ut=UnderTest()                 getattr(ut, t)(3)             return f          for f in funcs:             attrs['test_gen_' + f] = doTest(f)         return type.__new__(cls, name, bases, attrs)  class T(unittest.TestCase):      __metaclass__ = TestMeta      def testOne(self):         self.assertTrue(True)  if __name__ == '__main__':     unittest.main()