Question

I’m developing my first Python distribution package. My learning curve on
Python packaging seems to be leveling off a bit, but I’m still wrestling with
a few open questions. One is whether I should cause my unit tests to be
installed alongside my code.

I understand it’s important to include tests in a source distribution. What I’m wondering is whether I should actually configure them to be installed?

I’ve seen at least one popular package that appears to do this on purpose
(PyHamcrest), and at least one other that appears to do it by accident
(behave).

So my (multi-part) question is this:

• Does it ever make sense to install my package unit tests alongside my
  package code?

• If so, what is the use case? Who would use them and for what? That is, who
  would use them that wouldn’t be perfectly happy to download the source
  distribution and run python setup.py test instead?

• And how would they use installed unit tests? like import test; test.run() or something
  like that?

Answer 1

After researching this issue, and until someone more experienced has a minute to weigh in to the contrary, my understanding is that the simple answer is: “No, unit tests should not be installed, only included in the source distribution“.

In the handful of cases I found where tests were installed, all turned out to be accidental and it’s easier than one might think to make the mistake without noticing it.

Here’s how it happens:

1. The packages=find_packages() parameter is used in setup.py so packages can be found without having to list them out explicitly.
2. The test folder is made into a package (by adding __init__.py) so tests can reference the modules they test using relative naming (like from .. import pkg.mod).
3. setuptools installs test as a separate package, alongside the other(s) in the project. Note this means you can execute import test in the python interpreter and it works, almost certainly not what you intended, especially since a lot of other folks use that name for their test directory 🙂

The fix is to use the setting: packages=find_packages(exclude=['test']) to prevent your test directory from being installed.