In many open source projects, the test suite that is used is often comprehensive. This is done to allow many, many people to easily contribute changes without breaking too much along the way.
One piece of software that I’ve used is the testing framework FitnNesse. One interesting thing about FitNesse is that the test suite is part of the “installed” system.
In released software, are there any guidelines for what level of tests to incorporate?
Obviously, including all unit tests in a released product is probably not the right thing to do.
Another name for these incorporated tests might be “diagnostics”.
…well the question is then, what kind of issues would these diagnose? IOW how would you describe the purpose of such tests?
For example, in one of my projects we had a module officially included in product releases and thoroughly documented in product guide. It was called iirc readiness or diagnostic test suite. Its purpose was to verify whether the target platform meets basic requirements allowing to use our product. Failure of the test in this suite typically indicated some showstopper bug in target platform.
I don’t recall now whether adding above module was requested by our testers or customers but I clearly recall the reasoning behind that request – guys using (or testing) our product just felt fed up with mysterious crashes at installation / usage which causes after cumbersome investigations were being identified as simple (but severe) bugs in target platform.