Generally, I’m still very much a unit testing neophyte.
BTW, you may also see this question on other forums like xUnit.net, et cetera,
because it’s an important question to me. I apoligize in advance for my
cross posting; your opinions are very important to me and not everyone
in this forum belongs to the other forums too.
I was looking at a large decade old legacy system which has had over 700 unit tests
written recently (700 is just a small beginning). The tests happen to be written
in MSTest but this question applies to all testing frameworks AFAIK.
When I ran, via vs2008 “ALL TESTS”, the final count was only seven tests.
That’s about 1% of the total tests that have been written to date.
MORE INFORMATION: The ASP.NET MVC 2 RTM source code, including its unit tests,
is available on CodePlex; those unit tests are also written in MSTest
even though (an irrelevant fact) Brad Wilson later joined the ASP.NET MVC team
as its Senior Programmer. All 2000 plus tests get run, not just a few.
QUESTION: given that AFAIK the purpose of unit tests is to identify breakages
in the SUT, am I correct in thinking that the “best practice” is to always,
or at least very frequently, run all of the tests?
updated 2010-05-22
First, thank you to everyone who has provided excellent answers. Your answers
confirm my general conclusion that running all unit tests after every local rebuild is
the best practice regardless whether one is practicing TDD (test before) or classic
unit testing (test after).
imho, there is more than one best answer to this question but AFAIK SO lets
me select just one, so in an effort to be fair, I’ve given the check mark to
Pete Johns for being first and for earning the most up votes from the SO
community. Finland’s Esko Luontola also gave a great answer (I hope
he’s not getting buried in volcanic ash) and two very good links that
are worth your time imho; definitely the link to F.I.R.S.T. is to me inspirational;
AFAIK, only xUnit.net in the .NET world offers the “any order, any time”.
Esko’s second link is to a truly excellent 92 minute video “Integration Tests Are a Scam”
presented by J. B. (Joe) Rainsberger ( http://jbrains.ca where there is more content
worth my time). BTW, Esko’s weblog is also worth a visit http://orfjackal.net.
Since you have tagged this question ‘TDD’, all of the unit tests for the module being developed should be executed (and pass, bar the newest one, until you make it pass) with every compilation. Unit tests in other modules should not break by development in the current module or else they are testing too much.
A continuous integration loop should also be in place to ensure that all of the tests are run with every check-in to your source control system. This will trap breakages early.
At the very least, a nightly build should run each and every test and any breakages be fixed first thing of a morning. Tolerate no unit test failures!