Question

I have a number of projects in a solution file that have unit tests written for them and I want to set them up to be run by our continuous integration server. However, because many of the tests have been written poorly and have not been run regularly, there are many that are failing.

I don’t have the time at the moment to fix all of the tests, but I do believe there is value in having the existing tests run. What is the best way do deal with the failing unit tests?

What I am currently doing is marking each failing test as Explicit and leaving a TODO comment.

[Test, Explicit] //TODO: Rewrite this test because it fails 

Is there a better way of doing this? Or should I fix all the tests before including them in the tests that are run by the CIS?

Answer 1

Well, in NUnit you have the option to ignore tests using the ignore attribute:

[Test, Ignore('Test needs rewrite')] 

Personally though, there are two things that I do with such tests:

• Delete them if I don’t understand the test, or if the test is out of date/out of synch with current specs
• Refactor it to the correct spec if the fix is trivial

Gleaning from what you’ve written I would suspect that many of those failing tests are out of date and may not be relevant in the first place, so I think it would be fine to delete them.

There’s no point in keeping a test that nobody understands anyway.

UPDATE: Oren Eini has a blog post which outlines most of how I feel about activating old, failing tests:

The tests has no value by themselves: My most successful project didn’t have any tests

To quote:

Tests are a tool, and its usage should be evaluated against the usual metrics before applying it in a project. There are many reasons not to use tests, but most of them boil down to: ‘They add friction to the process’.

If retrofitting old, failing tests adds friction to the process, maybe it isn’t worth updating them at all.