Question

Consider the following code (from a requirement that says that 3 is special for some reason):

bool IsSpecial(int value)
   if (value == 3)
      return true
   else
      return false

I would unit test this with a couple of functions – one called TEST(3IsSpecial) that asserts that when passed 3 the function returns true and another that passes some random value other than 3 and asserts that the function returns false.

When the requirement changes and say it now becomes 3 and 20 are special, I would write another test that verifies that when called with 20 this function returns true as well. That test would fail and I would then go and update the if condition in the function.

Now, what if there are people on my team who do not believe in unit testing and they make this change. They will directly go and change the code and since my second unit test might not test for 20 (it could be randomly picking an int or have some other int hardcoded). Now my tests aren’t in sync with the code. How do I ensure that when they change the code some unit test or the other fails?

I could be doing something grossly wrong here so any other techniques to get around this are also welcome.

Answer 1

That’s a good question. As you note a Not3IsNotSpecial test picking a random non-3 value would be the traditional approach. This wouldn’t catch a change in the definition of “special”.

In a .NET environment you can use the new code contracts capability to write the test predicate (the postcondition) directly in the method. The static analyzer would catch the defect you proposed. For example:

Contract.Ensures(value != 3 && Contract.Result<Boolean>() == false);

I think anybody that’s a TDD fan is experimenting with contracts now to see use patterns. The idea that you have tools to prove correctness is very powerful. You can even specify these predicates for an interface.

The only testing approach I’ve seen that would address this is Model Based Testing. The idea is similar to the contracts approach. You set up the Not3IsNotSpecial condition abstractly (e.g., IsSpecial(x => x != 3) == false)) and let a model execution environment generate concrete tests. I’m not sure but I think these environments do static analysis as well. Anyway, you let the model execution environment run continuously against your SUT. I’ve never used such an environment, but the concept is interesting.