Question

I think understand the definition of State / Interaction based testing (read the Fowler thing, etc). I found that I started state based but have been doing more interaction based and I’m getting a bit confused on how to test certain things.

I have a controller in MVC and an action calls a service to deny a package:

public ActionResult Deny(int id)
{
    service.DenyPackage(id);

    return RedirectToAction("List");
}

This seems clear to me. Provide a mock service, verify it was called correctly, done.

Now, I have an action for a view that lets the user associate a certificate with a package:

public ActionResult Upload(int id)
{
    var package = packageRepository.GetPackage(id);
    var certificates = certificateRepository.GetAllCertificates();

    var view = new PackageUploadViewModel(package, certificates);

    return View(view);
}

This one I’m a bit stumped on. I’m doing Spec style tests (possibly incorrectly) so to test this method I have a class and then two tests: verify the package repository was called, verify the certificate repository was called. I actually want a third to test to verify that the constructor was called but have no idea how to do that! I’m get the impression this is completely wrong.

So for state based testing I would pass in the id and then test the ActionResult’s view. Okay, that makes sense. But wouldn’t I have a test on the PackageUploadViewModel constructor? So if I have a test on the constructor, then part of me would just want to verify that I call the constructor and that the action return matches what the constructor returns.

Now, another option I can think of is I have a PackageUploadViewModelBuilder (or something equally dumbly named) that has dependency on the two repositories and then I just pass the id to a CreateViewModel method or something. I could then mock this object, verify everything, and be happy. But … well … it seems extravagant. I’m making something simple … not simple. Plus, controller.action(id) returning builder.create(id) seems like adding a layer for no reason (the controller is responsible for building view models.. right?)

I dunno… I’m thinking more state based testing is necessary, but I’m afraid if I start testing return values then if Method A can get called in 8 different contexts I’m going to have a test explosion with a lot of repetition. I had been using interaction based testing to pass some of those contexts to Method B so that all I have to do is verify Method A called Method B and I have Method B tested so Method A can just trust that those contexts are handled. So interaction based testing is building this hierarchy of tests but state based testing is going to flatten it out some.

I have no idea if that made any sense.

Wow, this is long …

Answer 1

I think Roy Osherove recently twitted that as a rule of thumb, your tests should be 95 percent state-based and 5 percent interaction-based. I agree.

What matters most is that your API does what you want it to, and that is what you need to test. If you test the mechanics of how it achieves what it needs to do, you are very likely to end up with Overspecified Tests, which will bite you when it comes to maintainability.

In most cases, you can design your API so that state-based testing is the natural choice, because that is just so much easier.

To examine your Upload example: Does it matter that GetPackage and GetAllCertificates was called? Is that really the expected outcome of the Upload method?

I would guess not. My guess is that the purpose of the Upload method – it’s very reason for existing – is to populate and serve the correct View.

So state-based testing would examine the returned ViewResult and its ViewModel and verify that it has all the correct values.

Sure, as the code stands right now, you will need to provide Test Doubles for packageRepository and certificateRepository, because otherwise exceptions will be thrown, but it doesn’t look like it is important in itself that the repository methods are being called.

If you use Stubs instead of Mocks for your repositories, your tests are no longer tied to internal implementation details. If you later on decide to change the implementation of the Upload method to use cached instances of packages (or whatever), the Stub will not be called, but that’s okay because it’s not important anyway – what is important is that the returned View contains the expected data.

This is much more preferrable than having the test break even if all the returned data is as it should be.

Interestingly, your Deny example looks like a prime example where interaction-based testing is still warranted, because it is only by examining Indirect Outputs that you can verify that the method performed the correct action (the DenyPackage method returns void).

All this, and more, is explained very well in the excellent book xUnit Test Patterns.