Question

A Rails/tool specific version of: How deep are your unit tests?

Right now, I currently write:

• Cucumber features (integration tests) – these test against the HTML/JS that is returned by our app, but sometimes also tests other things, like calls to third-party services.
• RSpec controller tests (functional tests), originally only if the controllers have any meaningful logic, but now more and more.
• RSpec model tests (unit tests)

Sometimes this is entirely necessary; it is necessary to test behavior in the model that is not entirely obvious or visible to the end-user. When models are complex, they should definitely be tested. But other times, it seems to me the tests are redundant. For instance, do you test method foo if it is only called by bar, and bar is tested? What if bar is a simple helper method on a model that is used by and easily testable in a Cucumber feature? Do you test the method in rspec as well as Cucumber? I find myself struggling with this, as writing more tests take time and maintaining multiple “versions” of what is effectively the same behaviors, which makes maintaining the test suite more time intensive, which in turn makes changes more expensive.

In short, do you believe there is there a time when writing only Cucumber features is enough? Or should you always test at every level? If you think there is a grey area, what is your threshold for “this needs a functional/unit test.” In practical terms, what do you do currently, and why (or why not) do you think it’s sufficient?

---

EDIT: Here’s an example of what might be “test overkill.” Admittedly, I was able to write this pretty quickly, but it was completely hypothetical.

Answer 1

Good question, one I’ve grappled with recently while working on a Rails app, also using Cucumber/RSpec. I try to test as much as possible at every level, however, I’ve also found that as the codebase grows, I sometimes feel I’m repeating myself needlessly.

Using “Outside-in” testing, my process usually goes something like: Cucumber Scenario -> Controller Spec -> Model Spec. More and more I find myself skipping over the controller specs as the cucumber scenarios cover much of their functionality. I usually go back and add the controller specs, but it can feel like a bit of a chore.

One step I take regularly is to run rcov on my cucumber features with rake cucumber:rcov and look for notable gaps in coverage. These are areas of the code I make sure to focus on so they have decent coverage, be it unit or integration tests.

I believe models/libs should be unit tested extensively, right off the bat, as it is the core business logic. It needs to work in isolation, outside of the normal web request/response process. For example, if I’m interacting with my app through the Rails console, I’m working directly with the business logic and I want the reassurance that methods I call on my models/classes are well tested.

At the end of the day, every app is different and I think it’s down to the developer(s) to determine how much test coverage should be devoted to different parts of the codebase and find the right balance so that your test suite doesn’t bog you down as your app grows.

Here’s an interesting article I dug up from my bookmarks that is worth reading:
http://webmozarts.com/2010/03/15/why-not-to-want-100-code-coverage/