I have been watching Ryan Bates’ RailsCasts for years now, and I’m also a customer of the pay portion of the site. I learned BDD from watching Cucumber episodes in the past.
Now I have learned about TestUnit, RSpec, Capybara, and MiniTest. I am beginning to get confused about what is what.
What is the difference between these 4 items? I know obviously Cucumber executes plain text features, and I guess that could be considered integration testing.
But now I also see that recent versions of Cucumber require MiniTest? Is Cucumber simply a DSL that sits on top of a testing framework?
I know also that RSpec has its own syntactical sugar for doing assertions, the “describe” blocks. And it appears that MiniTest also supports this syntax.
I know that Capybara is used for looking at the web page content that is generated, I think.
Here is my question:
If I am creating a new Rails 3.2 application, what combination of these testing programs should I use? What would be extra helpful is a list that explains how these gems and their associated processes complement each other where applicable, like:
Cucumber is a DSL for driving BDD
Cucumber is for integration tests and is based on creating user stories that are customer-readable
It implements its tests behind the scenes via MiniTest
MiniTest is a new testing framework that comes with Ruby 1.9 and is very fast.
MiniTest is also used for unit testing, such as testing controllers and models
It does not yet have as many features as RSpec
Cucumber also uses Capybara to access DOM elements from a javascript-enabled browser simulator such as Selenium
When you test in Rails, you have the ability to do the following kinds of tests: controllers, views, models, and integration (MVC together)
Some people just do integration and model testing, because they feel that integration testing handles enough of the controller and view testing itself, and anything too complex can simply be moved up to the model
Thank you so much for any help you can offer to clear these ideas up for me.
Okay, so let me try and explain, based on my own experience
Cucumber, is a ATDD (or BDD) tool that lets you write tests in a business-oriented domain language. It’s primary use is as a conversation tool with you product owners. Instead of writing detailed requirements, you express those requirements as examples of the system under test. Each cucumber test, in effect, becomes a business requirement that must be satisfied.
The Cucumber tool itself, translates these plain-text statements into a little module for you to execute ruby code in. What Ruby Library you use inside of the step definitions depends entirely on the project you are working on.
The safest way to describe Cucumber is that it’s a testing framework with an emphasis on communication between IT and Business Partners, which is why it’s becoming so popular.
Rspec and Minitest are other frameworks with other strengths, but lack this business readability factor, in that they are mostly code and not nearly as readable to a non-technical person. That isn’t necessarily a bad thing, especially if your product owner is a little more hands off.
How does this tie in with something like Capybara? Capybara is an integration test automation library that drives a headless browser on the Rack::Test framework for very fast tests, and has a highly readable DSL. The only downside is Rack::Test doesn’t support javascript, so it provides a way to fail over to Selenium if you a running a javascript test. Rspec and Cucumber both have mechanisms for triggering this failover.
There are lots of other Ruby automation libraries. For instance, if you are doing testing against a non-rails Web App, your cucumber scenarios might be defined with Watir-Webdriver which will drive a full web browser like Capybara does when in Javascript mode. The main difference being that WW has a lot more robust set of selctors than Capybara does, so it’s a bit easier to write especially if your code isn’t super clean (Capybara only supports selecting by ID, Value and Text, whereas WW supports selecting by almost anything)
So chances are you are going to want to use RSpec or Minitest for your unit testing )or the default Test::Unit), and Cucumber for Integration Testing IF you have an interested product owner or a need for a common language for test scenarios. If you do not, you can write your integration tests as Rspec examples or the minitest equivalent without losing much.