I’ve just started looking at a project that has >20k unit tests written in Rspec (the project itself isn’t written in Ruby; just the test cases). The current number of test cases is expected to grow dramatically in the future, as more functionality is added. What’s already happened (over an extended period) is that RSpec started out being a particularly good solution for testing this project, but as the project has grown, the fairly ad-hoc structure of their RSpec test cases has bitten them badly. One of the biggest problems they’ve got is with taxonomy in their test code – the structure (or lack of) in naming their test cases, fixtures, helper code etc. in their test cases.
As you could imagine, with >20k unit tests, there’s a lot of methods with very similar names, using helper methods that are all loaded from a global namespace.
To highlight just one area where the problem is biting, there’s ~10 databases within this application. Checking the structure of tables/columns/views/constraints/stored procs/… for all of these databases is something that (quite reasonably) is covered within the existing RSpec unit tests. However, the total number of DDL entities within this collection of databases that needs to be checked is probably >10000; covering the whole range of DB structural checks and being able to selectively test only subsets of DB structure requires either:
-
10000 separate methods (and I’m ruling out that option straight away!)
- a fairly complex naming convention within the test cases (i.e. something encompassing the DB name + table name + column name + …),
- OR passing e.g. DB name, table name, column name, … to a generic method
- OR separation of concerns via namespaces (and I’m not aware of an elegantly scalable way of doing this within RSpec),
- OR some clever metaprogramming (that I suspect could ultimately make an already messy structure even more difficult to follow).
What exists now is a bit of all of the above, as far as I can tell, with not a lot of obvious planning…
Do you have any tips or references I could look at to try to straighten out this mess, and give them some sort of scalable structure to their RSpec testing? In particular, suggestions on how to structure the various RSpec files for very large projects would be very welcome.
The RSpec Book is an excellent resource for RSpec tips and tricks, and for BDD methodology in general (although your focus appears to be more about testing). There are several ways to simplify and DRY up specs to make them easier to manage, including shared examples (Chapter 12) and macros (Chapter 17).
I also recommend David Chelimsky’s blog.
Still it looks like your project could be a real challenge. Of the approaches you mentioned, I think using macros with DB, table, and column as parameters is the most promising.