Question

I came across a printed article by Bertrand Meyer where he states that tests can be generated from specifications. My development team does nothing like this, but it sounds like a good technique to consider. How are you generating tests from specifications? How would you describe the success your having in discovering program faults via this method?

Answer 1

There are all sorts of ways to do it, ranging from what I’d consider an ‘art form’ (and not necessarily good art) all the way to mathematically derived tests from formal specifications. At the end of the day, your development team needs to decided on what they can do based on the schedule they are working with. That being said, being able to test software against specs is a Good Thing.

Only your team can gauge the ‘depth’ of your tests, and that will probably be a function of how good your specs are. If they say something like, ‘the login UI needs to provide a cancel button and a login button, and they need to work’, your tests are going to be pretty general. But keep in mind – even very general tests are a Good Thing. Testing is a Good Thing. Too many developers have a bad attitude when it comes to testing, but at the end of the day, you’re shipping software which should work, and to me, that means a lot.

The effectiveness your tests will having in finding program faults will depend on the detail you put into them. What is especially nice about having test procedures written to specs is that you can test each build to the same level of detail as the previous build (typically referred to as a regression test).