Question

If I have unit tests for each class and/or member function and acceptance tests for every user story do I have enough tests to ensure the project functions as expected?

For instance if I have unit tests and acceptance tests for a feature do I still need integration tests or should the unit and acceptance tests cover the same ground? Is there overlap between test types?

I’m talking about automated tests here. I know manual testing is still needed for things like ease of use, etc.

Answer 1

I’d recommend reading chapters 20 – 22 in the 2nd edition of Code Complete. It covers software quality very well.

Here’s a quick breakdown of some of the key points (all credit goes to McConnell, 2004)

Chapter 20 – The Software-Quality Landscape:

• No single defect-detection technique is completely effective by itself
• The earlier you find a defect, the less intertwined it will become with the rest of your code and the less damage it will cause

Chapter 21 – Collaborative Construction:

• Collaborative development practices tend to find a higher percentage of defects than testing and to find them more efficiently
• Collaborative development practices tend to find different kinds of errors than testing does, implying that you need to use both reviews and testing to ensure the quality of your software
• Pair programming typically costs the about the same as inspections and produces similar quality code

Chapter 22 – Developer Testing:

• Automated testing is useful in general and is essential for regression testing
• The best way to improve your testing process is to make it regular, measure it, and use what you learn to improve it
• Writing test cases before the code takes the same amount of time and effort as writing the test cases after the code, but it shortens defect-detection-debug-correction-cycles (Test Driven Development)

As far as how you are formulating your unit tests, you should consider basis testing, data-flow analysis, boundary analysis etc. All of these are explained in great detail in the book (which also includes many other references for further reading).

Maybe this isn’t exactly what you were asking, but I would say automated testing is definitely not enough of a strategy. You should also consider such things as pair programming, formal reviews (or informal reviews, depending on the size of the project) and test scaffolding along with your automated testing (unit tests, regression testing etc.).