Question

I was taught that a regression test was a small (only enough to prove you didn’t break anything with the introduction of a change or new modules) sample of the overall tests. However, this article by Ron Morrison and Grady Booch makes me think differently:

The desired strategy would be to bring each unit in one at a time, perform an extensive regression test, correct any defects and then proceed to the next unit.

The same document also says:

As soon as a small number of units are added, a test version is generated and ‘smoke tested,’ wherein a small number of tests are run to gain confidence that the integrated product will function as expected. The intent is neither to thoroughly test the new unit(s) nor to completely regression test the overall system.

When describing smoke testing, the authors say this:

It is also important that the Smoke Test perform a quick check of the entire system, not just the new component(s).

I’ve never seen ‘extensive’ and ‘regression test’ used together nor a regression test described as ‘completely regression test the overall system’. Regression tests are supposed to be as light and quick as possible. And the definition of smoke test is what I learned a regression test was.

Did I misunderstand what I was taught? Was I taught incorrectly? Or are there multiple interpretations of ‘regression test’?

Answer 1

There are multiple interpretations. If you’re only fixing a bug that affects one small part of your system then regression tests might only include a small suite of tests that exercise the class or package in question. If you’re fixing a bug or adding a feature that has wider scope then your regression tests should have wider scope as well.

The ‘if it could possibly break, test it’ rule of thumb applies here. If a change in Foo could affect Bar, then run the regressions for both.

Regression tests just check to see if a change caused a previously passed test to fail. They can be run at any level (unit, integration, system). Reference.