Question

I’m working on a PHP project with solid unit-tests coverage.

I’ve noticed, that last time, I’m making very tricky manipulations with unit-tests Command-Line Test Runner’ –filter command.

Here is this command’s explanation from official documentation:

–filter

Only runs tests whose name matches the given pattern. The pattern can be either the name of a single test or a regular expression that matches multiple test names.

I ofter use it because sometimes it becomes very useful to run just a single test suite or test case from the whole test base.

I’m wondering if this is good practice or not?

I have heard that sometimes it is good practice to to run the whole test suite on your Continuous Integration machine, if you know for sure that you have modified only one component and 100% percent confident, that it won’t fail other component’s unit-tests.
What do you think about it?

Some time ago I thought that we shouldn’t care so much about time require to run the whole suite of all unit-tests, but when you have very complicated business logic and unit-tests – this can take significant time.

I understand, that “real” unit-tests shouldn’t interact with DB, use mock/stubs objects, I agree with that. But sometimes, it is much easier(cheaper) to use DB fixtures for the tests.

Please give me some advice, how this problem can be solved?

Answer 1

Good unit tests should:

• Have clear methods names and variable names to act as documentation
• Run fast. This will also be possible
  for test with complicated business
  logic. Test should run in an avarage
  time of something around 0.1 second.
• Test exactly one thing in one test method
• Not integrate with external resources like the filesystem, email,
  databases, webservices, and
  everything else. You can create
  seperate database integration tests
  to test your database ineraction.
  These test will slower then your unit
  test most of the time. I put my
  integration tests in a seperate
  project and I run them only when I am
  working on the integration code. I
  also run them on all builds on the CI
  server.
• Be completely isolated from each other. When you have tests depending
  on each other, you cannot see what
  your problem is from reading which
  tests are failed. You might have to
  debug to find the problem. Isolated
  tests will save you a lot of time.

Personally, I don’t use category names in my tests. I use 2 test projects per application. One for the unit test and one for the integration tests and slower tests.

Reaction on:

“But sometimes, it is much
easier(cheaper) to use DB fixtures for
the tests.”

When your code is written well, it will be easier to mock. I don’t know about mocking frameworks in Php, but I use them in other languages to save me a lot of time. Writing test first and code later might help you to design your code to be testable easier.

Personally I learned to test better by

• reading blogs about it
• reading books about it
• reading tested code written by others
• writing a lot of tests of course. It took me a few thousends of tests to become good at it.