Question

I am new to unit testing. I have created various tests and when I run test each one by one, all tests passing. However, when I run run on the whole as a batch, some tests failing. Why is that so? How can I correct that?

Answer 1

To resolve this issue it is important to follow certain rules when writing unit tests. Some of the rules are easy to follow and apply while other may need further considerations depending your circumstances.

Each test should set up a unique set of data. This is in particular important when you work with persistent data, e.g. in a database. When the test creates a user with a particular user id, then write the test so that it uses a different user id each time. For example (C#):

var user = new User(Guid.NewGuid());

At the end of each test, cleanup the data that the test has created. For example in the tear down method remove the data that you created (C#, NUnit):

[TearDown]
public void TheTearDownMethod() {
    _repository.Delete(_user);
}

Variations are possible, e.g. when you test against a database you may choose to load a backup just before you run the test suite. If you craft your tests carefully you don’t need to cleanup the database after each test (or subset of tests).

To get from where you are now (each test passes when run in isolation) to where you would like to be start with running the first two tests in sequence, make them pass. Then run three in sequence, make them pass, etc. In each iteration identify what previous test causes the added test to fail. Resolve that dependency. This way you learn a lot about your tests but also how to avoid writing tests that depend on each other.

Once the suite passes in one patch run it frequently so that you detect dependency as early as possible.

This doesn’t cover all scenarios and variations but hopefully gives you a guideline for building on what you already have.