We usually write lots of tests and execute them frequently in Eclipse. Furthermore, we use Maven and later on Jenkins, which also run the tests. Every now and then we run into a situation where it makes a difference whether we test each test method, a whole test class or an entire package at once (meaning tests suddenly become red).
Consider static fields that are initialised already, jobs that keep running unintentionally, test methods are executed in a rather random order, … In other words there are plenty of situations where tests can interfere each other. Obviously we strive to write the methods in a way that all of the above doesn’t matter/happen and all is green no matter what the circumstances are.
I don’t know how the aforementioned products actually run the tests, but it seems that the JVM is launched, then all selected tests (a method, a class or a project) are done and after that the JVM shuts down. I haven’t seen an option that allows me to run the tests in all three possible ways automatically.
My questions now are:
- How do you deal with it?
- Do you actually consider this a problem?
Tests should not depend on each other and not affect other tests. If you run into a situation where a test fails – you should find the reason and eliminate it.
I only once had a situation where I needed to fork the JVM for each test. This can be done by using the forkMode property of the maven-surefire-plugin.