Question

I am using junit with eclipse to write function tests.

When running an individual test it runs in the order that I have them set within the class.

Eg.

testCreateUser
testJoinUserToRoom
testVerify
testDeleteUser

However when I run this test as part of a suite, (so in a package) the order is random.

It will for example do the verify, then delete user then joinuserToRoom then Createuser.

My tests within the suite are not dependent on each other. However each individual test within a test is dependent on them being run in the correct order.

Is there any way I can achieve this?

Thanks.

Answer 1

You can’t guarantee the order of execution of test methods in JUnit.

The order of execution of test classes within a suite is guaranteed (if you’re using Suite), but the order of execution if the test classes are found by reflection isn’t (for instance, if you’re running a package in Eclipse, or a set of tests from maven or ant). This may be definable by ant or maven, but it isn’t defined by JUnit.

In general, JUnit executes the test methods in the order in which they are defined in the source file, but not all JVMs guarantee this (particulary with JVM 7). If some of the methods are inherited from an abstract base test class, then this may not hold either. (This sounds like your case, but I can’t tell from your description).

For more information on this, see my answer to Has JUnit4 begun supporting ordering of test? Is it intentional?.

So what can you do to fix your problem? There are two solutions.

In your original example, you’ve actually only got one test (verify), but you’ve got 4 methods, two setup (createUser, joinUserToRoom) and one teardown (deleteUser). So your first option is to better define your test cases, using a TestRule, in particular ExternalResource. ExternalResource allows you to define before/after behaviour for a test, similar to @Before/@After. However, the advantage of ExternalResource is that you can factor this out of your test.

So, you would create/delete the user in your external resource:

public class UsesExternalResource {
     @Rule
     public ExternalResource resource= new ExternalResource() {
         @Override
         protected void before() throws Throwable {
            // create user
         };

         @Override
         protected void after() {
            // destroy user
         };
     };

     @Test
     public void testJoinUserToRoom() {
        // join user to room
        // verify all ok
     }
}

For me, this is simpler and easier to understand, and you get independent tests, which is a good thing. This is what I would do, but you will need to refactor your tests a bit. You can also stack these Rules using RuleChain.

Your second option, if you really want to introduce dependencies between your test methods, is to look at TestNG, in which you can define dependencies from one test to another.