Question

Would you recommend doing any grouping of test cases within @Test methods, or have one @Test method per test scenario? For example, let’s suppose that there are different ways to set the context in an application.

Is the following idea acceptable?

@Test
public void testContextSetting() {
    // Test default setting
    assert(...)

    // Test setting a context variable
    assert(...)

    ...
}

Or, would you rather suggest having it like this, having each method as atomic as possible:

@Test
public void textDefaultSetting() {
    // Test default setting
    assert(...)
}

@Test
public void testSettingContextVar() {
    // Test setting a context variable
    assert(...)

    ...
}

Any feedback would be appreciated.

Answer 1

I prefer having one test case per method.

First it is easier to see what cases are being tested if they are split into methods as opposed to looking for comments embedded in the code. Most IDEs will give you a summary of methods, so instead of saying “did I test edgecase XYZ?” and then hunting for a comment, or looking for the code that sets up that edgecase, you just look for the method named setupContextEdgeCaseXYZ().

A second reason is if you have multiple cases together one may fail and then the others never execute.

 testDefaultCase()
 testInvalidInput()
 testEdgeCase1()
 testEdgeCase2()

With this structure it would be easier to determine that the input checking is bad and edge case 2 is handled improperly, but the others are OK (and you may find out that two failing cases are related and the problem is diagnosed faster).

A third reason is you may accidentally leave values from a previous test set that invalidates a latter test in a inconspicuous way. A simple example:

@Test
public void testMyMethod() {
  //test default
  String test = Foo.bar(null);
  assertEquals("foo", test);

  //test case 1
  Foo.bar(aValue);
  //Oops forgot to set value above, this passes regardless of 
  //what the above call does
  assertEquals("foo", test);
}

By breaking cases apart you can avoid mistakes as above as that would turn into a compile error or warning.