Question

Before this question is marked as duplicate, please read it. 😉
There are already several questions about coverage tools and such, however this is a bit different than the usual ones (I hope).

According to wikipedia there are several different kind of ‘coverage’ variations that affect several different aspects of the term ‘coverage’.

Here a little example:

public class Dummy {
    public int a = 0;
    public int b = 0;
    public int c = 0;

    public void doSomething() {
        a += 5;

        b += 5;

        c = b + 5;
    }
}

public class DummyTest {

    @Test
    public void testDoSomething() {
        Dummy dummy = new Dummy();

        dummy.doSomething();

        assertEquals( 10, dummy.c );
    }
}

As you can see, the test will have a coverage of 100% lines, the assertion on the value of field ‘c’ will cover this field and indirectly also cover field ‘b’, however there is no assertion coverage on field ‘a’.
This means that the test covers 100% of the code lines and assures that c contains the expected value and most probably also b contains the correct one, however a is not asserted at all and may a completely wrong value.

So… now the question: Is there a tool able to analyze the (java) code and create a report about which fields/variables/whatever have not been (directly and/or indirectly) covered by an assertion?

(ok when using getters instead of public fields you would see that getA() is not called, but well this is not the answer I’d like to hear 😉 )

Answer 1

As you can see, the test will have a coverage of 100% lines, the assertion on the value of field ‘c’ will cover this field and indirectly also cover field ‘b’, however there is no assertion coverage on field ‘a’. This means that the test covers 100% of the code lines and assures that c contains the expected value and most probably also b contains the correct one, however a is not asserted at all and may a completely wrong value.

Well, “cover” unfortunately means different things to different people… This test indeed exercises 100% of the code lines, but it does not test them all.

What you’re looking for is handled well by mutation testing.

Have a look at Jester, which uses mutation testing to report on code coverage.