Question

We’re using the maven-surefire-plugin to run our Java tests. The tests fall into two categories:

• Fast tests
• Slow tests

The whole “fast” suite runs in a couple of seconds while the slow tests take half an hour.

During development, I want to run only the fast tests. When I commit, I’d like to be able to run the slow tests as well, so running the slow tests should be an option while the fast tests should be the default.

On the CI server, I want to run both.

It’s OK (and even preferred) when the slow tests include the fast ones.

How should I set up Maven, JUnit and Surefire for this scenario?

Answer 1

In a commercial project I’ve made from scratch on my own, I divided tests into unit (which I named *Test.java), and integration (*IT.java), according to, respectively, Surefire and Failsafe plugin policies, which I used for running the tests. ITs run, of course, much slower than UTs.

This gives the power of running the group of tests with simple commands: mvn test for UTs and mvn integration-test for both UTs and ITs, as well as the possiblity of skipping only ITs with mvn install -DskipITs.

One more good thing is a possibility to be more lax with integration tests results, as they fail more often than unit tests, because of problems with environment (i.e. database taking too long to start, message broker shutting down too early, and so on). By default, failure of a Failsafe test does not kill the build, unless you include the “verify” goal explicitly:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.6</version>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>integration-test</goal>
            </goals>
        </execution>
        <!-- Uncomment this in order to fail the build if any integration test fail -->
        <!-- execution> <id>verify</id> <goals><goal>verify</goal></goals> </execution -->
    </executions>
</plugin>