Question

I have Junit 4.8.2 and maven 3
Some tests in my application should fail the build in case of failure and some of them shouldn’t (just report that the following optional tests failed)

How can I do it with junit and if I can’t then maybe testng can?

E.g. I have two test cases:

First is not really important and can failed because of connection timeout, service unavailability and so on so on. So if it fail, I don’t want to fail whole build, just to let user know about it and write to console

Second is really important one and if it fail – build should be failed as well

I know about @Ignore – it is not what I’m looking for, because I don’t want to skip any tests.

I know about @Category so if you know how to configure surefire plugin to say: if category com.me.Required – build should be failed in case of failure and if category com.me.Optional – build should not be failed

Answer 1

Consider using the failsafe plugin for your tests that are allowed to fail and set the testFailureIgnore flag to true.

To use the failsafe plugin you have to add the plugin to you pom

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>foo.bar</groupId>
    <artifactId>test</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.12.4</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <testFailureIgnore>true</testFailureIgnore>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

The surefire plugin will per default execute test named like Test. The failsafe plugin will per default execute the test named like IT.

Given the tests

import static org.junit.Assert.*;

import org.junit.Test;

public class SurefireTest {

    @Test
    public void test() {
        assertTrue(true);
    }

}

and

import static org.junit.Assert.*;

import org.junit.Test;

public class FailsafeIT {

    @Test
    public void test() {
        assertTrue(false);
    }

}

Running mvn install will now result in

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building test 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
.
.
.
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running SurefireTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.062 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
.
.
.
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running FailsafeIT
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.072 sec <<< FA
ILURE!
...
Results :

Failed tests:   test(FailsafeIT)

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
.
.
.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.174s
[INFO] Finished at: Sat Sep 29 08:19:38 CEST 2012
[INFO] Final Memory: 9M/245M
[INFO] ------------------------------------------------------------------------