Question

I have an application where I want to write unit tests to test the output, written to System.out (and perhaps System.err).
Each individual test works as expected, however when adding multiple tests in the same class some tests fail because JUnit4 appears to be multi-threaded (thus no guarantees exists on when exactly the stream is reset).
The same happens when I separate all test methods in their own class and use a test suite.

Any ideas?

private static final PrintStream SYS_OUT = System.out;
private static final PrintStream SYS_ERR = System.err;

private final ByteArrayOutputStream outContent = new ByteArrayOutputStream();
private final ByteArrayOutputStream errContent = new ByteArrayOutputStream();

@Before
public final void setUpStreams() throws UnsupportedEncodingException {
    System.setOut(new PrintStream(this.outContent, true, CHARSET));
    System.setErr(new PrintStream(this.errContent, true, CHARSET));
}

@After
public final void cleanUpStreams() {
    System.setOut(SYS_OUT);
    System.setErr(SYS_ERR);
}

@Test
public final void listServicesAndMethods() throws ServiceException, UnsupportedEncodingException {
    com.example.Main.main(new String[]{"--list-services"});
    LOG.debug(this.outContent.toString(CHARSET));
    assertTrue("String not found", this.outContent.toString(CHARSET).contains("Some string"));
    assertFalse("Other string found", this.outContent.toString(CHARSET).contains("Some other string"));
    this.outContent.reset();
    this.errContent.reset();
}

Edit: It turns out that the issue with the failing tests was not (just?) because of the streams, but due to the fact that I stored the options in static fields in my main class. This has the effect that several options stay active during consecutive tests. I realised that after implementing Arian’s suggestion, I then used the second class as an instance, instead of calling static methods, thus solving my issue.

Thanks to all who replied.

Answer 1

If not already done so, rewrite the parts of your application to take the output as a parameter, instead of writing directly to System.out. This is usually better design regardless of testing.

In each test, create a new output stream (or spy on System.out if you must) and pass it to the code unit under test.