Question

These tests should run after the jar file is built.
They should test whether it runs at all,
and whether it produces correct console output given certain input files.

Could you point at some examples of such tests for console applications? Do these test have to be written in Java to be used with Maven?

There is a similar question (Testing console based applications/programs – Java) but what I need is blackbox testing, not unit testing. And it does not mention Maven.

---

UPDATE:

It turned out that it vas very easy to do. My first problem was a misconception of integration tests after seeing so many definitions here (What is Unit test, Integration Test, Smoke test, Regression Test?; What's the difference between unit, functional, acceptance, and integration tests?; What are unit testing and integration testing, and what other types of testing should I know about?; What is an integration test exactly?). The second problem was that I did not intend to write these test in Java, but in the end I had to learn how to use java.lang.Runtime.

First of all, I have added this to my pom.xml:

<plugin>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.6</version>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Then I have created an *IT.java file in src/test/java:

public class MyAppIT {

    @Test
    public void test() throws IOException {
        Runtime runtime = Runtime.getRuntime();
        String cmd = "java -cp target/myapp.jar my.app.Main";
        Process process = runtime.exec(cmd);
        InputStreamReader isr = new InputStreamReader(process.getErrorStream());
        BufferedReader br = new BufferedReader(isr);
        Boolean containsUsage = false;
        String line;
        while ((line = br.readLine()) != null) {
            if (line.contains("Usage")) {
                containsUsage = true;
            }
        }
        assertTrue(containsUsage);
    }
}

Now I use mvn verify instead of mvn package.

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Answer 1

It seems you want integration testing.

In many cases, the application is built by a continuous integration (CI) system like Hudson, Travis or sth. This CI compiles and runs the unit-tests after each commit to the sourcecode.
After successful compilation, the integration-test projects are triggered.

The integration-tests reside in a separate maven source project. You would create the integration-test-classes like normal unit tests in src/test/java. Of course you don’t test a single class in isolation – that is unit testing – you setup your system for integration-tests (wire the right components) and execute the needed tests “smoke test”.

So, in your case, you could create another project for integration-test and execute this after the build of your application. You could create a maven parent-pom and use sub-modules to avoid the manual integration-test-build.

The test classes need to be java and located in src/test/java, they look like normal unit tests and are run by maven.
Of course you could start a non-java program in that tests and assert the outcome.

A console app has a “main” method, so I would call the main method in the test. Otherwise you have to run a shell command or something. (via java.lang.Runtime)

There is no speciality in testing console apps. Unless you want to test the user interface – automated testing of user interfaces is not easy and more difficult, please clarify if this is your focus.