Question

You can use e.g. JUnit to test the functionality of your library, but how do you test its type-safetiness with regards to generics and wildcards?

Only testing against codes that compile is a “happy path” testing; shouldn’t you also test your API against non-type-safe usage and confirm that those codes do NOT compile?

   // how do you write and verify these kinds of "tests"?

    List<Number> numbers = new ArrayList<Number>();
    List<Object> objects = new ArrayList<Object>();

    objects.addAll(numbers); // expect: this compiles

    numbers.addAll(objects); // expect: this does not compile

So how do you verify that your genericized API raises the proper errors at compile time? Do you just build a suite a non-compiling code to test your library against, and consider a compilation error as a test success and vice versa? (Of course you have to confirm that the errors are generics-related).

Are there frameworks that facilitate such testing?

Answer 1

Since this is not testing in the traditional sense (that is – you can’t “run” the test), and I don’t think such a tool exists, here’s what I can suggest:

1. Make a regular unit-test
2. Generate code in it – both the right code and the wrong code
3. Use the Java compiler API to try to compile it and inspect the result

You can make an easy-to-use wrapper for that functionality and contribute it for anyone with your requirements.