At work we are currently still using JUnit 3 to run our tests. We have been considering switching over to JUnit 4 for new tests being written but I have been keeping an eye on TestNG for a while now. What experiences have you all had with either JUnit 4 or TestNG, and which seems to work better for very large numbers of tests? Having flexibility in writing tests is also important to us since our functional tests cover a wide aspect and need to be written in a variety of ways to get results.
Old tests will not be re-written as they do their job just fine. What I would like to see in new tests though is flexibility in the way the test can be written, natural assertions, grouping, and easily distributed test executions.
I’ve used both, but I have to agree with Justin Standard that you shouldn’t really consider rewriting your existing tests to any new format. Regardless of the decision, it is pretty trivial to run both. TestNG strives to be much more configurable than JUnit, but in the end they both work equally well.
TestNG has a neat feature where you can mark tests as a particular group, and then easily run all tests of a specific group, or exclude tests of a particular group. Thus you can mark tests that run slowly as in the ‘slow’ group and then ignore them when you want quick results. A suggestion from their documentation is to mark some subset as ‘checkin’ tests which should be run whenever you check new files in. I never saw such a feature in JUnit, but then again, if you don’t have it, you don’t REALLY miss it.
For all its claims of high configuration, I did run into a corner case the a couple weeks ago where I couldn’t do what I wanted to do… I wish I could remember what it is, but I wanted to bring it up so you know that it’s not perfect.
The biggest advantage TestNG has is annotations… which JUnit added in version 4 anyways.