Question

I have a question regarding unit testing. I’ll outline it using an example.

Say I’m writing a command line application which has a variety of acceptable commands, and each command has its acceptable arguments. For example: myapp create -DprojectName=newProject -DprojectVersion=1.0.

If I have a class called Command, which has a validateArguments method, which takes in a list of valid argument names and compares it against the list of arguments that were specified by the user, how would you go about unit testing this?

The possibilities I see are:

1. You write two unit tests – one that ensures no error is thrown when a valid argument is passed in, and one that ensures an error is thrown when an invalid argument is passed.

2. You write a unit test that ensures that all acceptable arguments are not rejected. I would end up having a unit test for every acceptable argument:

   @Test
   public void validateArgments_ProjectNameArgPassed_NoExcepton thrown { … }

   @Test
   public void validateArguments_ProjectVersionArgPassed_NoException thrown { … }

and so on.

To me, the first approach makes sense. But it doesn’t ensure that every argument that should be accepted, is.

Answer 1

It’s hard to suggest without knowing the logic of the underlying code (there’s a reason unit tests are white box tests and not black box) but my approach to that code would be a suite of unit tests that contain tests along the lines of:

• All parameters are invalid
• All parameters are valid
• Different combinations of valid and invalid parameters (to test un-tested code paths from the above)
• Different types of invalid parameters, e.g. not specified, incorrect format (-projectVersion=Hippopotamus), incorrect values (-projectVersion=99.0), etc.
• Any other failure conditions I might think of.

I find the real benefit of unit testing isn’t in testing the success scenario, as some simple integration testing can also provide that benefit. I find the real benefit is in testing numerous erroneous scenarios, because it is often the code that is rarely run (i.e. error handling) that contains bugs that may slip through the levels of testing unnoticed.