Question

I was testing a String multiplier class with a multiply() method that takes 2 numbers as inputs (as String) and returns the result number (as String)

public String multiply(String num1, String num2);      

I have done the implementation and created a test class with the following test cases involving the input String parameter as

1. valid numbers
2. characters
3. special symbol
4. empty string
5. Null value
6. 0
7. Negative number
8. float
9. Boundary values
10. Numbers that are valid but their product is out of range
11. numbers will + sign (+23)

Now my questions are these:

1. I’d like to know if “each and every” assertEquals() should be in it’s own test method? Or, can I group similar test cases like testInvalidArguments() to contains all asserts involving invalid characters since ALL of them throw the same NumberFormatException ?

2. If testing an input value like character (“a”), do I need to include test cases for ALL scenarios?
   “a” as the first argument
   “a” as the second argument
   “a” and “b” as the 2 arguments

3. As per my understanding, the benefit of these unit tests is to find out the cases where the input from a user might fail and result in an exception. And, then we can give the user with a meaningful message (asking them to provide valid input) instead of an exception. Is that the correct? And, is it the only benefit?

4. Are the 11 test cases mentioned above sufficient? Did I miss something? Did I overdo? When is enough?

5. Following from the above point, have I successfully tested the multiply() method?

Answer 1

1) There is a tradeoff between granularity of tests (and hence ease of diagnosis) and verbosity of your unit test code. I’m personally happy to go for relatively coarse-grained test methods, especially once the tests and tested code have stabilized. The granularity issue is only relevant when tests fail. (If I get a failure in a multi-assertion testcase, I either fix the first failure and repeat, or I temporarily hack the testcase as required to figure out what is going on.)

2) Use your common sense. Based on your understanding of how the code is written, design your tests to exercise all of the qualitatively different subcases. Recognize that it is impossible to test all possible inputs in all but the most trivial cases.

3) The point of unit testing is to provide a level of assurance that the methods under test do what they are required to do. What this means depends on the code being tested. For example, if I am unit testing a sort method, validation of user input is irrelevant.

4) The coverage seems reasonable. However, without a detailed specification of what your class is required to do, and examination of the actual unit tests, it is impossible to say if you ave covered everything. For example, is your method supposed to cope with leading / trailing whitespace characters, numbers with decimal points, numbers like “123,456”, numbers expressed using non-latin digits, numbers in base 42?

5) Define “successfully tested”. If you mean, do my tests prove that the code has no errors, then the answer is a definite “NO”. Unless the unit tests enumerate each and every possible input, they cannot constitute a proof of correctness. (And in some circumstances, not even testing all inputs is sufficient.)

In all but the most trivial cases, testing cannot prove the absence of bugs. The only thing it can prove is that bugs are present. If you need to prove that a program has no bugs, you need to resort to “formal methods”; i.e. applying formal theorem proving techniques to your program.

And, as another answer points out, you need to give it to real users to see what they might come up with in the way of unexpected input. In other words … whether the stated or inferred user requirements are actually complete and valid.