Question

I have a pseudo random number generator (PRNG) class that I want to unit test. There are two approaches:

1. write a test case that takes a large amount of samples and test whether they are properly distributed. This approach may lead to a fairly long execution time for the test case;
2. calculate a small series of samples ‘by hand’ and verify if the PRNG algorithm reproduces it. This approach may lead to a not random sequence being generated without being noticed;

I would say that the first approach is not really unit testing because it does not perform a white box test of the generator, but on the other hand it properly tests the responsibility of the class. The second approach is more like a real unit test, focusing on the algorithm, but it does not provide as much evidence as to whether the class fulfills its responsibility.

Which approach do you prefer, and why?

---

Answer 1

Get another implementation of the same PRNG algorithm, generate a smallish number of longish test cases based on known seeds, and verify that your implementation of the algorithm matches everyone else’s implementations. The more data you test, the more chance it does. If you want to be serious, look into how FIPS validation is done for the algorithm.

There’s no need to test whether the output is random, since far more research has been done on the algorithm by others than you are capable of reproducing.

If you have invented your own PRNG algorithm then you have a rather different problem, because quite aside from testing your code you also need to test your new algorithm. There are various things to do — I think the most important are statistical testing on the output, and peer review by other cryptographers. Basically, though, if you were to design a PRNG algorithm without having enough knowledge in the field to know how to test it, then it will be rubbish.