As a hobby project I’m developing a small Bash unit testing framework. Now that the basic features are implemented the next step would be to write tests for it.
But how do I test my framework using the framework itself?
I’ve tried to Google an answer to this question but wasn’t able to find anything about this subject.
The idea I’ve come up with so far is writing tests for some trivial things which make use of all the features of the testing framework itself. If the tests are all green, then fine. These basic test could also serve as some sort of documentation.
But I don’t know if this is actually a good approach.
The problem will be to test that tests can fail. Unit tests should all be “green”. So choosing some tests that need to fail to prove it right is not an option.
I think that the solution will be quiet technology based. It is hard to give general advise about “unit testing a test framework”. Which parts belong to the framework? A part that reads files that contain unit tests, a part that runs them, a part that writes some report, a library of helpers, like asserts, … ?
You need to split these parts and make it possible to run them in isolation. Then it is no difference as any other tests.
Probably you find a way to run the whole testrunner within unit tests. This wouldn’t be unit tests as such, but rather integration tests. This kind of tests may become a maintenance problem, because they are usually very complex, use the file system and other stuff from the operation system which shouldn’t be used by unit tests. Consider if this is worth the effort.
There is always a part that can’t be tested. The part that puts everything together and runs the application (whatever it is) in a real environment. There is no difference here. It is even easier, because some of the “end user environment” can be tested by “green” tests.