In recent years I have been working on web applications written in Java using the Spring MVC framework. The projects have good test coverage with JUnit & Selenium. However on two occasions errors in the Spring Configuration have got through the testing process.
In one case a change was made to a parent bean in controllerContext.xml which also required a change to the two inheriting beans. But the required change was only made to one of the two inheriting beans. The error was only visible in a small, but critical, part of the Web application. Selenium UA tests were later extended to check this directly in the Web App. prior to deployment, but the damage had already been done as the error made it into the live environment.
In another case a property required to set the data format was not being injected properly via the applicationContext.xml. The only visible error was in the date format of a generated report downloaded from the web app. Difficult to test with Selenium.
One of the advantages of using Spring MVC is the ability to set the injected objects in your JUnit tests (i.e. a mock object), but this doesn’t tell you what you are actually going to get injected when the Application is running in the live environment.
The answer may be integration testing, however setting up & running integration testing has proved to be difficult in the past … but that’s another question …
So, I’d be really interested in learning how people have tried to catching possible errors introduced into Spring configuration files.
So my question is:
What is the best way to test for errors in the Spring Configuration?
The type of bugs you describe will only be caught by tests when someone thinks/remembers to test these conditions:
In both cases you have a developer who made a change without verifying that everywhere that change affects does not have errors. How can you catch this type of mistake in a JUnit-style test, without relying on the person making the change to explicitly add a test for the mistake they are making? In other words, you can only catch these types of bugs when you remember to test for them.
Personally I think the best approach for catching mistakes like this is more Selenium-like tests that actually invoke the application and assert the correct behavior.
But if your tests don’t know the correct behavior to test, you can’t catch mistakes like this – catching the mistakes requires you to first realize what mistakes need to be caught.
In other words – you can’t test that the correct values are injected without the test knowing what the correct values are. These tests won’t catch scenarios where what someone thinks the correct value is is actually incorrect.