Question

My test project consists of a set of input data files which is fed into a piece of legacy third-party software. Since the input data files for this software are difficult to construct (not something that can be done intentionally), I am not going to add new input data files.

Each input data file will be subject to a set of “test functions”. Some of the test functions can be invoked independently. Other test functions represent the stages of a sequential operation – if an earlier stage fails, the subsequent stages do not need to be executed.

I have experimented with the NUnit parametrized test case (TestCaseAttribute and TestCaseSourceAttribute), passing in the list of data files as test cases. I am generally satisfied with the the ability to select the input data for testing.

However, I would like to see if it is possible to customize its GUI’s tree structure, so that the “test functions” become the children of the “input data”. For example:

• File #1
  • CheckFileTypeTest
  • GetFileTopLevelStructureTest
  • CompleteProcessTest
    • StageOneTest
    • StageTwoTest
    • StageThreeTest
• File #2
  • CheckFileTypeTest
  • GetFileTopLevelStructureTest
  • CompleteProcessTest
    • StageOneTest
    • StageTwoTest
    • StageThreeTest

This will be useful for identifying the stage that failed during the processing of a particular input file.

Is there any tips and tricks that will enable the new tree layout? Do I need to customize NUnit to get this layout?

Edited: See the question Term for unit testing that separates test logic from test result data for an introduction to the concept of separating test data and test code. In my situation, the separation is driven by practical concerns, not by ideological reasons. The sources of my test data files are “in the wild”.

Other anecdotal use of data-driven tests:

• http://www.clear-lines.com/blog/post/Data-driven-tests-with-NUnit-25.aspx
• http://www.clear-lines.com/blog/post/No-Tolerance-for-NUnit-Data-Driven-tests.aspx

Answer 1

You are probably not going to be able to model the structure exactly like this. Tests can only exist in TestFixtures (classes) and the only thing that allows you to nest arbitrary levels in the GUI are namespaces.

What you could do is this:

* File001 (namespace)
  * Tests_File001 (class, text fixture)
    * CheckFileTypeTest (method, test)
    * GetFileTopLevelStructureTest (method, test)
  * CompleteProcessTest (namespace)
    * TestsCompleteProcessTest (class, text fixture)
      * StageOneTest (method, test)
      * StageTwoTest (method, test)
      * StageThreeTest (method, test)

I assume you currently have a setup more along the lines of

[Test]
[TestCaseSource("TestFiles")]
public void StageOneTest(String file)
{
}

where one test method checks a specific thing and you run that test for every file. In that case you’d just have to remove the attributes and call this method from your new tests.

I also get the feeling that you want your tests to execute in a specific order and the different test runners execute your test in different orders. I am not aware of any way to do that and generally speaking you shouldn’t. No test should depend on another test having succeeded.

Assuming your tests are somewhat time intensive and you don’t want to run all of the “later” tests if an “earlier” test already failed you can consider putting them in appropriate categories like: “Preliminary”, “Stage1”, etc. Then you can execute one category after the other and see where the first fails.

I hope this helps you.