Question

I have a large acceptance test suite that runs the source application many thousands of times, creating an NCOVER report for each run. After each test, it merges the generated code coverage report into a large “master” coverage report for the whole application.

My worry here is that I’m running into a Shlemiel the Painter problem, as the merge operation is parsing the large coverage file for every test.

I can pass more than one coverage file to NCover.Reporting.Exe, which actually does the merge, but when I try to pass them all at once I’m running into operating system limits on command line length.

Does NCover.Reporting provide some sort of input coverage report listing that I can save to a file, and have it merge all the reports in one go?

Answer 1

Well, what I settled on was kind of like Stephen Ward’s answer. I put all the coverage files into a directory and ran NCover.Reporting.exe Path\To\My\Directory\*.xml //s Merged.nccov.

He’s not getting the checkmark though because his answer telling me I could not use a wildcard on the commandline is incorrect; in fact, the wildcard is the only way NCover would recognise the XML files in that directory. I think he’s used to the Unix world (considering there was some Unix FUD in his answer), where the shell takes responsibility for handling wild cards. The windows shell doesn’t attempt to expand wildcards, so we didn’t run into length limits with that.

With this solution though, I was able to make our runs about 20% faster than the old merge-after-each-test method, which is good. (Coverage runs are still four times as long as non coverage runs, but we don’t need to run coverage for every test)

When I tried commondream’s answer, unfortuately NCover would crash on a regular basis. I don’t think it likes that the test system starts .NET 1, 1.1, 2.0, 3.0, 3.5, and 4.0 assemblies , as well as things like silverlight packages and the compact framework (Our product needs to be able to work correctly on all these kinds of assemblies). If you don’t need to be insane with running that sort of thing, then the //pn option he describes would probably work fine.

When I tried joe.feser’s answer, unfortunately I was unable to figure out the format of NCover’s reporting settings file. There’s ample documentation on the XML the reporting tool accepts; with the exception of the root XML node it looks for. It refused recognize any of the XML files I generated because the root node was named incorrectly; I assume it was validating against some form of DTD of which I was not aware. If someone would like to post an example file, please go ahead.

Hope this helps anyone with similar issues in the future 🙂