Question

I’m trying to figure out how to do a code coverage testing in a distributed system – i.e. a system that consists of several executables that only do something meaningful together (and are actually started by one another). Has anyone tackled a similar problem wand what tools did you use?

Answer 1

You can obviously collect test coverage data on each distributed part, independently. Your first problem is that the distributed pieces may be coded in different languages (e.g., C++ and Java), which mean you need a test coverage tool for each language. If you only want to view the data from the distributed elements independently, this should be straightforward modulo the inconvenience of managing the multiple parts. Use the test coverage display of each test coverage tool to display coverage for its corresponding piece.

If the question is, “how do you collect/see integrated test coverage data for the whole system”, you need to collect test coverage data on the individual distributed pieces all at the same time you run a test. (Whether you are running unit or system tests doesn’t matter).

Then you need to compose those results into a single overview for the whole system.

Many tools won’t combine test coverage from different programs. So you likely have to handle this on your own. If all your distributed code elements are in the one language, you have the advantage of only having one representation of the test coverage data, but the disadvantage that it probably isn’t documented very well. Normally this data is viewed directly by a viewer and the test coverage tool supplier has little reason to document how the test coverage data is encoded. People often end up building ad hoc tools to combine this data, if they can figure out the representation.

If you have multiple languages, and multiple test coverage tools, you likely have multiple test coverage representations and the ad hoc solution gets a lot harder if you can do it at all.

Our Test Coverage tools have the nice property that they cover a wide variety of languages and dialects (C, C++, C#, Java, PLSQL, COBOL, PHP, …),
that they all use the same representation for test coverage vectors, and that the test coverage display tool provided will combine this information for you (UI selectable action). No guessing or ad hocery, just a view of all of the code in the set of languages (even if on different machines) with the complete test coverage for the entire distributed application.