Question

Our codebase contains code in multiple languages, ranging from Python over C# to MATLAB and LaTeX. Currently we have unit tests in each individual language (using language-specific frameworks). This makes test automation cumbersome, especially collecting and checking all the different reports.

I am therefore looking for a test automation tool that

• is portable (at least Windows + Linux)
• is language-independent
• can be extended (custom reporting, additional languages, …)

Ideally the tool would connect via plugins to the existing tool-specific frameworks like Python’s unittest, C#’s NUnit, etc.

Is there such a tool? If not, how do you handle test scenarios like this?

Answer 1

Perhaps you’re looking for something like the Test Anything Protocol?

The Test Anything Protocol (TAP) is a protocol to allow communication between unit tests and a test harness. It allows individual tests (TAP producers) to communicate test results to the testing harness in a language-agnostic way.

It doesn’t avoid the need for language-specific tests, but it does make sure that your test results are all in a uniform format, so that you can build cross-langauge reports. It’s also fairly simple, so if your language of choice doesn’t have a TAP producer, it is not very difficult to write one (and maybe share it back to the world).