Question

I have tried many orderedtests and the .trx file always shows the wrong count?

for instance, if i had an orderedtest with 2 tests, the results look like this in the .trx file (results summary node):

<Counters total="3" executed="3" passed="3" error="0" failed="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0"/>

But there are only 2 tests!!! If i have 29 tests, it says 30 total, etc…

Any ideas?

Answer 1

I will place my money on the fact that the ordered test itself is also counted by MSTEST as a test that is run. This is because of the way it is structured:

1. Run Ordered test (test number 1), starts processing the inner tests in sequence recursively re-uses the standard mechanism for running any test.
2. Run first test in ordered test (test number 2)
3. Run second test in ordered test (test number 3)

So it always adds the parent ordered test container as a regular test being performed. This would also mean that if you run an ordered test (with to inner tests) from within an ordered test, your count would be 4 while actually only 2 test are functionally relevant and tested.

Personally what I find more disturbing, is that if not all tests in an ordered test are 100% successful (warnings, inconclusive) the ordered test always fails! Completely! Uncontrollable!

But that was an off topic frustration 🙂