Question

The problem appears when test case marked with “async” is failing. App is just crashing in that case, with an unhandled exception of TargetInvocationFail. As far as I understand, the test app is supposed to handle those exceptions, because in case of exception it should just mark respective case as failed. This is exactly what happens with the normal test case (without async / await stuff in it).

I’ve created an issue report for this as well, see http://phone.codeplex.com/workitem/10751. If you have same problem, please upvote the issue. And if you happen to know some workaround, please let me know here.

EDIT: As Stephen Cleary mentioned in the comment, the issue was caused by the fact that my test case procedure was async void, not async Task. I’d reformulate this question as follows then: why changing return type of test case changes the behavior of the exception handling?

Answer 1

A good general guideline is “avoid async void“. One reason for this is the difference in exception handling: async Task methods will place any exceptions on their returned Task, which can be observed when that Task is awaited. async void methods will raise their exceptions directly on the SynchronizationContext that was current at the time of the start of the async void method.

Another thing to keep in mind is that async is primarily a compiler transformation. If you (or anyone else) reflects over an async Task method at runtime, you’ll just see a method with a return type of Task; similarly, an async void method just has a return type of void.

So, when the test runner sees a method returning void (not knowing that it is an async void method), it executes it and sees it return without (directly) raising an exception, so it marks it as “passed”. Meanwhile, the exception thrown by that async void method is raised directly on the SynchronizationContext (most test runners, including MSTest, provide a thread pool SynchronizationContext), and this exception can cause the test run to report a non-specific error – or it could possibly be ignored if the test run completes quickly enough.

Modern test runners (including MSTest as of VS2012) understand async Task methods by detecting the return type of Task, and will wait for the Task to complete before considering the test method “finished” and marking it as passed (or failed, if the returned Task contains an exception).

I have examples of this behavior in MSTest on my blog (including screenshots showing both outputs of the race condition), but note that those blog entries are almost a year old and talk about async unit testing with VS2010. MSTest was updated with VS2012 so that it has reasonable behavior for async Task methods.