Question

I’ve been pushing my Google Fu to the limits trying to find the most recommended / stable setup for doing TDD + CI for Windows Phone applications. Can anyone who has successfully been doing this point me in the right direction?

Here’s what I want to be able to do (if it’s possible):

• Write unit tests for view models and application services that don’t require phone functionality
• Execute tests directly in Visual Studio via Resharper or TD.NET
• Execute the unit tests from the command line with XML out, without launching the emulator
• Preferably be resiliant (as far as third party libraries go) to SDK updates

Since I’d like to keep this question as a resource to others looking for the same thing, here’s what I’d prefer answers to avoid:

• Open source ports of projects that are either incomplete or abandoned
• Projects that are only available as an attachment on someone’s blog

I’d also like to get full BDD-style acceptance tests going, but that’s another issue entirely.

Answer 1

I’m adding this answer as Community Wiki so that others may modify it to keep it up to date.

Unit Testing

Intent: To run isolated (no phone functionality), fast-executing tests often from both the IDE and Continuous Integration server without requiring the emulator (eg. TDD View Models)

The method I’ve seen recommended in a number of presentations involves referencing your source files in a .NET 4 project and running the tests against those (referencing the desktop equivalents of the assemblies). If your code doesn’t use any APIs that are different to the desktop BCL and you can deal with keeping the reference project up to date (new files aren’t added automatically) than that should be sufficient.

Otherwise, you can follow the steps below to execute code that references WP7 assemblies in the desktop CLR:

1. Add a tests Silverlight 4 Class Library (not a Silverlight for Windows Phone project) to your solution
2. In the class library, set Copy Local for all the framework references except mscorelib (basically System.* and Microsoft.*) to true
3. Add a reference from this test libary to NUnit.Silverlight.Framework.dll and NUnit.Silverlight.Compatibility.dll from the NUnit-Silverlight project
4. Add a reference from the test libary to your main Windows Phone project, ignoring the version warning.
5. Run the tests from in the IDE using ReSharper (tested) or TestDriven.NET
6. Run the tests from the command line using the standard .NET 2.0 nunit-console.exe from the latest NUnit release, passing in /framework=v4.0.

(Many of the above workarounds are required because WP7 uses SL3. Once Mango is released with the SL4 runtime, it should be a cleaner setup)

Integration Testing

Intent: To run longer-running tests that interact with resources external to the code (like phone features and web services) on the emulator, both on demand and on the CI server

UPDATE Unit Testing Windows Phone 8 applications will be official supported in Visual Studio 2012 Update 2, including VS integration and command line support. These tests run in the emulator, so I’ve included it under Integration Tests.

This is not currently supported by the WP7 port of the Silverlight test framework (and that only ships as a download from a blog).

In the meantime, I have created a codeplex project that adds an MSBuild task that launches the emulator and collates the results into an XML file. The simplest method installation is to add the wp7-ci NuGet package.

NOTE: Installing the WP7 SDK on Windows Server requires modifying the installer configuration and is not supported, but works well.

Acceptance/System Testing

Intent: To run end-to-end automated tests that interact with the phone’s UI on the emulator, both on demand or on the CI server

Expensify’s (poorly named for SEO) Windows Phone Test Framework supports writing UI automation tests from a host PC using SpecFlow.