Question

Is is posable to disable selected automated tests at runtime?

I’m using VSTS and rhino mocks and have some intergation tests that require an external dependancy to be installed (MQ). Not all the developers on my team have this installed.

Currently all the tests that require MQ inherit from a base class that checks if MQ is installed and if is not sets the test result to inconclusive. This works as it stops the tests from running, but marks the test run as unsuccseessful and can hide other failures.

Any ideas?

Answer 1

Finaly got aroung to figuring this out, here is what I did.

In each of my test classes (or methods if only a small number of tests in the class require MQ) that had MQ dependancies I added the following to the class (or method) decleration

#if !RunMQTests
    [Ignore]
#endif

This disables the tests unless you have the conditional compalation symbol RunMQTests decleared, this symbol is not defined in the project files so the tests are disabled by default.

To enable these tests with out developers having to remember if they have MQ installed and adding or removing the conditional compalation symbol, I created a custom build task that will tell us if MQ is installed.

/// <summary>
/// An MSBuild task that checks to see if MQ is installed on the current machine.
/// </summary>
public class IsMQInstalled : Task
{
    /* Constructors removed for brevity */

    /// <summary>Is MQ installed?</summary>
    [Output]
    public bool Installed { get; set; }

    /// <summary>The method called by MSBuild to run this task.</summary>
    /// <returns>true, task will never report failure</returns>
    public override bool Execute()
    {
        try
        {
            // this will fail with an exception if MQ isn't installed
            new MQQueueManager();
            Installed = true;
        }
        catch { /* MQ is not installed */ }

        return true;
    }
}

Then we just need to hook this up into the build process by adding the task to the top of the test project file.

<UsingTask TaskName="IsMQInstalled" AssemblyFile="..\..\References\CustomBuildTasks.dll" />

And call the new custom task in the BeforeBuild target and set the conditional compalation symbol if this machine has MQ installed.

<Target Name="BeforeBuild">
  <IsMQInstalled>
    <Output TaskParameter="Installed" PropertyName="MQInstalled" />
  </IsMQInstalled>
  <Message Text="Is MQ installed: $(MQInstalled)" Importance="High" />
  <PropertyGroup Condition="$(MQInstalled)">
    <DefineConstants>$(DefineConstants);RunMQTests</DefineConstants>
  </PropertyGroup>
</Target>

This lets users that have MQ installed to run our MQ intergration tests, while not failing test runs for users that don’t.