Question

I’ve been trying to wrap my head around this issue: How to create a unit test that tests if a function aborts because a Debug.Assert (from System.Diagnostics) fails, marking it as passed when it does this and as failed if it doesn’t.

I knowNUnit has the feature [ExpectedException(typeof( ArgumentException ) )], but I cannot seem to find out what kind of exception it is from the MSDN website. Intuition would say it might be something like an AssertionException, and that one does exist… but is part of the NUnit framework. I guess that’s the exception NUnit assert throw around. I might be able to just nuke it by using:

[ExpectedException(typeof(Exception))]

But this gives rise to the problem that the standard windows debugging window shows up. In my search I came across ways to remove this window from showing up at all, but this feels like taking a butcheringknife to the surgerytable where you normally use a scalpel. Because I do want to be able to see this window when something unexpected happens, when I would execute my program.

I guess there is the workaround of replacing the Debug.Assert method with the NUnit counterpart (I’m still early in my project, so it’s not a too big of a refactoring), but I assume a lot of programmers stick with Debug.Assert functionality as it’s standard in .NET.

As such, I’d like to know how to ‘assert’ Debug.Assertion failures, without having to ‘murder’ the Windows debugging screen from my project?

To have an concrete example from a contract in my code, there is a sample below. For those it might seem familiar, it is the To-Wound table from Warhammer 40K tabletop wargame written as a function.

static public int assaultToHit(int _attacker_WeaponSkill,
            int _defender_WeaponSkill)
        {
            //Preconditions
            Debug.Assert(_attacker_WeaponSkill >= 1 && _attacker_WeaponSkill <= 10,
                "Weapon Skill stat must be in range [1,10]");
            Debug.Assert(_defender_WeaponSkill >= 1 && _defender_WeaponSkill <= 10,
                "Weapon Skill stat must be in range [1,10]");

            int target;
            if (_attacker_WeaponSkill > _defender_WeaponSkill)
            {
                target=3;
            }
            else if (_defender_WeaponSkill >= (_attacker_WeaponSkill + _attacker_WeaponSkill + 1))
            {
                target=5;
            }
            else
            {
                target=4;
            }

            //postconditions
            Debug.Assert(target >= 3 && target <= 5,
                "To hit target for assault must be in range [3,5]");

            return target;
        }

The function to test the preconditions would be in the line of something like this:

    [TestCase(-1,2)]
    [TestCase(1, -2)]
    [TestCase(-1, -2)]
    [TestCase(11, 2)]
    [TestCase(1, 20)]
    [TestCase(11, 20)] 
    [ExpectedException(typeof(Exception))]
    public void testContract_AssaultToHit(int _attacker_weaponskill, 
        int _defender_weaponskill)
    {
        Warhammer40kRules.assaultToHit(_attacker_weaponskill, 
            _defender_weaponskill);
    }

Answer 1

From https://stackoverflow.com/a/117247/605538 one can find a recommendation to use exceptions for public interfacing, while using assertions to verify internal code. When unit-testing preconditions of a function (and a similar thought can be applied for post-conditions, altough I personally would prefer to use assertions there), one can therefore be recommended to use exceptions instead of using assertions.

Using exceptions instead of would result in a sample like this:

static public int assaultToHit(int _attacker_WeaponSkill,
        int _defender_WeaponSkill)
    {
        //Preconditions
        if(!(_attacker_WeaponSkill >= 1 && _attacker_WeaponSkill <= 10))
        {
            throw new ArgumentOutOfRangeException("Attackers WeaponSkill must be in range [1,10]");
        }
        if(!(_defender_WeaponSkill >= 1 && _defender_WeaponSkill <= 10))
        {
            throw new ArgumentOutOfRangeException("Defenders WeaponSkill must be in range [1,10]");
        }

        ...
        //rest unchanged
    }

Coupled with the following NUnit test:

[TestCase(-1,2)]
[TestCase(1, -2)]
[TestCase(-1, -2)]
[TestCase(11, 2)]
[TestCase(1, 20)]
[TestCase(11, 20)] 
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void testContract_AssaultToHit(int _attacker_weaponskill, 
    int _defender_weaponskill)
{
    Warhammer40kRules.assaultToHit(_attacker_weaponskill, 
        _defender_weaponskill);
}

Sidebar [12 June 2014]:
I’ve been recently coming to the opinion that you should not have to test precondition violations in the context of Design by Contract. If you design with DbC in mind, you basically state the following: “If you call a method and meet its preconditions, you are guaranteed a certain behavior. If you do not meet the method’s preconditions, you can expect undefined behavior.”. In the broadest sense of the term ‘undefined behavior’, this means that you cannot expect any sort of state. You might get exceptions, maybe nothing happens at all and maybe your hard disk gets formatted (well… you get the point 😉 ). As such, you cannot test for ‘undefined behavior’.

What you can test, is your defensive programming that makes sure that a precondition failure does not cause the function to operate, for example by throwing an Exception. This behavior is testable.