Question

I have a post build event that runs nunit-console to execute some tests. some of those tests fail, which in turn causes the post build event to fail (exit code equals failure count). I would like the post build event to succeed, always, no matter the exit code of nunit-console. How do I do this? I have tried adding ‘exit 0’ statements or calling a batch script that runs the tests and then does ‘exit 0’ but that doesn’t seem to prevent the post build event from failing (exit code -1).

UPDATE: as I mentioned earlier, using exit 0 (and exit /b 0) did not work. so unfortunately, even though the answer suggested by Hans Passant works in that specific scenario (failed copy), it does not work in this scenario (failed nunit):

"C:\libraries\nunit\nunit-2.5.7-net-2.0\bin\nunit-console.exe" $(TargetPath) /nologo /nodots /timeout=1000 /noshadow
exit /b 0

maybe I should mention that $(TargetPath) refers to a location that is different from the path of the nunit-console executable (C:\Snaps…\myproject.nunit.dll), but upto now that doesn’t appear to cause any harm.

Answer 1

well, finally got round to fixing this. as it turns out errorlevels had nothing to do with this specific issue. instead, the format of the text that ends up in the Visual Studio Output window appears to be paramount…

the failed NUnit tests showed up in the Visual Studio Error List window with the Description of the item set to the namespace and method name of the NUnit error message and the File set to the cryptic string ‘EXEC’. if you open the Output window that same string ‘EXEC’ is also present at the start of the NUnit error message. but it’s not part of the original NUnit error message. I added a man-in-the-middle to see what’s going on, like this:

    static void Main(string[] args)
    {
        Console.WriteLine("args: " + string.Join(" ", args));
        Process p = new Process()
        {
            StartInfo = new ProcessStartInfo(@"C:\libraries\nunit\nunit-2.5.7-net-2.0\bin\nunit-console.exe", string.Join(" ", args))
            {
                UseShellExecute = false,
                RedirectStandardOutput = true
            }
        };
        int lineNumber = 0;
        p.OutputDataReceived += (s, e) =>
        {
            //if (!Debugger.IsAttached && !string.IsNullOrWhiteSpace(e.Data) && e.Data.Contains("Test Error"))
            //{
            //    Debugger.Launch();
            //}
            //string line = e.Data;
            string line = string.IsNullOrWhiteSpace(e.Data) ? string.Empty : e.Data.Replace("Test Error", "Test Critical Failure");
            Console.WriteLine("[" + (++lineNumber) + "] " + line);
        };
        p.Start();
        p.BeginOutputReadLine();
        p.WaitForExit();
        Console.WriteLine("nunit exited with code " + p.ExitCode);
        Environment.Exit(0);
    }

I called this program instead of nunit from the post-build event and sure enough there was the NUnit error message without the string ‘EXEC’ when debugging. also, the string ‘EXEC’ turned up even before the line numbers that I was now adding to the test output. when I skipped all of the lines with the string ‘Test Error’, or replaced ‘Test Error’ with something else like ‘Test Critical Failure’, the build of the project succeeded. but when I added the following to the post-build event of the project it failed again and the description of the item in the Visual Studio Error List window was set to ‘foo.bar’:

echo Test Error : foo.bar

so either Visual Studio or some extension appear to intervene with the build and make it fail as soon as a string with the format ‘Test Error : bla.bla’ shows up in the Visual Studio Output window. the entire post-build event then reportedly fails with exit code -1 even if the last command of the post-build event sets the errorlevel to some other value. now to be fair I’m not sure if this applies to the standard Visual Studio 2010 edition or some extension so just to be safe this is my whole environment:

Microsoft Visual Studio 2010 Version 10.0.40219.1 SP1Rel Microsoft
.NET Framework Version 4.0.30319 SP1Rel
Installed Version: Professional

Microsoft Office Developer Tools
Microsoft Visual Basic 2010
Microsoft Visual C# 2010
Microsoft Visual C++ 2010
Microsoft Visual F# 2010
Microsoft Visual Studio 2010 Team Explorer
Microsoft Visual Web Developer 2010
Add Reference Dialog Plus! 1.0
AlignAssignments 1.0
AutoBraceComplete 1.0
ClearCase Search 1.0
Crystal Reports Templates for Microsoft Visual Studio 2010
Devart Code Compare 2.70.3
Document Well 2010 Plus 1.0.10916.0
Microsoft Visual Studio 2010 Professional – ENU Service Pack 1
(KB983509)
Microsoft Visual Studio 2010 SharePoint Developer Tools 10.0.40219
Microsoft.VisualStudio.QuickAccess.Package 1.0
OptionsPageImpl 1.0
PowerCommands for Visual Studio 2010 1.0
QuickFind 1.0
Rational ClearCase
Visual Nunit 1.0
VSCommands 2010 3.8.0.0
FeiLoader
ObjectWizard

ok, so there’s a workaround using the man-in-the-middle, but if there’s anyone out there who knows some Visual Studio or extension setting or fix that will solve this issue, or a better alternative to my solution, then I’m all ears!

UPDATE: After disabling all extensions and add-ins the problem persists. it’s starting to look like a native Visual Studio peculiarity.