Question

Background:

I wrote a test for a new class ( in OO System for Mathematica ). I ran the test using Workbench and I expected a failure in the MUnit test-report because the class does not exist yet ( test first, code later method ).

The test is:

 Test[
     clist = loadClass["UList"]         
     ,
     Class["UList`"]
     ,
     TestID->"UListTest-20110730-loadClass"
 ]

MUnit gave no report at all, MUnit did not get control, it ended prematurely, this was reported in the Console window in Workbench.

 -------------------
 Start Run Session
 Connected
 <<X3/Kernel/init.m
 Initialization Complete
 Running Test File: X3/Tests/UListTest.mt

 {Get::noopen}
 Error -> Exit
 End Session
 -------------------

The message {Get::noopen} occurs when you try to load a non-existing class. I would like to catch this in MUnit and report the testcase as failed.

Question:

How can I trap a situation like the one as described above ( i.e. something unexpected goes wrong ) in MUnit ? ( Deciding if it is an error or test failure should be handled programmatically in the test code. )

Answer 1

Ok, there are several parts here.

First, I set up a WB project to test your input, and while I do get the same error messages when running the unit test, I do not get the kernel hang – rather I either get the test to fail (if I do not wrap the test code in Quiet – see below for the explanation), or it exits prematurely (for the reason, also see below). But note that I changed the oosys' context name (also see below).

One thing to check is that in the WB project, your packages’ contexts should be sub-contexts of the main context of your application. Therefore, oosys' becomes (in my case) OOSystemTests'oosys'. This should be reflected both in the package itself, and in the package imports (calls to Needs ets). For example, your version of oosys will start as BeginPackage["OOSystemTests'oosys'"]; (change single quotes to context marks).

Next, one thing which is good to know is that messages produced during the run of the unit test are by default considered an error, so the test will fail. You can specify that the messages are expected, to avoid that (see this page for details), or you can wrap your test code in Quiet). You should do either of this to get past through it, to the end of your test (I wrapped the test code in Quiet).

Once you do that, your test runs, but MUnit reports that the kernel exits prematurely. And this brings us back to the application, OO System. To my mind, it does contain interesting ideas, but it falls short of being a professionally written mma extension package, on several grounds. First, the reason why it exits prematurely: if you look at loadClass implementation, you will notice the call to checkClass function, inside which you can see a number of calls to Quit[]. I don’t know whether or not I should explain why this is a wrong way of doing it, but I’ve said quite a bit about mma error-reporting techniques before, and calling Quit[] inside a lower-level function is not listed there, for a reason. This is not even mma-specific: it is a standard wisdom in software engineering that the lower-level functions normally should not make decisions of the scale that only high-level functions address (such as exiting an application). Another place which I noticed upon a really superficial reading of the code is in the newContext function, where the oosys' context is listed explicitly as string (hard-coded), which means a possibility of breaking whenever the context name is changed (say, as in our case). The way debugging is organized also does not look very exciting to me. Perhaps, there are other things, I did not read very carefully.

I hope I gave enough pointers so that you can make it through the MUnit with the desirable outcome, perhaps modifying some of the OO System source code in the process (if its license allows that). In this regard, I would repeat an advice I already gave to you: if you want to use it really productively, read the code carefully, take the good ideas (which I am sure there are there), and implement your own object model in the way you want it, so that you control all the code.