Question

I’m planning some experiments in symbolic execution of C code, using an off-the-shelf SMT solver, and wondering which solver to use; looking at e.g. the SMT contest entrants, and taking only the open-source systems, narrows it down to Beaver, Boolector, CVC3, OpenSMT, Sateen, Sonolar, STP, Verit; which is still a long list.

Trying to narrow it down a little further, I notice that some of the systems advertise the ability to handle bit vector arithmetic, whereas others only advertise the ability to handle general integer arithmetic. In principle, the former is correct for C, where variables are machine words, not unbounded integers. How much difference does it make in practice? What happens if you try to use a general integer system for this kind of job? Does one of the following scenarios apply?

1. A bit vector system is slightly more efficient, but you can use either, no problem.

2. You can use a general integer system with a bit of tweaking.

3. A general integer system is fine for signed int (because the result of overflow is undefined) but will give the wrong answer for unsigned.

4. A general integer system just isn’t correct for machine word arithmetic, and I can reduce my short list to only those systems that provide bit vector arithmetic.

5. Something else…?

I’ve tried to ask as specific a question as possible, but if anyone can suggest any other criteria for narrowing down the list, that would be great!

Answer 1

I’ve had good experience using STP for symbolic execution. STP was designed precisely for this task. Also, there have been a number of symbolic execution tools that have successfully used STP for this purpose, so there is reason to believe that STP doesn’t suck. I would definitely recommend STP to others as a default choice for this sort of experimentation.

However, I haven’t tried the other systems, so I don’t know how STP compares to them.

Personally, I see STP as the baseline and the default choice for this kind of application. So, if you only have time to try one solver, trying STP seems like a pretty reasonable choice.

If I had to guess, my guess would be that bit-vector arithmetic is important to support, because any large systems code is going to have a non-trivial amount of code that performs bitwise operations. Also, I’d suspect/worry some systems code may rely upon the behavior of unsigned arithmetic to wrap modulo 2ⁿ, and if you try to model it with integers, you will not get the semantics of C right (because, as you say, integers just aren’t correct for machine-word arithmetic) and consequently, if you try to use an integer-only solver, you may experience some difficulties. However, I have no hard evidence for either of these suspicions.

P.S. Z3 might also be a contender to add to your list to consider. (Do you really need your solver to be open-source, as long as it is free? I’d expect that a symbolic execution tool would use it only as a blackbox, without modification.)