Question

I have a rules engine, which supports two modes of operations:

1. Compilation into a C# program and linked into the engine
2. Parsing into a reverse-polish stack-based instructions and interpreted

The rules are simple arithmetical expressions with function calls (max, min, sin, cos etc.)

I would have assumed the compiled version (i.e. #1) to be much faster than the interpreted version (i.e. #2) — in fact, that’s the main reason for having a compiled mode in the first place. However, my speed tests showed otherwise.

Compiled Version

Action<double>[] Rules = new[] { calc1, calc2, calc3 ... };
double[] v = new double[...];   // Variables

void calc1(double arg) { v[3]=v[12]+v[15]/v[20] };   // "x3=x12+x15/x20"
void calc2(double arg) { ... };
   :
// Start timer now
Rules.AsParallel().ForAll(r => r(...));
// End timer

Interpreted Version

Expression[] Rules = ...
// Each rule is already parsed into an Expression object, which is a set of
// reverse-polish stack-based instructions.
// For example, "x3=x12+x15/x20" will be parsed to:
//     [ Push(12), Push(15), Push(20), Divide(), Add() ]
// Start timer now
Rules.AsParallel().ForAll(r => r.Evaluate(...));
// End timer

Here, “Expression” is part of a third-party library that parses a simple string into a simple set of reverse-polish stack-based instructions, which can then be interpreted. It is not the expression-tree object in LINQ — just for clarification.

Note: don’t worry about concurrency since in the real code, I sort the rules by “layers” and calculate the layers sequentially, which each layer only depending on values calculated in previous layers. Both modes have the exact same layers structure.

The Results

Shockingly, the interpreted version runs MUCH faster than the compiled version, average a factor of 4x! In other words, the compiled version took 0.3s to run through around 1,200 rules, while the interpreted version took on average 0.08-0.1s.

My computer is a so-so dual-core Core2.

I am using .NET 4.0, Visual Studio 10.

Performance is similar in Debug or Release builds.

My Question

What can be causing the significant slowdown in the compiled mode?

NOTE: I HAVE POSTED ONE POSSIBLE ANSWER

Answer 1

.NET is a JIT-compiled environment, so the more code to JIT-compile, the slower it is. It can be that 1,200 methods are JIT-compiled at run-time at the point of execution, vs. in the interpreted mode only the interpreter is JIT-compiled, once. I may be seeing extra JIT times in my loops for the compiled mode.

Experiment:

1. Run each mode 5 times (just to complete any JIT compilation and fill caches)
2. Time 50 runs, take the average

Results:

1. Compiled mode: 1.6ms per run
2. Interpreted mode: 5.3ms per run

Observations:

It appears that a large amount of time is spent during the FIRST run of the compiled mode.

The SECOND run of the compiled mode is already similar in speed to the interpreted mode.

The interpreted mode does not speed up significant with the number of runs.

Thus suggesting my point of view that the rule code is JIT-compiled during the first run.