Question

I’d like to write a function that would have some optional code to be executed or not depending on user settings. The function is cpu-intensive and having ifs in it would be slow since the branch predictor is not that good.

My idea is making a copy in memory of the function and replace NOPs with a jump when I don’t want to execute some code. My working example goes like this:

int Test()
{
    int x = 2;
    for (int i=0 ; i<10 ; i++)
    {
        x *= 2;

        __asm {NOP}; // to skip it replace this
        __asm {NOP}; // by JMP 2 (after the goto)
            x *= 2; // Op to skip or not

        x *= 2;
    }
    return x;
}

In my test’s main, I copy this function into a newly allocated executable memory and replace the NOPs by a JMP 2 so that the following x *= 2 is not executed. JMP 2 is really “skip the next 2 bytes”.

The problem is that I would have to change the JMP operand every time I edit the code to be skipped and change its size.

An alternative that would fix this problem would be:

__asm {NOP}; // to skip it replace this
__asm {NOP}; // by JMP 2 (after the goto)
goto dont_do_it;
    x *= 2; // Op to skip or not
dont_do_it:
x *= 2;

I would then want to skip or not the goto, which has a fixed size. Unfortunately, in full optimization mode, the goto and the x*=2 are removed because they are unreachable at compilation time.

Hence the need to keep that dead code.

I’m using VStudio 2008.

Answer 1

You can cut the cost of the branch by up to 10, just by moving it out of the loop:

int Test()
{
    int x = 2;
    if (should_skip) {
        for (int i=0 ; i<10 ; i++)
        {
            x *= 2;
            x *= 2;
        }
    } else {
        for (int i=0 ; i<10 ; i++)
        {
            x *= 2;
            x *= 2;
            x *= 2;
        }
    }

    return x;
}

In this case, and others like it, that might also provoke the compiler into doing a better job of optimising the loop body, since it will consider the two possibilities separately rather than trying to optimise conditional code, and it won’t optimise anything away as dead.

If this results in too much duplicated code to be maintainable, use a template that takes x by reference:

    int x = 2;
    if (should_skip) {
        doLoop<true>(x);
    } else {
        doLoop<false>(x);
    }

And check that the compiler inlines it.

Obviously this increases code size a bit, which will occasionally be a concern. Whichever way you do it though, if this change doesn’t produce a measurable performance improvement then I’d guess that yours won’t either.