Question

I’m implementing a simplistic JIT compiler in a VM I’m writing for fun (mostly to learn more about language design) and I’m getting some weird behavior, maybe someone can tell me why.

First I define a JIT “prototype” both for C and C++:

#ifdef __cplusplus 
    typedef void* (*_JIT_METHOD) (...);
#else
    typedef (*_JIT_METHOD) ();
#endif

I have a compile() function that will compile stuff into ASM and stick it somewhere in memory:

void* compile (void* something)
{
    // grab some memory
    unsigned char* buffer = (unsigned char*) malloc (1024);

    // xor eax, eax
    // inc eax
    // inc eax
    // inc eax
    // ret -> eax should be 3
    /* WORKS!
    buffer[0] = 0x67;
    buffer[1] = 0x31;
    buffer[2] = 0xC0;
    buffer[3] = 0x67;
    buffer[4] = 0x40;
    buffer[5] = 0x67;
    buffer[6] = 0x40;
    buffer[7] = 0x67;
    buffer[8] = 0x40;
    buffer[9] = 0xC3; */

    // xor eax, eax
    // mov eax, 9
    // ret 4 -> eax should be 9
    /* WORKS!
    buffer[0] = 0x67;
    buffer[1] = 0x31;
    buffer[2] = 0xC0;
    buffer[3] = 0x67;
    buffer[4] = 0xB8;
    buffer[5] = 0x09;
    buffer[6] = 0x00;
    buffer[7] = 0x00;
    buffer[8] = 0x00;
    buffer[9] = 0xC3; */


    // push ebp
    // mov ebp, esp
    // mov eax, [ebp + 6] ; wtf? shouldn't this be [ebp + 8]!?
    // mov esp, ebp
    // pop ebp
    // ret -> eax should be the first value sent to the function
    /* WORKS! */
    buffer[0] = 0x66;
    buffer[1] = 0x55;
    buffer[2] = 0x66;
    buffer[3] = 0x89;
    buffer[4] = 0xE5;
    buffer[5] = 0x66;
    buffer[6] = 0x66;
    buffer[7] = 0x8B;
    buffer[8] = 0x45;
    buffer[9] = 0x06;
    buffer[10] = 0x66;
    buffer[11] = 0x89;
    buffer[12] = 0xEC;
    buffer[13] = 0x66;
    buffer[14] = 0x5D;
    buffer[15] = 0xC3;

    // mov eax, 5
    // add eax, ecx
    // ret -> eax should be 50
    /* WORKS!
    buffer[0] = 0x67;
    buffer[1] = 0xB8;
    buffer[2] = 0x05;
    buffer[3] = 0x00;
    buffer[4] = 0x00;
    buffer[5] = 0x00;
    buffer[6] = 0x66;
    buffer[7] = 0x01;
    buffer[8] = 0xC8;
    buffer[9] = 0xC3; */

    return buffer;
}

And finally I have the main chunk of the program:

int main (int argc, char **args)
{
    DWORD oldProtect = (DWORD) NULL;
    int i = 667, j = 1, k = 5, l = 0;

    // generate some arbitrary function
    _JIT_METHOD someFunc = (_JIT_METHOD) compile(NULL);

    // windows only
#if defined _WIN64 || defined _WIN32
    // set memory permissions and flush CPU code cache
    VirtualProtect(someFunc,1024,PAGE_EXECUTE_READWRITE, &oldProtect);  
    FlushInstructionCache(GetCurrentProcess(), someFunc, 1024);
#endif

    // this asm just for some debugging/testing purposes
    __asm mov ecx, i

    // run compiled function (from wherever *someFunc is pointing to)
    l = (int)someFunc(i, k);

    // did it work?
    printf("result: %d", l);

    free (someFunc);
    _getch();

    return 0;
}

As you can see, the compile() function has a couple of tests I ran to make sure I get expected results, and pretty much everything works but I have a question…

On most tutorials or documentation resources, to get the first value of a function passed (in the case of ints) you do [ebp+8], the second [ebp+12] and so forth. For some reason, I have to do [ebp+6] then [ebp+10] and so forth. Could anyone tell me why?

Answer 1

Your opcodes look suspicious: they’re full of 0x66 and 0x67 address/data size override prefixes, which (in a 32-bit code segment) will turn 32-bit operations into 16-bit ones. e.g.

buffer[0] = 0x66;
buffer[1] = 0x55;
buffer[2] = 0x66;
buffer[3] = 0x89;
buffer[4] = 0xE5;
...

is

push bp
mov  bp, sp

rather than

push ebp
mov  ebp, esp

(which seems to explain the observed behaviour: pushing bp decrements the stack pointer by 2 instead of 4).