Question

I have a piece of C++ code (compiled with g++ under a GNU/Linux environment) that load a function pointer (how it does that doesn’t matter), pushes some arguments onto the stack with some inline assembly and then calls that function, the code is like :

unsigned long stack[] = { 1, 23, 33, 43 };

/* save all the registers and the stack pointer */
unsigned long esp;
asm __volatile__ ( "pusha" );
asm __volatile__ ( "mov %%esp, %0" :"=m" (esp));

for( i = 0; i < sizeof(stack); i++ ){
    unsigned long val = stack[i];
    asm __volatile__ ( "push %0" :: "m"(val) );
}

unsigned long ret = function_pointer();

/* restore registers and stack pointer */
asm __volatile__ ( "mov %0, %%esp" :: "m" (esp) );
asm __volatile__ ( "popa" );

I’d like to add some sort of

#ifdef _LP64
   // 64bit inline assembly
#else
   // 32bit version as above example
#endif

But i don’t know inline assembly for 64bit machines, anyone could help me?

Thanks

Answer 1

While it shouldn’t be much of a problem to call a function pointer with the appropriate arguments in inline assembly, I don’t think recoding this naively in x64 will help you, because the calling conventions to be used are very probably different (defaults for 32bit and 64bit linux are definitely different). Have a look here for details. So I guess, if you can get away without inline assembly in this case (see the other answer), it’ll be easier to port.

Edit: OK, I see you may have to use assembly. Here are some pointers.

According to Agner Fog’s document, linux x64 uses RDI, RSI, RDX, RCX, R8, R9 and XMM0-XMM7 for parameter transfer. This implies that in order to achieve what you want (disregarding floating-point use) your function will have to:

(1) save all registers that need to be saved (RBX, RBP, R12-R15): Set aside space on the stack and move these registers there. This will be somthing along the lines of (Intel syntax):

sub rsp, 0xSomeNumber1
mov [rsp+i*8], r# ; insert appropriate i for each register r# to be moved

(2) Evaluate the number of arguments you will have to pass by stack to the target function. Use this to set aside the required space on the stack (sub rsp, 0xSomeNumber2), taking into account 0xSomeNumber1 so that the stack will be 16-byte aligned at the end, i.e. rsp must be a multiple of 16. Don’t modify rsp after this until your called function has returned.

(3) Load your function arguments on the stack (if necessary) and in the registers used for parameter transfer. In my view, it’s easiest if you start with the stack parameters and load register parameters last.

;loop over stack parameters - something like this
mov rax, qword ptr [AddrOfFirstStackParam + 8*NumberOfStackParam]
mov [rsp + OffsetToFirstStackParam + 8*NumberOfStackParam], rax

Depending on how you set up your routine, the offset to the first stack parameter etc. may be unnceccessary. Then set up the number of register-passed arguments (skipping those you don’t need):

mov r9, Param6
mov r8, Param5
mov rcx, Param4
mov rdx, Param3
mov rsi, Param2
mov rdi, Param1

(4) Call the target function using a different register from the above:

call qword ptr [r#] ; assuming register r# contains the address of the target function

(5) Restore the saved registers and restore rsp to the value it had on entry to your function. If necessary, copy the called function’s return value wherever you want to have them. That’s all.

Note: the above sketch does not take account of floating point values to be passed in XMM registers, but the same principles apply.
Disclaimer: I have done something similar on Win64, but never on Linux, so there may be some details I am overlooking. Read well, write your code carefully and test well.