Question

I’ve recently been working on detouring functions (only in Linux) and so far I’ve had great success. I was developing my own detouring class until I found this. I modernized the code a bit and converted it to C++ (as a class of course). That code is just like any other detour implementation, it replaces the original function address with a JMP to my own specified ‘hook’ function. It also creates a ‘trampoline’ for the original function.

Everything works flawlessly but I’d like to do one simple adjustement. I program in pure C++, I use no global functions and everything is enclosed in classes (just like Java/C#). The problem is that this detouring method breaks my pattern. The ‘hook’ function needs to be a static/non-class function.

What I want to do is to implement support for _thiscall hooks (which should be pretty simple with the GCC _thiscall convention). I’ve had no success modifying this code to work with _thiscall hooks. What I want as an end result is something just as simple as this; PatchAddress(void * target, void * hook, void * class);. I’m not asking anyone to do this for me, but I would like to know how to solve/approach my problem?

From what I know, I should only need to increase the ‘patch’ size (i.e it’s now 5 bytes, and I should require an additional 5 bytes?), and then before I use the JMP call (to my hook function), I push my ‘this’ pointer to the stack (which should be as if I called it as a member function). To illustrate:

push 'my class pointer'
jmp <my hook function>

Instead of just having the ‘jmp’ call directly/only. Is that the correct approach or is there something else beneath that needs to be taken into account (note: I do not care about support for VC++ _thiscall)?

NOTE: here’s is my implementation of the above mentioned code: header : source, uses libudis86

Answer 1

I tried several different methods and among these were JIT compile (using libjit) which proved successful but the method did not provide enough performance for it to be usable. Instead I turned to libffi, which is used for calling functions dynamically at run-time. The libffi library had a closure API (ffi_prep_closure_loc) which enabled me to supply my ‘this’ pointer to each closure generated. So I used a static callback function and converted the void pointer to my object type and from there I could call any non-static function I wished!