Question

I’ve been practicing detours using the same approach as Microsoft Detours (replace the first five bytes with a jmp and an address). More recently I’ve been reading about detouring by modifying the virtual table. I would appreciate if someone could shed some light on the subject by mentioning a few pros and cons with this method compared to the one previously mentioned!

I’d also like to ask about patched vtables and objects on the stack. Consider the following situation:

// Class definition
struct Foo
{
 virtual void Call(void) { std::cout << "FooCall\n"; }
};

// If it's GCC, 'this' is passed as the first parameter
void MyCall(Foo * object)
{
 std::cout << "MyCall\n";
}

// In some function
Foo * foo = new Foo; // Allocated on the heap
Foo foo2; // Created on the stack

// Arguments: void ** vtable, uint offset, void * replacement
PatchVTable(*reinterpret_cast<void***>(foo), 0, MyCall);

// Call the methods
foo->Call(); // Outputs: 'MyCall'
foo2.Call(); // Outputs: 'FooCall'

In this case foo->Call() would end up calling MyCall(Foo * object) whilst foo2.Call() call the original function (i.e Foo::Call(void) method). This is because the compiler will try to decide any virtual calls during compile time if possible (correct me if I’m wrong). Does that mean it does not matter if you patch the virtual table or not, as long as you use objects on the stack (not heap allocated)?

Answer 1

Stack vs. heap doesn’t matter – what matters is that the compiler knows the type of the object at compile time. The following would probably generate the same results unless the optimizer is being very clever:

Foo foo2; // Created on the stack
Foo * foo = &foo2; // Also on the stack, in fact the same object

Since the type of foo2 is known, the compiler can call the function directly without looking in any vtable. It can’t do the same for foo because a pointer can point to derived objects also.