To be short the problem is this. I’m writing a kernel-mode Windows driver, that gets notified when a kernel-mode DLL (or other executable module) is loaded. In some situations I have to intercept the DLL entry point routine. That is, override it so that my routine is called first, and then I may pass control to the original entry point.
On 32-bit (x86 to be exact) there was no problem to do that. I get the module base mapping address, which actually begins with the standard PE header (used by Windows executables). There there’s an RVA (address relative to the image base) of the DLL entry point. I just override it by the address of my routine minus the module base address. Voila!
Now, the things are more complicated in 64-bit. The problem is that RVAs are still 32-bit integers. Such RVAs cover the address range starting from the image base address and ending with 4GB offset. There’s no problem to reference any symbol inside the same executable module (assume it doesn’t exceed 4GB size), however this imposes problems for cross-module interception. Naturally my executable module and the one that I’m trying to hook don’t have to fall into the same 4GB range, hence there’s a problem.
Temporarily I solved this by overriding the original routine prolog code by an unconditional jmp into my code. This takes 12 bytes on 64-bit platform. Then, in order to call the original code from my routine I restore the overridden 12 bytes (means – I save them before overwriting).
So far – no problems. But now the things are changing, and I’ll have to support multi-threaded access to the entry point routine (please don’t ask why, it’s related to multi-session DLL loaded into a so-called “user space”, separate for each terminal session).
One of the solutions is to use a global lock, but I’d like to avoid this.
I know about the so-called “trampoline functions”, but I’d like to avoid this as well. Doing this requires a run-time decoding of the function prolog code to properly identify the instruction boundary and possible branching.
Recently I thought about another idea. What if I could find some “unneeded” portion of the original DLL, which is at least 12 bytes length (size of mov RAX addr + jmp RAX). Then this portion could be overridden by jmp into my hands. Then the entry point RVA could be set to this portion!
All that is needed for this to work is the appropriate portion that can be overwritten. I suppose there is such a possibility, since the PE header contains a lot of historical fields that are no more used for decades.
Is this idea worth trying, or is this a well-known technique? Andy other suggestions?
Thanks in advance.
You have several options. Unfortunately, you can only choose 2 out of these 3: 100% solid; easy to implement; cheap.
There is very high likelihood that at the end of the .TEXT section you’ll find unused space. This is because Windows maps image sections into memory in chunks of 4k, and typically the .text section isn’t an exact multiply.
Another easy to implement is to use the PE header. An area very safe to override is the DOS stub. The problem there is that there is no guarantee the PE header is in the same section as the entry routine (Microsoft linker put it in same section though, don’t know about GNU or the others).
Another easy but will work only for system DLL’s is to do what ‘Hot Patching’ is doing, and reuse the 15 bytes set to ‘nop’ in front of each function, and the ‘mov edi,edi’ instruction. This is the case for all DLL’s released with Windows, to support Hot Patching.
The reliable but hard option is to do what @David Heffeman suggestions. This technique is called ‘landing function’ where you copy the first 12 bytes into a landing function, which will then jmp to the original function.
The easy, and reliable option is using MS Detour. Microsoft Detour is a product from Microsoft Research that does exactly that, and works great, and it is supported, and it takes care of bunch of corner cases and race conditions that may pop (along with other stuff), and its x86 version is open source. The downside is that a commercial usage is very expensive – last time I checked it was 10k.