At my previous employer we used a third party component which basically was just a DLL and a header file. That particular module handled printing in Win32. However, the company that made the component went bankcrupt so I couldn’t report a bug I’d found.
So I decided to fix the bug myself and launched the debugger. I was surprised to find anti-debugging code almost everywhere, the usual IsDebuggerPresent, but the thing that caught my attention was this:
; some twiddling with xor ; and data, result in eax jmp eax mov eax, 0x310fac09 ; rest of code here At the first glance I just stepped over the routine which was called twice, then things just went bananas. After a while I realized that the bit twiddling result was always the same, i.e. the jmp eax always jumped right into the mov eax, 0x310fac09 instruction. I dissected the bytes and there it was, 0f31, the rdtsc instruction which was used to measure the time spent between some calls in the DLL.
So my question to SO is: What is your favourite anti-debugging trick?
My favorite trick is to write a simple instruction emulator for an obscure microprocessor.
The copy protection and some of the core functionality will then compiled for the microprocessor (GCC is a great help here) and linked into the program as a binary blob.
The idea behind this is, that the copy protection does not exist in ordinary x86 code and as such cannot be disassembled. You cannot remove the entire emulator either because this would remove core functionality from the program.
The only chance to hack the program is to reverse engineer what the microprocessor emulator does.
I’ve used MIPS32 for emulation because it was so easy to emulate (it took just 500 lines of simple C-code). To make things even more obscure I didn’t used the raw MIPS32 opcodes. Instead each opcode was xor’ed with it’s own address.
The binary of the copy protection looked like garbage-data.
Highly recommended! It took more than 6 month before a crack came out (it was for a game-project).