Question

A while ago I posted this question regarding strange behavior I was experiencing in trying to step through a MASM program.

Essentially, given the following code:

; Tell MASM to use the Intel 80386 instruction set.
.386
; Flat memory model, and Win 32 calling convention
.MODEL FLAT, STDCALL
; Treat labels as case-sensitive (required for windows.inc)
OPTION CaseMap:None

include windows.inc
include masm32.inc
include user32.inc
include kernel32.inc
include macros.asm

includelib masm32.lib
includelib user32.lib
includelib kernel32.lib

.DATA
    BadText     db      "Error...", 0
    GoodText    db      "Excellent!", 0

.CODE
main PROC
        int 3
        mov eax, 6
        xor eax, eax
_label: add eax, ecx
        dec ecx
        jnz _label
        cmp eax, 21
        jz _good
_bad:   invoke StdOut, addr BadText
        jmp _quit
_good:  invoke StdOut, addr GoodText
_quit:  invoke ExitProcess, 0
main ENDP
END main

I could not get the int 3 instruction to trigger. It was clear why it didn’t, examining the disassembly:

00400FFD  add         byte ptr [eax],al  
00400FFF  add         ah,cl  
--- [User path]\main.asm 
        mov eax, 6
00401001  mov         eax,6  
        xor eax, eax
00401006  xor         eax,eax  
_label: add eax, ecx

The int 3 instruction had been replaced with add al,cl, but I had no idea why. I managed to track the problem to whether or not Incremental Linking was enabled. The above disassembly was generated with Incremental Linking disabled (/INCREMENTAL:NO option on the command line). Re-enabling it would result in something like the following:

.CODE
main PROC
        int 3
00401010  int         3  
        mov eax, 6
00401011  mov         eax,6  
        xor eax, eax
00401016  xor         eax,eax 

I should note that the interleaving lines are references back to the original code (I guess a feature of Visual Studio’s disassembly window). With Incremental Linking enabled, the disassembly corresponds exactly to what I had written in the program, which is how I expected it to behave all along.

So, why would disabling Incremental Linking cause the disassembly of my program to be altered? What could be happening behind the scenes that would actually alter how the program executes?

Answer 1

The “add” instruction is a two byte instruction, the second of which is the 1-byte opcode of your int3. The first byte of the two byte add instruction is probably some garbage just before the entrypoint. The address of the add instruction is probably 1 byte before where the int3 instruction would be.

I quickly assembled and then disassembled those two instructions with GNU as en objdump, and the result is:

  8:    00 cc                   add    %cl,%ah
  a:    cc                      int3   

Here you can clearly see that the the add instruction contains the second byte 0xcc, while int3 is 0xcc

IOW make sure that you start disassembling on the entry point to avoid this problem.