Question

C Code:

 #include <stdio.h>

int main()
{
  unsigned guess;          /* current guess for prime      */
  unsigned factor;         /* possible factor of guess     */
  unsigned limit;          /* find primes up to this value */

  printf("Find primes up to: ");
  scanf("%u", &limit);

  printf("2\n");    /* treat first two primes as special case */
  printf("3\n");

  guess = 5;        /* initial guess */
  while ( guess <= limit ) {
    /* look for a factor of guess */
    factor = 3;
    while ( factor*factor < guess && guess % factor != 0 )
      factor += 2;
    if ( guess % factor != 0 )
      printf("%d\n", guess);
    guess += 2;    /* only look at odd numbers */
  }
  return 0;
}

Assembly Code (NASM):

    %include "asm_io.inc"

segment .data
Message         db      "Find primes up to: ", 0


segment .bss
Limit           resd    1               ; find primes up to this limit
Guess           resd    1               ; the current guess for prime



segment .text
        global  _asm_main
_asm_main:
        enter   0,0               ; setup routine
        pusha

        mov     eax,  Message
        call    print_string

        call    read_int             ; scanf("%u", & limit );
        mov     [Limit], eax

        mov     eax, 2               ; printf("2\n");
        call    print_int
        call    print_nl
        mov     eax, 3               ; printf("3\n");
        call    print_int
        call    print_nl

        mov     dword [Guess], 5     ; Guess = 5;

while_limit:                         ; while ( Guess <= Limit )
        mov     eax,[Guess]
        cmp     eax, [Limit]
        jnbe    end_while_limit      ; use jnbe since numbers are unsigned

        mov     ebx, 3               ; ebx is factor = 3;
while_factor:
        mov     eax,ebx
        mul     eax                  ; edx:eax = eax*eax
        **jo      end_while_factor     ; if answer won't fit in eax alone**
        cmp     eax, [Guess]
        jnb     end_while_factor     ; if !(factor*factor < guess)
        mov     eax,[Guess]
        mov     edx,0
        div     ebx                  ; edx = edx:eax % ebx
        cmp     edx, 0
        je      end_while_factor     ; if !(guess % factor != 0)

        add     ebx,2                ; factor += 2;
        jmp     while_factor
end_while_factor:
        **je      end_if               ; if !(guess % factor != 0)**
        mov     eax,[Guess]          ; printf("%u\n")
        call    print_int
        call    print_nl
end_if:
        mov     eax,[Guess]
        add     eax, 2
        mov     [Guess], eax         ; guess += 2
        jmp     while_limit
end_while_limit:

        popa
        mov     eax, 0            ; return back to C
        leave                     
        ret

As you can see, I have marked with ** two instructions.

First of all, the MUL instruction multiplies EAX*EAX and stores the value in EDX:EAX if it’s too big to fit EAX, right?
Then the program checks for overflow.
So when the value is too big to fit EAX the system detects an overflow and set OF=1?
Why? The value isn’t to be stored in both EAX & EDX if needed?

Secondly, the JE instruction. The comment explains: if !(guess% factor !=0).

Thats ok when the programs jumps there from:

cmp edx,0
je end_while_factor

But what if the jump is made because of the overflow check ,or if !(factor*factor < 0)?
Will it be ok? Which values are being compared? It just checks the ZF?
But isn’t it modified previously for another reason (another instruction).

Thanks in advance for your help.

Answer 1

Yes, the MUL instruction sets the OF flag when the EDX register becomes non-zero. In other words, when the multiplication produces a result that no longer fits in a 32-bit unsigned int. The expression that’s being tested is factor*factor < guess where guess is a 32-bit unsigned int. So the code generator generates nice code, if the overflow flag is set then the expression will always be false, regardless of the value of guess. The alternative is worse, if there is an overflow then the result of the expression could be true.

Unfortunately it then fumbles the jump target, the Intel manual specifies that MUL leaves the ZF flag undefined. So that looks like a code generator bug to me. It isn’t clear what compiler generated this code or if this is just a hand-typed alternative of the C code. Code generators don’t typically generate code like this so I’ll readily assume it is just a bug in the hand-typed code.