Question

Using AT&T assembly syntax, I’m trying to understand how testl is used in assembly code. Specifically:

testl  %edx, %edx
jle    .L3

I know testl does a bitwise and of the same value to set the condition flags, but how can I interpret ‘jump if less than or equal to’ if it isn’t comparing two values?

Answer 1

Here’s an excerpt from the official documentation from Intel on test:

Operation
TEMP ← SRC1 AND SRC2;
SF ← MSB(TEMP);
IF TEMP = 0
    THEN ZF ← 1;
    ELSE ZF ← 0;
FI:
PF ← BitwiseXNOR(TEMP[0:7]);
CF ← 0;
OF ← 0;

And the same on jle:

Jump if less or equal (ZF=1 or SF≠OF)

So, the jump will be performed if edx is 0 (because edx AND edx = edx and that’s 0 only when edx is 0, and because ZF is set to 1 when the result of AND is 0) or if the most significant bit of edx is 1 (because SF = most significant bit of edx AND edx (or, equivalently, of edx itself) and OF is always 0, which means SF ≠ OF is only true when SF ≠ 0).

IOW, the jump will be performed only if edx is ≤ 0 when interpreted as a signed integer or, equivalently, when edx is either 0 or greater or equal than 0x80000000 when interpreted as an unsigned integer.