Question

I’m trying to switch to the protected mode in intel x86.

I’ve loaded my gdt with lgdt, set the P flag of cr0 to 1 and all the segments selectors but when I return from the function call, I can’t call any other function or I get this error

qemu: fatal: Trying to execute code outside RAM or ROM at 0xfeeb7c5b

Here is my switch_to_pmode function:

gdtr:
.short      23  // limit
gdtr_base:
.long       0   // base

switch_to_pmode:
    movl $null_segment, %eax        // Address of the first byte of the GDT
    movl %eax, gdtr_base

    cli             // disable interrupts

    lgdt (gdtr)

    movl %cr0, %eax
    or $0x1, %eax
    movl %eax, %cr0         // Set the PE flag

    push $0x8
    push $reload_segments
    lret

reload_segments:
    movl $0x10, %eax
    movl %eax, %ds
    movl %eax, %ss
    movl %eax, %es
    movl %eax, %fs
    movl %eax, %gs

    ret

foo:
    ret

And my calls

_start:
    call switch_to_pmode
    call foo // <----- Ouch!

Thank you

Answer 1

You need to make sure the assembler translates the code following the protected mode switch as 32 bit code, with a .code32 (or use32 in nasm) directive.

Additionally your return address after your protected mode routine is no longer valid. You cant really return to anything after that. Instead set esp to something useful and go on.