Is it possible to write a C function that does the following?
- Allocate a bunch of memory in the heap
- Writes machine code in it
- Executes those machines instructions
Of course, I would have to restore the state of the stack to what it was prior to the execution of those machine instructions manually, but I want to know if this is feasible in first place.
It’s certainly possible. For various reasons, we’ve spent a lot of effort of the last 30-40 years trying to make it as difficult as possible, but it is possible. In most systems now, there are hardware and software mechanisms that attempt to protect data space from being executed.
The basics, though, are fairly straightforward: you construct a piece of code, and assemble it, either by hand or4 via a compiler. You then need a fragment of code space, so you insert the code into your program
since you wanted to use the heap you need to malloc the space
Now, what you need is a way to get the program counter to point to that chunk of data that is also your chunk of code. Here’s where you need a little craftiness. Setting the program counter is no big deal; that’s just a JUMP instruction for your underlying machine. But how to do that?
One of the easiest ways is by purposefully messing with the stack. The stack, again conceptually, looks something like this (the details depend on both your OS and compiler pairs, and on your hardware):
The basic trick here is to sneakily get the address of your code into the return address; when a routine returns, it basically jumps to that return addrfess. If you can fake it out, the PC will be set to where you like.
So, what you need is a routine, let’s call it ‘goThere()’
Will it work? Well, maybe, depending on the hardware and OS. Most modern OS’s will protect the heap (via memory mapping or similar) from the PC moving into it. This is a useful thing for security purposes, because we’d just as well not let you take that kind of complete control.