Question

This is not something most people would probably use, but it just came to mind and was bugging me.

Is it possible to have some machine code in say, a c-string, and then cast its address to a function pointer and then use it to run that machine code?

Answer 1

In theory you can, per Carl Norum. This is called “self-modifying code.”

In practice what will usually stop you is the operating system. Most of the major modern operating systems are designed to make a distinction between “readable”, “readwriteable”, and “executable” memory. When this kind of OS kernel loads a program, it puts the code into a special “executable” page which is marked read-only, so that a user application cannot modify it; at the same time, trying to GOTO an address that is not in an “executable” page will also cause a fault exception. This is for security purposes, because many kinds of malware and viruses and other hacks depend upon making the program jump into modified memory. For example, a hacker might feed an app data that causes some function to write malicious code into the stack, and then run it.

But at heart, what the operating system itself does to load a program is exactly what you describe — it loads code into memory, flags the memory as executable, and jumps into it.

In the embedded hardware world, there may not be an OS to get in your way, and so some platforms use this pretty regularly. On the PlayStation 2 I used to do this all the time — if there was some code that was specific to, say, the desert level, and used nowhere else, I wouldn’t keep it in memory all the time — instead I’d load it along with the desert level, and fix up my function pointers to the right executable. When the user left the level, I’d dump that code from memory, set all those function pointers to an exception handler, and load the code for the next level into the same space.