Question

I had a little too much time on my hands and started wondering if I could write a self-modifying program. To that end, I wrote a “Hello World” in C, then used a hex editor to find the location of the “Hello World” string in the compiled executable. Is it possible to modify this program to open itself and overwrite the “Hello World” string?

char* str = "Hello World\n";

int main(int argc, char* argv) {

  printf(str);

  FILE * file = fopen(argv, "r+");

  fseek(file, 0x1000, SEEK_SET);
  fputs("Goodbyewrld\n", file);      
  fclose(file);    

  return 0;
}

This doesn’t work, I’m assuming there’s something preventing it from opening itself since I can split this into two separate programs (A “Hello World” and something to modify it) and it works fine.

EDIT: My understanding is that when the program is run, it’s loaded completely into ram. So the executable on the hard drive is, for all intents and purposes a copy. Why would it be a problem for it to modify itself?

Is there a workaround?

Thanks

Answer 1

On Windows, when a program is run the entire *.exe file is mapped into memory using the memory-mapped-file functions in Windows. This means that the file isn’t necessarily all loaded at once, but instead the pages of the file are loaded on-demand as they are accessed.

When the file is mapped in this way, another application (including itself) can’t write to the same file to change it while it’s running. (Also, on Windows the running executable can’t be renamed either, but it can on Linux and other Unix systems with inode-based filesystems).

It is possible to change the bits mapped into memory, but if you do this the OS does it using “copy-on-write” semantics, which means that the underlying file isn’t changed on disk, but a copy of the page(s) in memory is made with your modifications. Before being allowed to do this though, you usually have to fiddle with protection bits on the memory in question (e.g. VirtualProtect).

At one time, it used to be common for low-level assembly programs that were in very constrained memory environments to use self-modifying code. However, nobody does this anymore because we’re not running in the same constrained environments, and modern processors have long pipelines that get very upset if you start changing code from underneath them.