Question

I’ve been programming for years (mainly Python), but I don’t understand what happens behind the scenes when I compile or execute my code.

In the vein of a question I asked earlier about operating systems, I am looking for a gentle introduction to programming language engineering. I want to be able to define and understand the basics of terms like compiler, interpreter, native code, managed code, virtual machine, and so on. What would be a fun and interactive way to learn about this?

Answer 1

Code to execution in a nutshell

A program (code) is fed into the compiler (or interpretor).

Characters are used to form tokens (+ , identifiers, numbers) and their value is stored in some thing called a symbol table.

These tokens are put together to form statements: (int a = 6 + b * c;). Mostly in the form of a syntax tree:

                     =
                    / \
                   /   \ 
                  a     +
                       / \
                      /   \
                     6     *
                          / \
                         b   c

Within an interpretor the tree is executed directly.

With a compiler, the tree is finally translated into either intermediate code or assembler code.

You now have one or more “object files”. These contain the assembler code without the precise jumps (because these values are not known yet especially if the targets are in other object files).
The object files are linked together with a linker which fills in the blanks for the jumps (ans references). The output of the linker is a library (which can be linked too) or an executable file.

If you start the executable, the program data is copied into memory and there is some other link jugling to match the pointers with the correct memory locations. And then control is given to the first instruction.