Question

Can anyone explain how compilation works?

I can’t seem to figure out how compilation works..

To be more specific, here’s an example.. I’m trying to write some code in MSVC++ 6 to load a Lua state..

I’ve already:

• set the additional directories for the library and include files to the right directories
• used extern "C" (because Lua is C only or so I hear)

• include’d the right header files

But i’m still getting some errors in MSVC++6 about unresolved external symbols (for the Lua functions that I used).

As much as I’d like to know how to solve this problem and move on, I think it would be much better for me if I came to understand the underlying processes involved, so could anyone perhaps write a nice explanation for this? What I’m looking to know is the process.. It could look like this:

Step 1:

• Input: Source code(s)
• Process: Parsing (perhaps add more detail here)
• Output: whatever is output here..

Step 2:

• Input: Whatever was output from step 1, plus maybe whatever else is needed (libraries? DLLs? .so? .lib? )
• Process: whatever is done with the input
• Output: whatever is output

and so on..

Thanks..

Maybe this will explain what symbols are, what exactly "linking" is, what "object" code or whatever is..

Thanks.. Sorry for being such a noob..

P.S. This doesn’t have to be language specific.. But feel free to express it in the language you’re most comfortable in.. 🙂

EDIT: So anyway, I was able to get the errors resolved, it turns out that I have to manually add the .lib file to the project; simply specifying the library directory (where the .lib resides) in the IDE settings or project settings does not work..

However, the answers below have somewhat helped me understand the process better. Many thanks!.. If anyone still wants to write up a thorough guide, please do.. 🙂

EDIT: Just for additional reference, I found two articles by one author (Mike Diehl) to explain this quite well.. 🙂 Examining the Compilation Process: Part 1 Examining the Compilation Process: Part 2

Answer 1

From source to executable is generally a two stage process for C and associated languages, although the IDE probably presents this as a single process.

1/ You code up your source and run it through the compiler. The compiler at this stage needs your source and the header files of the other stuff that you’re going to link with (see below).

Compilation consists of turning your source files into object files. Object files have your compiled code and enough information to know what other stuff they need, but not where to find that other stuff (e.g., the LUA libraries).

2/ Linking, the next stage, is combining all your object files with libraries to create an executable. I won’t cover dynamic linking here since that will complicate the explanation with little benefit.

Not only do you need to specify the directories where the linker can find the other code, you need to specify the actual library containing that code. The fact that you’re getting unresolved externals indicates that you haven’t done this.

As an example, consider the following simplified C code (xx.c) and command.

#include <bob.h> int x = bob_fn(7);  cc -c -o xx.obj xx.c 

This compiles the xx.c file to xx.obj. The bob.h contains the prototype for bob_fn() so that compilation will succeed. The -c instructs the compiler to generate an object file rather than an executable and the -o xx.obj sets the output file name.

But the actual code for bob_fn() is not in the header file but in /bob/libs/libbob.so, so to link, you need something like:

cc -o xx.exe xx.obj -L/bob/libs;/usr/lib -lbob 

This creates xx.exe from xx.obj, using libraries (searched for in the given paths) of the form libbob.so (the lib and .so are added by the linker usually). In this example, -L sets the search path for libraries. The -l specifies a library to find for inclusion in the executable if necessary. The linker usually takes the ‘bob’ and finds the first relevant library file in the search path specified by -L.

A library file is really a collection of object files (sort of how a zip file contains multiple other files, but not necessarily compressed) – when the first relevant occurrence of an undefined external is found, the object file is copied from the library and added to the executable just like your xx.obj file. This generally continues until there are no more unresolved externals. The ‘relevant’ library is a modification of the ‘bob’ text, it may look for libbob.a, libbob.dll, libbob.so, bob.a, bob.dll, bob.so and so on. The relevance is decided by the linker itself and should be documented.

How it works depends on the linker but this is basically it.

1/ All of your object files contain a list of unresolved externals that they need to have resolved. The linker puts together all these objects and fixes up the links between them (resolves as many externals as possible).

2/ Then, for every external still unresolved, the linker combs the library files looking for an object file that can satisfy the link. If it finds it, it pulls it in – this may result in further unresolved externals as the object pulled in may have its own list of externals that need to be satisfied.

3/ Repeat step 2 until there are no more unresolved externals or no possibility of resolving them from the library list (this is where your development was at, since you hadn’t included the LUA library file).

The complication I mentioned earlier is dynamic linking. That’s where you link with a stub of a routine (sort of a marker) rather than the actual routine, which is later resolved at load time (when you run the executable). Things such as the Windows common controls are in these DLLs so that they can change without having to relink the objects into a new executable.