Question

Is there any software available in linux which compiles a source code containing large number of files parallely on either multicore or distributed systems. Libraries like gcc or xserver takes very time for compilation on unicore/dual machine and most of the times it is frustrating when you need lot of recompilation. Is there any technique for compiling such source code parallely ?

Answer 1

On distributed-memory systems, you can use distcc to farm out compile jobs to other machines. This takes a little bit of setup, but it can really speed up your build if you happen to have some extra machines around.

On shared-memory multicore systems, you can just use make -j, which will try to spawn build jobs based on the dependencies in your makefiles. You can run like this:

$ make -j

which will impose no limit on the number of jobs spawned, or you can run with an integer parameter:

$ make -j8

which will limit the number of concurrent build jobs. Here, the limit is 8 concurrent jobs. Usually you want this to be something close to the number of cores on your system.