Question

‘make’ is not only useful for building your programming project, but it seems to be under-used in other areas.

For example, many shell scripts can be rewritten as Makefiles to allow independent parts to run in parallel (with ‘make -jXX’) to keep all your CPU cores busy, with the explicitly declared dependencies as an added benefit in case you’d ever consider reordering some tasks with side effects in your shell script.

Do you have any interesting stories with unusual uses of make / Makefiles to share? Do you use some other utility as a cheap job scheduler?

Answer 1

Make’s parallelism is particularly handy for shell scripting. Say you want to get the ‘uptime’ of a whole set of hosts (or basically perform any slow operation). You could do it in a loop:

cat hosts | while read host; do echo '$host: $(ssh $host uptime)'; done 

This works, but is slow. You can parallelise this by spawning subshells:

cat hosts | while read host; do (echo '$host: $(ssh $host uptime)')&; done 

But now you have no control over how many threads you spawn, and CTRL-C won’t cleanly interrupt all threads.

Here is the Make solution: save this to a file (eg. showuptimes) and mark as executable:

#!/usr/bin/make -f  hosts:=$(shell cat) all: ${hosts}  ${hosts} %:         @echo '$@: `ssh $@ uptime`'  .PHONY: ${hosts} all 

Now running cat hosts | ./showuptimes will print the uptimes one by one. cat hosts | ./showuptimes -j will run them all in parallel. The caller has direct control over the degree of parallelisation (-j), or can specify it indirectly by system load (-l).