Question

I’m using a tool chain where I do not know the names of all of the intermediate files.

E.g. I know that I start out with a foo.s, and go through several steps to get a foo.XXXX.sym and a foo.XXXX.hex, buried way down deep. And then running other tools on foo.XXXX.hex and foo.XXXX.sym, I eventually end up with something like final.results.

But, the trouble is that I don’t know what the XXXX is. It is derived from some other parameters, but may be significantly transformed away from them.

Now, after running the tool/steps that generate foo.XXXX.{sym,hex}, I now typically scan the overall result directory looking for foo.*.{sym,hex}. I.e. I have code that can recognize the intermediate outputs, I just don’t know exactly what the names will be.

I typically use make or scons – actually, I prefer scons, but my team highly prefers make. I’m open to other build tools.

What I want to do is be able to say (1) “make final.results”, or “scons final.results”, (2) and have it scan over the partial tree; (3) figure out that, while it does not know the full path, it definitely knows that it has to run the first step, (4) after that first step, look for and find the foo.XXX.* files; (5) and plug those into the dependency tree.

I.e. I want to finish building the dependency tree after the build has already started.

A friend got frustrated enough with scons’ limitations in this area that he wrote his own build tool. Unfortunately it is proprietary.

I guess that I can create a first build graph, say in make with many .PHONY targets, and then after I get through the first step, generate a new makefile with the new names, and have the first make invoke the newly generated second makefile. Seems clumsy. Is there any more elegant way?

Answer 1

Make constructs the graph before running any rule, so there won’t be a perfect answer. Here are some reasonably clean solutions.

1) use PHONY intermediates and wildcards in the commands. (You can’t use Make wildcards because make expands them before running rules.)

final.results: middle                                                         
    # build $@ using $(shell ls foo.*.sym) and $(shell ls foo.*.hex)

.PHONY: middle
middle: foo.s
    # build foo.XXXX.sym and foo.XXXX.hex from $<

2) Use recursive Make (which is not as bad as people say, and sometimes very useful.)

SYM = $(wildcard foo.*.sym)
HEX = $(wildcard foo.*.hex)

# Note that this is is the one you should "Make".
# I've put it first so it'll be the default.
.PHONY: first-step
first-step: foo.s
    # build foo.XXXX.sym and foo.XXXX.hex from $<
    @$(MAKE) -s final.results                                               

final.results:                                                                
    # build $@ using $(SYM) and $(HEX)

3) Similar to 2, but have a rule for the makefile which will cause Make to run a second time.

SYM = $(wildcard foo.*.sym)
HEX = $(wildcard foo.*.hex)

final.results:                                                                
    # build $@ using $(SYM) and $(HEX)

Makefile: foo.s
    # build foo.XXXX.sym and foo.XXXX.hex from $<
    @touch $@