Question

I am using GNU make, where I have a top level makefile, which invokes
another makefile, for different types of builds, like:

LIST_OF_TYPES: 32 64 ...

tgt-name: deps
          $(foreach i,$(LIST_OF_TYPES), \
            $(MAKE) -f $(MY_MAKEFILE) ARCH=$i mylib;)

when running with higher j factor like -j100 etc, one of the build fails, but
the return value is still 0 so, I cannot make out if the build really did work!
Is there anything wrong with the way I’m using the foreach construct?
or its just the higher j with foreach which is causing problems?

Answer 1

I’ve never seen this kind of use of foreach, but it seems to work for you. Usually I use a bash for loop

tgt-name: deps
    for i in $(LIST_OF_TYPES); do $(MAKE) -f $(MY_MAKEFILE) ARCH=$$i mylib; done

But this is not the problem, since in either case the makes are run sequentially, AFAICS.

Since you build a library, there’s the possible clash of two objects being inserted into an archive simultaneously. When this happens the archive might become corrupted.

As with every parallel execution, be it make jobs or threads, you must protect the shared resources (the archive in your case). You must add the objects at the end of the library build or protect the insertion with some lock (e.g. man lockfile or similar).

There might be other problems, of course. Look out for the simultaneous access to shared resources (object files, archives, …) or incomplete defined dependencies.

Update:

foreach seems not to be a problem. Set LIST_OF_TYPES to a single type (e.g. 32 only) and then do a make -j100 mylib. If the problem is with the building of a single archive, it will fail with only one type as well.

You can also test with make ARCH=32 -j100 mylib. This should show the problem too.