Question

A git tool that meets the specs below is needed. Does one already exist? If not, I will create a script and make it available on GitHub for others to use or contribute. Is there a completely different and better way to solve the need to build/test every commit to a branch in a git repository? Not just to the latest but each one back to a certain starting point.

Background: Our development environment uses a separate continuous integration server which is wonderful. However, it is still necessary to do full builds locally on each developer’s PC to make sure the commit won’t “break the build” when pushed to the CI server. Unfortunately, with auto unit tests, those build force the developer to wait 10 or 15 minutes for a build every time.

To solve this we have setup a “mirror” git repository on each developer PC. So we develop in the main repository but anytime a local full build is needed. We run a couple commands in the mirror repository to fetch, checkout the commit we want, and build. It’s works extremely lovely so we can continue working in the main one with the build going in parallel.

There’s only one main concern now. We want to make sure every single commit builds and passes tests. But we often get busy and neglect to build several fresh commits. Then if the build fails you have to do a bisect or manually figure build each interim commit to figure out which one broke.

Requirements for this tool:

1. The tool will look at another repo, origin by default, fetch and compare all commits that are in branches to 2 lists of commits. One list must hold successfully built commits and the other lists commits that failed.

2. It identifies any commits not yet in either list and begins to build them in a loop in the order that they were committed. It stops on the first one that fails.

3. The tool appropriately adds each commit to either the successful or failed list after it as attempted to build each one.

4. The tool will ignore any “legacy” commits which are prior to the oldest commit in the success list. This logic makes the starting point possible in the next point.

5. Starting Point. The tool building a specific commit so that, if successful it gets added to the success list. If it is the earliest commit in the success list, it becomes the “starting point” so that none of the commits prior to that are examined for builds.

6. Only linear tree support? Much like bisect, this tool works best on a commit tree which is, at least from it’s starting point, linear without any merges. That is, it should be a tree which was built and updated entirely via rebase and fast forward commits.

7. If it fails on one commit in a branch it will stop without building the rest that followed after that one. Instead if will just move on to another branch, if any.

8. The tool must do these steps once by default but allow a parameter to loop with an option to set how many seconds between loops. Other tools like Hudson or CruiseControl could do more fancy scheduling options.

The tool must have good defaults but allow optional control.

1. Which repo? origin by default.

2. Which branches? all of them by default.

3. What tool? by default an executable file to be provided by the user named “buildtest”, “buildtest.sh” “buildtest.cmd”, or buildtest.exe” in the root folder of the repository.

4. Loop delay? run once by default with option to loop after a number of seconds between iterations.

Answer 1

I wrote a tool I called git test-sequence a while back to do exactly this.

I was just considering blogging about it because a friend was telling me it saved him a ton of work this week.