Question

We got all psyched about from from svn to hg and as the development workflow is more or less flushed out, here remains the most difficult part – staging and integration system.

Hopefully this question goes a bit further then your common ‘how do I move from xxx to Mercurial’. Please forgive long and probably poorly written question 🙂

We are web shop that does a lot of projects(mainly PHP and Zend), so we have one huge svn repo, with like 100+ folders, each representing a project with it’s own tags,branches and trunk of course. On our integration and testing server(where QA and clients look at work results and test stuff) everything is pretty much automated – Apache is set to pick up new projects automatically creating vhost for each project/trunk; mysql migration scripts right there in trunk too and developers can apply them through simple web-interface. Long story short our workflow is this now:

1. Checkout code, do work, commit
2. Run update on the server via web interface(this basically does svn up on server on a particular project and also run db-migration script if needed)
3. QA changes on the server

This approach is certainly suboptimal for large projects when we have 2+ developers working on the same code. Branching in svn was only causing more headaches, well, hence moving to Mercurial. And here is where the question lies – how does one organize efficient staging/integration/testing server for this type of work(where you have many projects, say single developer could be working on 3 different projects in 1 day).

We decided to have ‘default’ branch tracking production essentially and then make all changes in individual branches. In this case though how can we automate staging updates for each branch? If earlier for one project we almost always were working on trunk, so we needed one DB, one vhost, etc. now we potentially talking about N-databases per project, N-vhost configs and etc. Then what about CI stuff(such as running phpDocumentor and/or unit tests)? Should it only be done on the ‘default’? On branches?

I wonder how other teams solve this issue, perhaps some best practices that we’re not using or overlooking?

Additional notes:

Probably worth mentioning that we’ve picked Kiln as a repo hosting service(mostly since we’re using FogBugz anyway)

Answer 1

This is by no means the complete answer you’ll eventually pick, but here are some tools that will likely factor into it:

• repositories without working directories — if you clone -U or hg update null you get a repository with no working directory (only the .hg). They’re better on the server because they take up less room and no one is tempted to edit there
• changegroup hooks

For that last one the changegroup hook runs whenever one or more changesets arrive via push or pull and you can have it do some interesting things such as:

• push the changesets on to another repo depending on what has arrived
• update the receiving repo’s working directory

For example one could automate something like this using only the tools described above:

1. developer pushes five changesets to central-repo/project1/main
2. last changeset is on branch ‘my-experiment’ so csets are automatually re-pushed to optionally created repo central-repo/project1/my-experiment
3. central-repo/project1/my-experiment automatically does hg update tip which is certain to be on the my-expiriment branch
4. central-repo/project1/my-experiment automatically runs tests in its working dir and if they pass does a ‘make dist’ that deploys, which might set up database and vhost too

The biggie, and chapter 10 in the mercurial book covers this, is to not have the user waiting on that process. You want the user to push to a repo that contains possibly-okay-code and the automated processed do the CI and deploy work, which if it passes ends up being a likely-okay repo.

In the largest mercurial setup in which I’ve worked (20 or so developers) we got to the point where our CI system (Hudson) was pulling from the maybe-ok repos for each periodically then building and testing, and handling each branch separately.

Bottom line: all the tools you need to setup whatever you’d like probably already exist, but gluing them together will be one-off sort of work.