Question

Our team has been very interested in continuous deployment recently, but we’ve run into a little bit of a roadblock in regards to how to actually get code deployed on Heroku – it seems inevitable that there needs to be some amount of downtime to do a code push to Heroku.

In a traditional environment, code deployment would probably look something like this:

1. Push the code up to a staging directory somewhere (old code is still live)
2. Run migrations against the database (more often than not, it’s safer to run migrations beforehand, and the few that will break the code can be guarded against)
3. Take half (or some percentage of servers) out of the load balancer.
4. Deploy code to those servers.
5. If possible, run some sort of automated smoke test / exercise the servers so they’re “hot”
6. Switch which servers are in and out of the load balancer
7. Rinse and repeat.

With Heroku, I have very little control over two critical steps:

• I can’t run database migrations first. One way I’ve considered to get around this is to keep database migrations branched seperately, and push those to heroku first – which while painful, would solve the problem – but only exacerbate…
• Dyno spin-up time can take a fairly long time – obviously, this is more the fault of Rails than of Heroku, but the key problem is that I can’t do something like the load balancer shuffle above to ensure that my app is ready and loaded before a newly deployed server is put back into the load balancer. Instead, I pretty much have no choice but to give users a 10-15 second load screen and hope for the best (and do that TWICE if I use the database deployment strategy from above)

We do use the maintenance screen currently, but it’s not going to be a scalable solution if we move to full continous deployment (we’d probably have about 10-20 deployments a day, and 10-20 * 30 seconds of maintenance screen starts to add up)

Has anyone run into similar issues? How did you address them? Any great case studies / success stories for true continuous deployment on heroku?

Answer 1

Regarding the dynos spin-up time, Heroku has a beta feature to address just that:

https://devcenter.heroku.com/articles/labs-preboot/

It basically boots your new dynos first, waits a while, switched traffic and only then kills the old ones. My app saw a marked improvement in performance during deploys. You can read it here:

http://ylan.segal-family.com/blog/2012/08/27/deploy-to-heroku-with-near-zero-downtime/