Question

I’m really trying to use best-practices for dependencies management in my web apps but stuff happens. Some problems arise in production only anyways. I want to minimize this as much as possible.

I was thinking that If I can test new deployments in a virtual machine with the latest production snapshot this can solve this problem to a certain degree can’t it?

Answer 1

• Make sure your development environments is a snapshot of your production environment.
• Make sure your build/deploy system works the same way in Dev as it does in Prod.
• Make sure each deployment is isolated from all other deployed apps/services.
• Make sure dev/test environments have easy access to prod data.
  Either read only or a copy to be fiddled with.

One company I worked for had every box Dev/Test/Prod had the same base image.

Now us developers tend to get freaky and install our own custom tools etc (and I have never met a dev who did not customize their environment), but that was OK at this company: The build tools would build and deploy the code on your box in the same way that the code would have deployed in production (so that any custom tools you installed would not be accessible (by default)).

The build tools auto detected all dependencies and deployed them with your application (into the same directory (or symbolic links in your application directory)) and set up the path appropriately. The build system could also depoly to another dev’s desktop or test or prod in the exact same way with only a few extra parameters.

Another company I worked for did use virtual machines.
I did not find that environment as comfortable. The problem was that though theoretically the virtual machine was an exact replica of the prod environment that just made things harder to diagnose when things went wrong.

• The problem here was that because the virtual machine was the testing environment the company thought the dev boxes did not need to be up-to the same specs as the prod machine or require the same OS (mistake one: as you now can’t debug reliably in dev).
• The tools we devs install on our machine are often quite usefull when things actually go wrong. We install them once then we can configure and use them quickly when things do go wrong. If you are using a virtual machine (that is a copy of prod) you need to keep all your extra tools as installers and go in and start installing all these extra tools on-to the virtual machine (a pain and time consuming).
  • Note: Your build system has to be able to deploy your code locally so it is by default not affected by your tools. You need to be able to plug these into running system.
• I always assumed that virtual machines were just an image that could be started.
  • I don’t know why this build system took so long but it took 45 minutes to provision a virtual machine and set up the appropriate environment (Now I don;t know why I never looked into it (we had a build and test team they were looking at it, so I left them too it)).
• Because of constant security patches the virtual machines images tended to drift from the production machine (and not be as accurate).
  • While in company A the dev/test/prod all got the same patches at the same time. Note: Security patches were not automatic. All patches had to be verified by the production and build teams to make sure they did not adversely affect the production systems. But once that had been established patches were applied to all systems automatically.
  • While in company B virtual machine images are not running thus security patches do not get applied automatically. It turned out that it takes a concerted effort to build a brand new virtual image (I would have liked this to be automated (I assume it can ?)) so even when security patches are applied to prod/dev they were not automatically applied to the virtual image. Thus the virtual images were only re-synced with production every six months.