As a guy who frequently switches between QA, build and operations, I keep running into the issue of what to do about operating system updates on the build server. The dichotomy is the same on Windows, Linux, MacOS or any other o/s that can update itself via the internet:
- The QA team wants to keep the build server exactly as it is from the beginning of the product release cycle to the end, since installing updates could destabilize the server and means that successive builds aren’t made against the same baseline.
- The ops team wants the software to be deployed on a system with all the latest security patches; this can mean that the software isn’t deployed on exactly the same version of the o/s that it was built on.
I usually mitigate this by taking release candidate builds and installing them on a test server that has a completely up-to-date o/s, repeating the automated tests that are run on the build server and doing some additional system level testing to make sure everything looks good before deployment. However, this seems inefficient to me; does anyone have a better way ?
Personally i don’t think you have much of an issue here – just apply the latest updates to the build server. The main reasons i say this are:
it is highly unlikely that your code or any of the dependencies on the build server are so tightly coupled to the OS version that installing regular updates is going to affect anything, let alone break it. There can be minor differences between window messages etc between Windows versions, but those are few and far between, and are usually quite well documented out there on teh interweb. If you are using managed technology stacks like WPF/Silverlight or ASP.Net and even mostly Winforms then you will be isolated from these changes – they should only affect you if you are doing hardcore stuff using the WinAPI directly to create your windows or draw your buttons.
it is a good practice to always engineer your product against the latest version of the OS, because you need to encourage your customer to implement those updates too – IOW you should not be in a position where you have to say to your client to not install update xyz because your application will not run against it – especially if that update is a critical security update
testing for differences between OS versions should be done by the QA team and should independant of what is on the build server
you do not want your build server to get in to such a state that it has been so isolated from the company update process that when you finally do apply them all it barfs and spits molten silicon everywhere. IOW, the longer you wait to update, the higher the risk of something going wrong and doing so catastrophically. Small and frequent/incremental updates are lower risk than mass updates once per decade 🙂
The build server updates that you do have to be cautious about are third party controls or library updates – they can frequently contain breaking changes or considerably altered behavior. They really should be scheduled, and followed up by a round of testing looking for any changes.