Question

I’m working in a company that uses Continuous Integration (TeamCity). Every time someone does a check in the CI software starts a build and runs all the Unit/Automated Test . The problem is that we got more than 7000 unit tests + 756 automated tests (used to test the JavaScript as we got a very complex UI logic for making calculation etc). As you can imaging every time someone does a check in the all process takes more than 2 hours to go through all the steps (build-unitest-automated test) so that I need to wait that much before I can get a result to understand if my check in has broken perhaps an automated test or a unit test. Worst situation is when more than one people check in something so that TeamCity start queue up the build and before I can get a valid result (udated) I can wait up to half day ! what strategy should we adopt to speed up a bit this process? Is it a best practice run all the automated tests even against a little change?

Answer 1

I would look at breaking up your test suite in two ways – with the goal of making it so you and your team can check in, go get a cup of coffee and have some meaningful feedback from team city when you get back to your desk.

1. decide what you really want to test on every commit, move the remaining tests to a suite that runs at a scheduled interval (hourly, nightly – whatever works for you).

2. If the set of tests agreed upon to run every commit is still large – break that set up and distribute across multiple nodes running in parallel.

You may also want to beef up your CI machine, depending on the nature of your stuff have the working directory for the tests live in tmpfs (RAM disk).