Continuous Integration servers like Hudson are a pretty amazing addition to software development. I work in an academic research lab, and I’d love to apply similar principles to scientific data analysis. I want a dashboard-like view of which collections of data are fine, which ones are failing their tests (simple shell scripts, mostly), and so on. A lot like the Chromium dashboard (WARNING: page takes a long time to load).
It takes work from at least 4 people, and maybe 10 or 12 hours of computer time, to bring our data (from behavioral studies) from its raw form to its final, easily-analyzed form. I’ve tried Hudson and buildbot, but neither is really appropriate to our workflow. We just want to run a bunch of tests on maybe fifty independent collections of subject data, and display the results nicely.
SO! Does anyone have a recommendation of a way to generate this kind of report easily? Or, can you think of a good way to shoehorn this kind of workflow into a continuous integration server? Or, can you recommend a unit testing dashboard that could deal with tests that are little shell scripts rather than little functions?
Thank you!
I’m guessing your data changes on a daily basis, and thus you want to run the same tests on new batches of data on a -ly basis.
Buildbot can run tests which are shell scripts. You’ll just need to write a bit of Python to extend Buildbot to make it happen. That being said, if you don’t want to use Buildbot or Hudson and you simply want to run your tests every 24 hours or so, you can install your scripts as cron jobs or write a master script that will run all your tests and install that as a cron job. Given that you’re likely going to have to write something that will generate the results of your test into HTML if you want some sort of dashboard anyway, have your master script generate that and have apache or twistd or something serve it up.
Otherwise, I’m unaware of CI systems that will do things much differently than Buildbot or Hudson.