We are migrating our test report data (unit, regression, integration, etc..) from an XML format to a database format for better analysis. Right now the majority of our test analysis is done using the CruiseControl.NET dashboard, but this is limited to primarily the most recent test data. Older test data can be accessed but not easily compared to new test data. We want to pin point problem components and better narrow down bugs. With the onset of tons of information brought on by our newly implemented regression and integration testing I would like to see some better metrics generated (possibly performance and the like). Have you worked with any business intelligence systems that will provide a framework for accurately and easily implementing some sort of analysis and reporting?
I have looked into JasperReports and Pentaho but I’m struggling with implemetation of Pentaho at the moment. Should I continue my fight with the system? Is this what I’m looking for?
You could always just use SQL Server Reporting Services and Report Builder (MS’s web based designer) or Report Designer (component of Visual studio). It’s pretty easy to get this set up too.
There are a few end user reporting solutions around as well that make it easier to dynamically create reports, if you’re willing to invest a bit of cash.
My company produce one: http://www.rsinteract.com has a very cheap standard edition with a limited number of reports (30 day free trial). It reports directly off SQL server with Reporting Services installed. It won best of TechEd 2006 – http://windowsitpro.com/article/articleid/53944/best-of-tech-ed-2006-winners.html
We actually use ours to analyse the support requests from clients i.e. which component is failing most, who reports the most bugs etc. Not tried it on test data.
There’s also Proclarity, ApexSQL Report, and Tableau all of which are good.