Question

In the project that I am working on, we have functional tests written over Selenium. The application undergoes functional changes with each feature release.

Is there a tool / mechanism that we can keep track of the gaps in automated functional tests, so that at least the manual testers can keep an eye for these areas?

Note: we are not doing FTDD, so the functional test coverage can be quite poor, even though we ensure high unit test coverage. We use NCover to check unit test coverage.

Answer 1

There are at least two (commercial but cheap) tools that I know of that allow you to attach to the IIS process to capture coverage data for IIS applications.

NCover:

NCover includes the //iis command line
switch. This switch sets up the
coverage environment within IIS and
restarts the web server. You’ll run
NCover like this to analyze coverage
for your web applications:

NCover.Console.exe nunit-console.exe
TestAssembly.dll //iis When you run
NCover in this way, IIS will be
restarted to allow NCover to monitor
your coverage, and your tests will be
run. Once finished, NCover will stop
IIS and detach itself.

See: http://docs.ncover.com/how-to/code-coverage-of-asp-net-applications-on-iis/

DotCover by jetbrains:

Dotcover has visual studio integration which allows you to attach to an IIS application in the same way you would do if you’d want to trace your IIS application. This probably can also be started with the commandline dotCover tool although i’ve never actually tried this.

See http://www.jetbrains.com/dotcover/

I think Rational and Microsoft Teamsystem also have solutions but they’re a bit more expensive.