Question

About three years ago I switched from using commercial testing tools to using an open source testing framework (WatiN), and a UI automation framework I developed myself.

Since doing this I think life is much better using these libraries and Visual Studio, than expensive dedicated testing tools with either their own languages, or VBA.

Are open source automated testing tools and frameworks better than commercial products

Or do expensive commercial automated testing tools such as WinRunner, QuickTest Pro, Testpartner etc … still have a future?

Answer 1

It’s hard to make that sort of determination for a generic category like “testing tools”. Usually, it’s best to evaluate both the commercial solution and the open source solution on a case-by-case basis. From the wording of your question, it sounds like you’ve found the open source solution a better fit for your needs.

However, there are some points you can use in your decision making process:

Commercial Tool Benefits:

• Support – usually, the company is paying people to support the product. In addition, many commercial companies offer support contracts for various levels of support. If you need support in a crunch, commercial support is the way to go.

Open Source Tool Benefits

• Price – pretty obvious…it’s hard to compete against free
• Openness – open source projects tend to adapt open standards more readily than commercial products (a lot of commercial products also adapt open standards, but open source software tends to do so more frequently).
• Self-support – If your company allows it, you can fix the bugs you find in the tools yourself. No need to wait for a third party to get around to fixing them.

It’s also worth mentioning that a lot of the commercial testing tools are built on their open source counterparts. If that is the case, then you might be better off going with the open source versions, unless you need the added support.

I think you’ll find that commercial products and Open Source products tend to have similar feature sets. In other cases, they may solve the problem with completely different approaches. Again, you’ll probably want to make the evaluation for every case.