8 Things to Consider When Selecting the Right Automated Testing Tool

  September 17, 2012

“If the only tool you have is a hammer,” psychologist Abraham Maslow said, “you tend to see every problem as a nail.”

But in software development and testing, as in most other endeavors, not every problem is a nail; you need the right tools to help you complete your projects on time, on budget, and with maximum effectiveness and efficiency. Here are some things to consider when selecting an automated testing tool:


1.) Platforms, Technology and Types            

Whether you’re testing .Net, C#, or WPF applications, Windows-based or relying on the LINUX kernel, your automated testing tool must support these systems and technologies. Comprehensive automated testing tools that support unit, functional (or UI), regression, data-driven, manual and distributed testing of Windows applications exist, so why not use an integrated solution? Look for flexibility and support for unit tests created in NUnit, JUnit, DUnit, and MSTest.


2.) Tester Skills        

The bench strength within any given QA department can vary considerably, and you need to take into account the skill levels your team has. Can your tests write automated test scripts or will they need help? Is keyword testing part of the process? If so, how well can the automated testing applications you’re looking at handle those tasks? Your testers may have a wide range of skills, a narrow set of skills, or something in-between with gaps here and there. Choose an automated testing application compatible with the skill-set your QA team needs.


3.) Feature-Rich     

No matter where you are in the application lifecycle, one thing is clear, the app will only get more sophisticated; it will only have more features, not less, as the application lives on. Choose an automated testing tool that is feature-rich to keep up with the development lifecycle and the challenges that each new waterfall release or iteration is certain to bring. Look for the ability to implement checkpoints to verify values, databases, or key functionality in your applications.


4.) Manual and Automatic                

Look for automated testing tools that support record-and-playback test creation and manual creation options.


5.) Change Management  

Over time, it's certain that some changes will need to be made in the application’s user interface (UI). Choose automated testing tools that won’t break down and fail when the UI changes. You want an automated testing tool that is easily maintained and whose tests can be reused as long as possible.


6.) Results Management    

You want to be able to see the results of your testing clearly. Logs, dashboards, and other instrumentation that automatically records and logs report results make managing the testing process smoother and more effective from start to finish.


7.) Collaboration   

The ability for testers to share their work and have developers and other colleagues share theirs too is invaluable. Look for automated testing applications that can support concurrent work with project files, sharing script code among several projects and integration with source code control and issue-tracking systems.


8.) Data In, Data Out           

Data sources and the venues to which you need to provide data are in constant flux. Look for an automated testing tool that can provide reports in XML+HTML, Excel, and MHT. The ability to intake data from multiple sources, such as text and XML files, Excel worksheets , and databases, like SQL Server, Oracle, and MySQL, also make the tool more valuable.

Prepare a list of the capabilities and features you’d like to have in your automated testing tool. Identify which are “must-haves” and which are “nice-to-haves.” Leave a little space on the list for the stuff you’ll soon discover along the way but you don’t know you want yet.