ITAGroup

Need for an Automated Testing Tool

ITA Group had several large projects using new technology that required quick cycle times on testing. "We also wanted to build a key-word driven automation framework that would ease script maintenance and improve error handling. These scripts would form the basis for a test library that could be used to regression test the whole system when maintenance builds were needed in the future," said Scott Anderson from ITA Group.

From Days to Hours

"With TestComplete we built automated smoke tests for the main paths of two major applications. Smoke tests are run after each build to verify the build install. Full regression tests are run when significant system changes are made. The smoke tests helped improve build stability and consistency. Full regression testing with TestComplete allows one QA member to do the regression testing in just a few hours, where it previously took five QA members several days to complete," said Scott.

Key-Word Driven Automation

"TestComplete has a full featured programming language we used to develop a key-word driven automation framework. The framework is used to verify application builds on a variety of environments. The scripts are written so that both human and machine can test from them allowing us to maintain only one set of test documents. Our standard library of tests, which we call suites, allow for consistently reproducible testing of new builds to ensure compatibility between new and old versions. By using automation to execute both the smoke and full regression tests we have spread our work load around. The full regression can run after hours with only minor monitoring remotely. The QA member can review specific enhancement/defect areas of the system while the smoke test verifies the basic flows."

Scott has high praise for AutomatedQA tech support, "TestComplete customer service is the best of any automated test tool vendor. They quickly provide solutions to very technical issues encountered while developing our framework."

TestComplete Saves Time

Scott explained how TestComplete has saved them time when testing: "We reduced the time spent on build acceptance testing by covering basic flows while the QA members cover the new features or the defect fixes. With TestComplete we can perform full regression virtually overnight with little to no human interaction, where before, it took several QA members several days to execute these tests."

Improved Time to Market

"With TestComplete we can now execute a full regression suite in hours instead of days. We can also run specific groups of regression tests to increase our coverage on maintenance builds and ensure related functionality is unaffected, which turns into our product reaching the market faster," said Scott.

Increased Test Coverage

"Since we can execute more tests with less people we execute the full regression more often then before. Also because we have error and exception tests included, we get more consistent coverage of those areas," said Scott.

"We now run several hundred tests across multiple systems. Depending on the system we are testing, one system may execute tests twice every two weeks. Another system executes a smoke test with every build release and full regression as needed. Before TestComplete, we did not run our tests that often."

TestComplete is Versatile and Robust

"We have used three different types of test scripts with TestComplete. We wrote our own key-word driven format for long term regression maintenance testing, and quick smoke tests. We used pure record and play back scripts for special case testing and to reproduce production issues. Also we have used data driven scripts that use both record and play back and the key-word driven code to perform large database loads and setups. TestComplete's versatility and robustness are probably the biggest asset of the tool. Using a standard programming language like VBScript was also a plus as we were able to use code developed by our developers and outside parties without converting it to an application specific language," said Scott.

TestComplete has allowed ITA Group to increase their test coverage while decreasing the amount of bugs, man power, time and money spent on testing.