QAComplete Helps save 7X

Business Need

The software development team at a large communications company in Canada consists of nearly 750 internal and external technical resources who are involved with software development lifecycle stages, including the requirements, test and defect management phases. Prior to implementing SmartBear QAComplete, different groups used a variety to tools for their requirements, test, and defect management efforts. Depending on the teams and purpose, these tools ranged from spreadsheets to Bugzilla, Redmine, IBM Rational Quality Manager and Quality Center.

The use of disparate tools came with its own set of challenges. For instance, getting one consolidated report across these different systems was challenging. Secondly, each department utilized its own jargon and definition. Therefore, identifying how far along groups had advanced in completing the testing process or in fixing defects required a lot of extra manual work. Finally, from the test management aspect, the use of various tools made the process of figuring out actual test coverage nearly impossible.

“Using multiple tools across teams made it difficult to know which tests had been run or were still pending,” says one of the enterprise architects for the company. “We also found it cumbersome to manage the defect and requirements phases and thereby wanted to standardize the process across different teams.”

In searching for an answer to these challenges, the company hoped to identify a single tool into which all testing, defect management and requirements management reports could be consolidated and managed. “To start with, a standardized process at the basic minimum would help ensure that status indicators, such as <ready to test>, <open> and <hold> would mean the same thing from test-to-test and for each application,” the enterprise architect adds. “This would in turn enable us to be more efficient and move application releases into production sooner.”

Targeted Solution

In searching for a new solution, the communications company first turned to one of its other business units and while doing so, discovered they were already using an effective centralized test management tool: QAComplete from SmartBear. At the same time, other groups in the organization were using spreadsheets, Bugzilla, Redmine, IBM Rational Quality Manager, and HP’s Quality Center. While selecting among these different tools, five things were critical:

• Easy-to-Use: Rolling out a new tool could get challenging if the new tool came with a steep learning curve. As a result, the company wanted to go with a tool that made it easier for users to get going without significant effort.
• Clean Interface: A clean user-interface would go a long way in improving speed and accuracy with which users can use the new tool. Navigational design elements such as menus, links and other buttons in particular would help reduce time and effort for making action choices.
• Dashboard and Reporting: Dashboard and reporting were critical as they would help generate a single view across requirements, tests and defects. This in turn would help the team be on the same page and ensure proper test coverage exists for all requirements.
• Customization Capabilities: Having the ability to change the tool as per the workflow was critical. An ability to configure would allow developers to map specific use cases in the tool, thereby further simplifying the roll out of the tool across the organization.
• Quick return on Investment: Finally, the cost of the tool and the time it took to break-even was looked into. For this purpose specifically, the company performed cost-benefit analysis across the different tools involved.

“Given all these requirements, QAComplete offered the best combination of functionality and price,” the enterprise architect adds. “The cost was a prohibitive factor against IBM and Quality Center. With similar functionality, the price of these tools was approximately 7X the cost of QAComplete. Other tools such as spreadsheets, Bugzilla and Redmine fell short on the test, requirement and defect management aspects. Additionally, QAComplete, allowed us to set up customized workflows and resolution codes to match our business needs. The clean GUI—along with the ability to easily import and export test case data to-and-from various systems—were other benefits of the solution.”

After choosing QAComplete over the other tools, the communication company started using QAComplete for defect management purposes. Once the use of QAComplete for defect management proved successful, QAComplete was rolled out for test, release and requirement management efforts.

In order to ensure a smooth roll out across different teams, a test center management team was created. This team had a representative from each group that was given access to the sandbox to try QAComplete along with video tutorial. The representatives then interacted with their group members as ambassadors of QAComplete to train them on the tool and answer questions.

Benefits and Results

Demand for QAComplete increased so rapidly that the company needed to quickly increase its licenses by 5X over the last two years. Today, the company has close to 150 projects with approximately 60,000 test cases within QAComplete. In the meantime, QAComplete has managed to scale easily as per the company’s growing needs.

“We started with QAComplete two years ago, using it at first for defect management purposes,” says the enterprise architect. “Over time, we have rolled it out for test management and requirements management as well as release management activities, resulting in more than 60,000 test cases. The tool easily links from one phase to the next so we can easily know the progress as well as the traceability across different stages.”

The firm also uses SmartBear SoapUI to perform functional API testing and eventually plans to integrate SoapUI tests into QAComplete so that it can link these tests to requirements, defects and releases as well.

“We now have one place to look for defects and test cases, and we can run more accurate reports much faster than before,” says the enterprise architect. “QAComplete gives us standardized reports as well as an effective way to identify the status of tests, defects and requirements. In the future, we are looking to take this process further and integrate QAComplete with SoapUI.”

• Trying to replace multiple disparate application test, defect release and requirements tools.
• Generating consolidated reports to identify the status of tests, defects and requirements.
• Rolling out a single test management tool across different groups using different tools.
• Improving coverage across different tests.
• Streamlining software development to move applications into production sooner.

• Provides intuitive and friendly user-interface.
• Delivers comparable features to Quality Center and Rational TestManager—but at one-seventh the cost.
• Creates highly-customizable testing processes according to internal development methodologies.
• Enables repurposing and reusing of development assets across testing cycles.
• Helps determine test coverage and ensures tests exist for all requirements.
• Establishes clear traceability among requirements, test cases and defects.

• Facilitates bringing multiple teams onboard—after initial implementation caused software license usage to increase by 5X over two years.
• Manages over 60,000 test cases.
• Accelerates the overall software development process.
• Provides managers with consolidated reports on the progress of tests, defects and requirements.
• Allows software developers to identify the status of tests and defect mitigation.
• Streamlines hand-offs across the test, defect and requirements management phases.
• Enables teams to use a single management tool rather than fragmented multiple tools.