Corel Corporation

TestComplete Addressed Corel's Needs

Corel has plenty of experience with automated testing and has continued, over the years, to employ automation as an effective tool while efficiently assuring the quality of their products. However, the Corel Test Automation Initiative project was conceived to address several outstanding needs.

"First, we needed to address the problem of the waxing and waning interest and commitment to automation that stems from an individualistic approach lacking in standardization" says Dave Meisenheimer, Automation Initiative - Project Leader at Corel. "Over the years, individuals have been highly motivated to use various automation strategies to become more efficient in their daily tasks. However, while an individualistic approach is quite effective in some ways, we found that maintaining and building on individual successes was difficult as people changed roles within the company. We wanted to reduce the amount of time we spent re-inventing the wheel."

"Secondly, we needed to build a system that was accessible to multiple groups and levels within the company. For instance, reporting results needed to be centralized and standardized so that the value of automation was more consistently obvious, and interest and commitment to automation could be sustained."

For example, Corel wanted:

• "Our development groups to be hooked on results as feedback for their success/failure on a build by build basis."
• "Our managers to be hooked on results as a means of making decisions regarding software releases."
• "Our QA testers to be hooked on results for the purpose of easing their testing load."

"With everyone more focused on automation and relying heavily on results, we felt that automation could contribute more effectively to our overall testing efficiency. Past experience had shown that both an individualistic approach and, alternatively, a specialized automation team approach had caused communication gaps that prevented automation from reducing duplication of effort. With the Corel Test Automation Initiative, everyone is more aware of the automation test coverage and efficiency gains are highly tangible and significant."

Thirdly, and pivotally, Corel needed to be more efficient. The clear and precise mandate from their management was that automation was to make them more efficient and enable them to release SKUs with greater speed and sustained confidence. "TestComplete’s connected application feature and the power of C# is what has permitted us to really nail this requirement. We’ve been able to create a multi-tier framework that is robust, maintainable and flexible to the demands of our highly customizable software. Language versions, various product SKUs, and a highly customizable UI are no longer problems for us as our framework and TestComplete allows us the flexibility to accommodate the running of these test cases in all these different environments without any awareness required on the part of the test cases. The test cases are written once, on our base SKU, and the framework handles all filtering. We can now budget fewer manual testing resources toward acceptance testing of our SKU releases. These releases receive better test coverage and benefit from a quicker turn around of test results."

Shopping for a Testing Solution

Corel evaluated the tools available from all of the major vendors and found TestComplete to meet all of their criteria. "TestComplete excelled, particularly with respect to the following criteria:

• Able to recognize and deal with most of the custom controls within our applications
• Highly customizable with respect to how tests are written, stored, and run, and with respect to how results are reported
• Unicode-enabled
• Restricted users
• Deals well with web controls, Win32, Mfc, MSAA interface, and custom controls

Corel Saves 5-7 Days Every Release

"We now save approximately 5-7 days of manual testing on the release of every major SKU. Additionally, we are performing more testing with automation than we would have with a manual test effort and we can now release with greater confidence. Likewise, while we would not run every test daily through a release cycle under a manual test regimen, automation affords us the opportunity to do so without cost - as a result, we run our automated tests daily, and find defects immediately and earlier in the cycle, resulting in a more efficient triage, debugging, and defect resolution by our dev team."

By automating the most commonly executed manual test cases, Corel's manual testers are now free to do more “use-case based” testing that emulates how their products are exercised by real users.

"In addition to the affordability, the greatest feature in our opinion was the connectedapplication feature. This feature is what provides us with the flexibility to be extremely creative and to have the freedom to do pretty much anything we’re asked to do with the system we’ve created. With this feature, we really aren’t limited by the tool in any way."

"As part of our due diligence we spoke directly with the folks at AutomatedQA before purchasing the product. They were willing and eager to answer our questions and even helped us get over a few hurdles during our evaluation period. That level of commitment to our success with the tools has continued as we’ve built up our solution."

The Corel Test Automation Initiative

The Corel Test Automation Initiative is designed to unify processes for test automation across teams and products. An integrated system for test case creation and management has been created to enforce a standardized approach and reduce maintenance costs. TestComplete and TestExecute are used to aid in the testing of five of Corel's applications to date. From the WordPerfect Office suite, Corel has automated their application acceptance tests for WordPerfect, QuattroPro, and Presentations. They are currently in the process of automating acceptance tests for CorelDRAW and Corel PHOTO-PAINT applications in their CorelDRAW Graphics Suite.

"We have used automation tools in various ways in the past. Strategies have varied from individual testers maintaining their own ‘record and playback’ tests to having a full automation team focused on creating automated test cases. Our current project attempts to address a number of the short-comings of past initiatives."

For example, automation assets owned by individuals tended to be lost or poorly maintained as personnel shifted roles within the company. "At one time, a dedicated team of automation specialists tended to contribute less to overall efficiency than we would have liked since it was sometimes difficult to manage separate teams in such a way as to reduce duplication of effort. Our current project attempts to overcome these problems by centralizing our automation efforts in such a way as to unify the approach across teams, but simultaneously maintains control of the effort in the hands of the QA and development teams that require the testing."

Corel's automation strategy, and tactics for executing the strategy, are vetted by an Automation Work Group that is comprised of QA testers and software developers from various application groups, and two dedicated automation engineers. Automation engineer and project architect, Christopher Manley has designed a tiered framework that abstracts and centralizes much of the maintenance costs. "We have evolved a base framework that is common for all of our application testing. The base framework is written in C# and references TestComplete’s libraries such that nearly all of the interaction with the application under test is proxied via TestComplete’s connected application features. We are very excited about the ability to structure our framework this way, as it provides us with ultimate control over everything from how we setup a test run, to how we interact with the application and how we log results."

Overlaid on Corel's base framework is a keyword framework that describes all of the UI elements in their application (every button, combo box, etc.) from TestComplete’s perspective. "We have developed a fairly sophisticated relationship between our keyword framework and base framework such that our various versions, languages, and customized SKUs of our products are easily supported and maintained. This way test cases do not have requirements to accommodate such complexities as it is all handled in our framework. We’ve already reaped great rewards for this abstraction as a significant update to the UI of our WordPerfect Office Suite was easily handled with centralized maintenance of our base framework and keyword framework with relatively little impact on our test cases."

"One of the biggest advantages of the keyword framework is the ease with which non-programmers can be trained to write and maintain test cases. Our automation engineers have been able to focus mainly on extending the framework while QA testers take on responsibility for creating their own test cases. The keyword framework is essentially a language for writing test cases where each line of the test case corresponds to a particular user action in the application UI. An object model was designed, such that our keyword language corresponds 1:1 with application UI, making it very easy for test case writers to compose each line of their test case. Figure 1a and Figure 1b show how the application UI is tightly coupled with the keyword language using a few lines of code from a short test case that would launch WordPerfect’s “Open File” dialog, point to a file and click the open button in the dialog."

// Fig 1a - TestComplete code snippet
fw.WPO.WP.MainMenu.Corel.WP13.File.Open.selectItem();
fw.WPO.WP.Dialogs.File.OpenFile.cbFileName.SetText(@"\\server1\Qaoffice\AUtomation\TestFiles\WPO\WP\WordImport\Table\Word2003-Table-Alignment-Right.doc");
fw.WPO.WP.Dialogs.File.OpenFile.btnOpen.clickButton();

Figure 1a: Code snippet - UI-based object model provides a keyword language that makes test case creation and maintenance easy.

Figure 1b: UI-based object model provides a keyword language that makes test case creation and maintenance easy.

"We’ve also created our own test case editing environment (fig. 2) so that test case writers have the luxury of intellisense (fig. 3 and 4) to guide them in their choices as they build each line of their test code. This frees the non-programmer from remembering thousands of obscure function names and procedures. None of this is possible without TestComplete’s connected application feature."

Figure 2: Test case editor that supports keyword language with intellisense. In addition to easing the process of test case creation/maintenance we standardize the approach across applications and development teams.

Figure 3: The test case writer is guided by a UI-based object model and intellisense.