Choosing the Right Automated Testing Tool
"TestComplete receives a lot of awards and reviews in magazines, plus the price of the product was very competitive compared to other products on the market. The most important factor was the evaluation of the TestComplete demo version. During this evaluation period we discovered that the tool could recognize the most important components in our applications and the support department was very helpful even though we hadn’t purchased a license yet," said Mikkel.
Black Box Testing with TestComplete
Mikkel said that the first step was to create a test in TestComplete that tested their applications' installation. "When we invested in TestComplete we had already set up an auto build system that makes installation images of our applications. We decided that our automated test should be connected to the tail of this pipeline. This meant that we had to rely on black box testing since the installation images were not compiled as an open application. There are, of course, ups and downs in black box testing vs. white box testing that I will not discuss here. Basically, it can be harder to do black box testing since you don’t have access to all parts of the AUT, but one thing I like about black box testing is that you actually test the exact same program that your are going to ship to your customers."
Overcoming Challenges with TestComplete
Mikkel said there were two main challenges they needed to overcome:
- Making tests work with all localized versions
- Distributed platform testing (integration with VMware server)
"The first challenge was to address the fact that the test needed to work with all localized versions of the installation program. To avoid writing scripts for each supported language we decided to connect the test framework with our translation system. The baseline is that the test scripts were based on the original strings from the source code, and we created a DLL library with the functionality to lookup strings during the test's runtime. TestComplete’s DLLAccess plug-in was just the feature we needed for this challenge. I was impressed how this actually turned out. We can now run the same script on all language versions and we also avoid stopping test scripts if a translation is missing. When a translation is missing, the framework knows that it should look for the original string," said Mikkel.
"Once we had our test cases up and running the next challenge was to distribute the tests to all platforms. Long ago we decided to run most of our tests on virtual machines and we wanted to continue that line in our automation. To solve that problem we made a solution with TestComplete and VMware Server from VMware. Here the ‘Network suite’ of TestComplete was a really strong component that made it possible to create a system that controls the virtual machines. It starts the virtual machine, launches TestComplete and runs the test cases, fetches the results and shuts down the virtual machine."
"To control the virtual machines in VMware server we had to use the VMware COM API, and fortunately it’s very easy to work with COM objects within TestComplete. Now, we have a framework where we can choose which tests we want to run on a selection of platform configurations."