Since TestComplete 5 is currently under development, the information contained in this article is not final and may change at any time.
Software developers have always tried to integrate their products to provide maximum efficiency when in use and AutomatedQA is no exception. A new version of our automated testing product, TestComplete 5, has an enhanced integration with our profiling and debugging tool, AQtime 4 and 5.
With this version you will be able to launch your application from TestComplete and profile it with AQtime with ease. The tested application will be profiled by AQtime and tested by TestComplete at the same time. AQtime will trace the application execution while TestComplete will be simulating user actions over it. At the end of the test run, you will get two result sets: one in TestComplete and another in AQtime.
Unlike previous versions, in TestComplete 5 you can configure all settings visually, there is no need to write script code, though the scripting approach is also supported.
Due to these innovations, you can easily perform coverage testing and find answers to such questions as "What code is being tested with your automated test?", "What proportion of the code has already been tested?", "Were any exceptions thrown during the execution of the automated test?" Of course, with AQtime's profilers you can find answers to other questions about the application’s performance and memory leaks.
Let's demonstrate how the TestComplete 5 and AQtime 5 integration works and learn what portion of the code is covered by our automated tests. For this purpose, we will use the sample Coverage application created for Visual C++ 2005. The sample is shipped with AQtime. It is located in the SamplesUnmanagedCoverageVC2005 folder.
- Compile the sample application with debug information.
- Create a new TestComplete project and add the Win32 Tested Applications project item to it.
- Add the coverage.exe file of this sample to the project. To do this, right-click the TestedApps node in the Project Explorer, select Add | New Item from the context menu and choose coverage.exe in the ensuing Open File dialog.
The Win32 Tested Applications project item has been greatly improved in TestComplete 5. Now you can select one of several modes to launch a tested application. You specify the desired mode from the Run Mode column in the TestedApps Editor:
We will select Profile mode, so TestComplete will run the tested application under AQtime. RunAs is another available mode. It should be used when you want to start your application under a specific user account. Simple mode means an ordinary application launch.
After we have selected the launch mode, we need to set launch parameters. To do this, press the ellipsis button of the Parameters cell. TestComplete will display the Advanced Parameters dialog. Here we can select the version of AQtime that will be used for the run, the profiler and some other parameters. In general, other parameters could be left by default, however, since we want to find out what portion of our application code has been tested, we need to define profiling areas to include only specific application files in profiling. To do this:
- Launch AQtime and create a new project in it.
- Add the coverage.exe application to the project.
- Create a new profiling area of the including type and add the coverage.obj and coverageDLG.obj units to it.
- Save your AQtime project.
The following figure shows the AQtime project with the added executable file and the created profiling area:
Now switch back to TestComplete. You can see the Advanced Parameters dialog window with the selected settings in the figure below. We have specified the AQtime version, the profiler that we want to use and the name and path of our AQtime project.
Press OK to save changes.
Now, when launching the coverage tested application, TestComplete will perform the following actions:
- Run AQtime 5 and load the specified project in it.
- Select the Coverage profiler.
- Start the AQtime profiler run.
The following script launches the coverage.exe application, clicks buttons in it, simulates some user input and then closes the application and gets profiling results from AQtime:
Please pay special attention to the fact that the behavior of the TestedApp.Run method has been changed in TestComplete 5. If Profile mode is selected for the tested application, this method returns TestComplete's AQtime object, but not the process object as it would if the tested application were running in Simple mode.
After the script is played back, the test log will contain references to .xml files containing profiling results:
A click on a link will open the .xml file containing profiling results. In our case, the figure shows that automated GUI testing covers 83.33% of functions and 88.24% of code lines.
As you can see, you can easily take advantage of AQtime and TestComplete's integration and build more thorough tests. In this article, we have demonstrated the integration with AQtime's Coverage profiler, and you can now perform coverage testing with AQtime and TestComplete. Another possible alternative is to use the Allocation or Resource profilers to combine testing and search for memory or resource leaks.