Using Microsoft Application Verifier Together With TestComplete

Microsoft’s Application Verifier is a tool that monitors the execution of unmanaged applications, traces the application’s interaction with the operating system and reports errors and problems with kernel objects, the registry, the file system, and Microsoft Win32 API calls (heaps, handles and others).

It’s a good idea to use the Application Verifier with TestComplete: TestComplete will simulate user actions over the application under test and the Verifier will monitor the execution and report problems. The general scheme of using TestComplete with the Verifier includes the following steps:

1. You add your application to Verifier and set up the options for checking your application.
2. Use TestComplete to write or record a test script over your application.
3. Run the script and analyze results.

When using Verifier with TestComplete you can take advantage of TestComplete’s DbgServices object: if a tested application was launched with the object’s methods, TestComplete will trace the application run, detect access violations and other events that occur in the application under test and post detailed information about the error to the test log. If an error occurs, the test log will contain the stack of function calls that led to the error. Verifier provides limited information about the call stack, so additional information will help you find the cause of an error faster and easier.

Now we can see how you can use the Application Verifier and TestComplete together. To demonstrate this, we will use a sample Visual C++ application named AppVerifierSampleApp (please find thedownload link below). The application’s main window contains two buttons: Simulate Error and Exit:

The OnClick event handler of the Simulate Error button contains code that allocates a memory block and then writes more bytes to it than was allocated. Thus, this code causes an access violation that Verifier will trace and log.

1. Preparing your application.

Compile the sample application with debug information. Debug info is needed in order for TestComplete to be able to display the function names in the call stack.

Use the following link to download the sources and the compiled executable.

• Sources and compiled executable (in zip)

2. Add the application to Verifier.

• Launch Microsoft’s Application Verifier.
• Select File | Add Application from the main menu and choose the sample executable in the ensuing Open File dialog.
• Verifier will display the executable in the Applications list. On the right, you can select the tests to be performed over the tested application. Check the Basics test group.
  

  

• Press Save to save changes. Upon saving, Application Verifier may display a message informing you that to perform the selected tests, a debugger is required. We will not use any additional debugger. So, simply close this notification message.

Now, Verifier will check the sample application whenever you run it.

3. Create a TestComplete project.

• Launch TestComplete, open the File | Install Extensions dialog and make sure the DbgServices plug-in is installed. This plug-in contains the DbgServices object that will be used for the run.
• Create a new TestComplete project and include the TestedApps project item in it.
• Add the sample application to the tested applications list.
• Launch the tested application.

Now we can record a script:

• Click the Recording toolbar button to start script recording.
• In the application’s window click Simulate Error.
• A window asking to send a report to Microsoft will appear:
  

  

  Click Don’t Send.

• An error message window with three buttons (Abort, Retry and Ignore) will appear:
  

  

  Click Abort. The tested application will be terminated.

• Stop recording.

The recorded script looks as follows (the script you recorded may slightly differ from what you see below) —

‘[TestComplete VBScript]<br />Sub Test1<br />Dim p1<br />Set p1 = Sys.Process(“AppVerifierSampleApp”)<br />Call p1.Window(“#32770”, “AppVerifierSampleApp”).Window(“Button”, “Simulate E&rror”).ClickButton<br />Call Sys.Process(“dwwin”).Window(“#32770”, “AppVerifier Sample Application”).Window(“Button”, “&Don’t Send”).ClickButton<br />Call p1.Window(“#32770”, “Microsoft Visual C++ Debug Library”).Window(“Button”, “&Abort”).ClickButton<br />End Sub<br />

Now we can add calls to script functions that will launch the application under test. The easiest way to do this is to add the following lines to the Main script routine:

‘[TestComplete VBScript]<br />Sub Main<br />‘ Enter your code here.<br />Call DbgServices.LaunchTestedApplication(0)<br />Call Test1()<br />End Sub<br />

The first line calls the LaunchTestedApplication method of the DbgServices object and passes the index of the tested application to be launched. We assume that AppVerifierSampleApp has index 0.

The second line calls the Test1 routine (the routine that we recorded).

Launch the test script and then review the results.

4. Analyzing results.

After the test run is over, you get two result sets – one in the Application Verifier and another in TestComplete.

To view results in Verifier:

• Run the Application Verifier (if you closed it).
• Select AppVerifierSimpleApp in the Applications list.
• Choose View | Logs from Verifier’s main menu. This will call the following window.
  

  

• Select the topmost record in the Logs list and press View. Verifier stores the logs in XML format, so, when you press View, it will launch the application associated with .xml and load the log file in it:
  

  

As you can see, the log contains information about the access violation, but the included information is rather difficult to understand. It’s hard to decrypt the following records:

vrfcore!VfCoreRedirectedStopMessage+81 
vfbasics!VfBasicsStopMessage+1c9 
vfbasics!AVrfpCheckFirstChanceException+139
vfbasics!AVrfpVectoredExceptionHandler+18
ntdll!RtlInitializeSListHead+9dbf
ntdll!LdrAddRefDll+1c1
ntdll!KiUserExceptionDispatcher+e
AppVerifierSampleApp!_AfxDispatchCmdMsg+9c
AppVerifierSampleApp!CCmdTarget::OnCmdMsg+285
AppVerifierSampleApp!CDialog::OnCmdMsg+21
AppVerifierSampleApp!CWnd::OnCommand+169
AppVerifierSampleApp!CWnd::OnWndMsg+57
AppVerifierSampleApp!CWnd::WindowProc+2e
AppVerifierSampleApp!AfxCallWndProc+e0
AppVerifierSampleApp!AfxWndProc+9e
USER32!GetDC+6d
USER32!GetDC+14f
...

TestComplete’s log contains more information:

TestComplete’s log lists all events (such as module loads and access violations) and debug strings that were sent by the application to the debugger. One of these strings is from the Application Verifier:

The access violation message is before this debug string. The Remarks field contains the stack of function calls that caused the AV.

As you can see, TestComplete not only automates tests that are performed with the Application Verifier, but it also provides additional information about errors that occur during the application run.