In addition to automation of unit, regression, distributed and other automated testing types, automation tools often offer support for traditional record-and-playback-style GUI and functional testing. While certainly not a comprehensive method of automated testing on its own, recordable automated tests represent an excellent starting place for creating more robust and complete test projects. One of the complaints leveled against traditional playback-type automated testing is that it is inflexible and that the automated test cases are not easily extended. To answer this argument, the Data-Driven Testing (DDT) methodology was born. Although this technique does not originate with TestComplete, it is a fairly straightforward concept and is easily implemented within TestComplete’s automated tests.

Data-Driven Testing – Overview

The simplest explanation of data-driven testing is this: data that is external to your functional tests is loaded and used to extend your automated test cases. One of the best examples is that of a customer order form. To use data-driven testing in this scenario, you might record a single automated test, entering values into the various fields. After you have recorded the automated test, it only contains values that you entered during the recording. Most likely, data you have specified does not cause errors in the application, but other data may cause them. To be sure that the application works as expected you can alter the test to accept variables, entering those variables into the data fields. Thus, data-driven testing allows you to run this test each time you want to add an order record, passing in a new set of data each time. Our recent webinar, Become a Data-Driven Testing Ninja, covers this topic in great detail.

It is convenient to keep data for automated tests in special storages that support sequential access to a set of data, for example, Excel sheets, database tables, arrays, and so on. Often data is stored either in a text file and are separated by commas or in Excel files and are presented as a table. This lets you modify them easily. If you need to add more data, you simply modify the file either in any text editor or in Microsoft Excel (in case of hard-coded values, you should modify both data and code). 

Each row in the file holds values to be assigned or checked. One row may contain values for one automated test. However, you are free to use several rows for one automated test. In this case, you may need to add a field that will specify which values are input, and which are verifying values.

Note that you can create and populate such storages manually, or you can command TestComplete to do this automatically. For this purpose, TestComplete includes the Data Generator that lets you generate test data of various types and store it either to a TestComplete Table variable or to an Excel file. After you specify all the desired parameters for the generated data, TestComplete will automatically generate it according to the defined parameter values and save these values to the variable or file. This approach decreases the time spent on preparing data for data-driven tests. For more information, see the Using Data Generators section in TestComplete’s help.

During the data-driven testing, you can also execute expressions specified in cells of the processed storage (for example, your storage can contain the value of 5+5). To execute this expression, you can either use the Eval function (in VBScript and JScript) or the Evaluate function (in DelphiScript). To do that, you need to obtain the expression first and then pass it as a parameter of the Eval or Evaluate function. After you run your test, it will automatically retrieve the needed expression from the data storage, execute it and return its result (in the example described above, the test will return 10).

Data-driven testing lets you create automated test projects that can be infinitely extended by simply adding new lines of text to a text file or a spreadsheet. For example, data is read from an external source and fed line-by-line into the functional test until there is no more external data. In this way, new automated test cases that are added to the external data simply extend the loop for each new line of data.

When the storage contains both input and verifying data, a data-driven test includes the following operations performed in a loop:

• Retrieving input data from storage
• Entering data in an application form
• Verifying the results
• Continuing with the next set of input data

Later, we will illustrate how to perform these steps with an example.

TestComplete’s Support for Data-Driven Testing

TestComplete offers you two possible ways that can be used to create data-driven tests:

• DB Table variables. These variables have been introduced in TestComplete 8. They provide you with forward and read-only access to the desired database records. These variables support retrieving data from the following storages:
  • Database tables or queries
  • Excel files
  • CSV (comma-separated values) files

  Each DB Table store links to database tables and queries and provide serial access to their records.

  The DB Table variables are usually used by the Data-Driven Loop keyword-driven test operation. It allows you to iterate through the table rows. When you are adding this operation to the test, you specify the DB Table variable that will access the specified database table and specify the records that will be processed by the operation. This approach is very accessible and lets you create data-driven tests very quickly. See below for more information.

• Table variables. These variables let you store test data in two-dimensional arrays (tables) and then use this data in your data-driven tests. The array elements have the Variant type and can store any value: a number, a string, a date and so on. You can also specify the names of array columns and then address the values using these names.
• One more approach to creating data-driven tests, is using the methods of the DDT program object. Via this object you can create scripts for data-driven tests. It can be very useful, if you want to create a custom test, go through the database records in a defined direction (from the last record to the first, for instance) and it is also used for backward compatibility.

You can use these approaches to iterate through records of the desired table, retrieve them and use in your automated tests. In this article, we will exemplify creating data-driven tests via the DB Table variables.

The Data-Driven Testing Sample Project

Let’s see how to perform data-driven testing with TestComplete. We will create a simple data-driven test illustrating how to add new records from the external source to the sample application using DB Table variables. For this purpose, we will use the Orders sample application that is shipped with TestComplete. The application displays a list of orders and lets users create, modify and delete orders. You can find the application in the following folder: <TestComplete 10 Samples>\Desktop\Orders\C#\bin\Release\Orders.exe.

Note: In this article, we will tell you how to perform data-driven testing of a desktop application.

We will use the Excel sheet as data storage. You can find it in the following folder: <TestComplete 10 Samples>\Common\Data-Driven Testing\TestBook.xlsx. Each row of this table contains the values for one order, whose input will be simulated. Note that the data contains not only input values, but also the verifying values. You can see them in the last three columns of the table. One of these values will be used as a baseline copy for comparison with a calculated value.

The purpose of our data-drive test is to read data from the data storage, simulate data input into the form’s edit boxes and check whether the application calculates values for the Total edit box correctly.

Creating a New Project

At first, we will create a new TestComplete project. To do this:

• If TestComplete is not already running, launch it now. Once it is running, select New | New Project from the TestComplete File menu. This will call the Create New Project wizard. In this wizard, specify the project’s name and location. On the next pages of this wizard, you can specify tested applications for your test, select the desired scripting language and so on. However, we will close the wizard now. We will specify tested applications later.
• Click Finish to close the wizard. TestComplete will create the project and display its contents in the Project Explorer panel.
• Right-click the TestedApps node in the Project Explorer panel.
• Choose Add | New Item from the context menu. The standard Open File dialog will appear.
• Select the Orders application executable in the Add Tested Application dialog and click Open. You can find the executable in the following folder: <TestComplete 10 Samples>\Desktop\Orders\C#\bin\Release\Orders.exe.

Recording an Automated Test

In the following steps, we will record a keyword test (keyword-driven test). The test should perform actions like opening the tested application, opening the Order form, populating the form, creating checkpoints, closing the form and closing the application.

Note: If you test a mobile application, you cannot record a test, so you have to create it manually. See TestComplete help for details.

To record the test:

• Select Record Keyword Test from TestComplete’s Test Engine toolbar, or choose Test | Record | Record Keyword Test from TestComplete’s main menu, or select Record New Test from the toolbar of the KeywordTest Editor.
• Add the "Launching the Orders application" comment to a test. To do this:
  • Expand the Recording toolbar by clicking the Show Additional Buttons item.
  • Click Add Comment to Test on the Recording toolbar. This will invoke the Add Comment to Test dialog:
    
  • Enter "Launching the Orders application" and click Add.
• Click Run Tested Application on the Recording toolbar.
• Add the "Opening the Order form" comment to a test as described above.
• Choose Orders | New order from the main menu of the Orders application.
• Add the "Populating the Order form" comment to a test.
• Enter the following values into the edit boxes of the Order form:

  Field	Value
  Product	FamilyAlbum (select it from the drop-down list)
  Quantity	20
  Date	05/06/2010
  Customer Name	John Smith Jr
  Street	12, Orange Blvd
  City	Grovetown, CA
  State	US
  Zip	111155
  Card	Select MasterCard.
  Card No	555777555888
  Expiration Date	05/06/2010

• Add the "Creating the checkpoint" comment to a test.
• Perform the following actions to create a checkpoint for the Total value:
  • Select Create Property Checkpoint from the Recording toolbar. This calls the Create Property Checkpoint wizard.
  • On the Select Object page of the wizard, specify the Total edit box of the Order form. After specifying this edit box, TestComplete displays its name in the Object field of the page.
  • Click Next to proceed with the checkpoint creation.
  • On the Select Property page, choose the wText property from the list of the object’s properties and click Next.
  • On the Comparison Parameters page, make sure that the Equal condition is selected.
    
  • Click Finish to exit the wizard and add the property checkpoint to a test.
• Add the "Closing the Order form" comment to a test.
• Click OK on the Order window to close the Order form and add new record to the Orders application.
• Add the "Closing the Orders application" comment to a test.
• Close the Orders application.
• Click No on the Confirmation window to close the application without changes.
• Click Stop on the Recording toolbar.

The recorded test looks like this:

Modifying the Data-Driven Test and Assigning Input Values

Now we will modify the recorded data-driven test to make it use data stored in an external Excel file. The test will iterate through the rows of the specified file, each time storing one row to the DB Table variable and then using stored data to populate the Order form’s fields.

You can create the DB Table variable on the Variables edit page of the KeywordTest editor manually or add this variable to the test automatically while creating a data loop.

We will create a data-driven loop including the operations between the RunTestedApp operation and the operation that closes the Orders application. To do this:

• Select the specified operations in the Keyword Test editor and right-click them. The editor’s context menu will appear.
  

  Creating a data-driven loop

• Select Make Data Loop from the ensuing context menu item. The Data-Driven Loop wizard will appear.
• On the first page of the wizard, select Create New Variable, type DBTable1 in the edit box and click Next.
  
• On the next page of the wizard select Excel Worksheet from the ensuing table and click Next.
• Now, specify the path to the TestBook.xlsx file that is located in the <TestComplete 10 Samples>\Common\Data-Driven Testing folder. You can enter the file path and name, or click the ellipsis button and select the file in the subsequent Open File dialog. After you've specified the file, click Next.
• Select TestSheet from the ensuing table and click Next.
• Specify the database records to process. We will need to process all of the records, so, click Next.
• On the next page of the wizard, you can change the parameters that are used by the selected operations. In our example, we need to assign the values that are stored in the Excel file to some of the selected operations. To do this:
  • Select ProductNames ClickItem | Param1 parameter. Click the corresponding Value cell and choose DBTable1 -> Product from the drop-down list.
    
  • Similarly, you can specify new values for other parameters: Quantity, Date and so on.
• Click Finish to create a data-driven loop.

TestComplete then adds the Data-Driven Loop operation to the keyword-driven test, and all the selected operations will be moved as its child operations.

Also, the selected operations now have the new parameter values that are specified through the DBTable1 variable. You can see this in the picture below:

Changing the Operation’s Execution Order

Note that in the test described above the Master Card option button is always selected. However, you may want your test to select the card type against the Credit Card column of the TestBook.xlsx file. In this case you will have to specify the conditions for this manually. For example, you can do it using the If … Then operation. To demonstrate this approach, we’ll change the keyword-driven test in the following way: delete the MasterCard.ClickButton operation and insert the segment below:

After modifying, your test becomes more reliable and flexible. So you can use this approach to improve your test’s reliability.

Executing a Data-Driven Test and Checking Results

Now we will run the data-driven test. As you can see, TestComplete opens the tested application and adds new records until there is no more data in the TestBook sheet. After the data-driven test is over, you can find the test results in the Test Log.

As you can see, the data-driven test run was successful.

If the output value had not matched the appropriate verifying value, the "The property value does not equal the template value" error messages would have been posted into the log. This indicates that the application works incorrectly. In this case, the values should be revised. You can include any information you need in error messages. For instance, you can specify the values of all input parameters that were used when the error occurred.