Automated Test Runs – Test Batches

Keyword Tests – Can Do More Than You Think

With the introduction of TestComplete 7 came Keyword-Driven testing , otherwise known as Keyword Testing. Now, most think that this is a feature for non-technical testers, or beginners, people who don’t know scripting. While this is true to a point, you can do some really cool things with keyword tests, like building logic into your test runs.

When I say “build logic into your test runs”, I mean: organize your test’s workflow and control how tests are run. The great thing about doing this is that you can organize your automated tests in TestComplete and run them in batches. Then if one test fails, a message will be posted to the Test Log and no other tests will be run.

Organizing Test Batches with Keyword Tests

It’s pretty easy to organize your automated tests into batches. You start out by creating a new keyword test that calls your other keyword tests. This is accomplished by using the Run Keyword Test and If… Then operations. You just use one Run Keyword Test operation for each individual test and separate them with If… Then operations. Introducing the logic into each individual automated test is where it becomes interesting.

Introducing Logic into Test Batches

There are actually two parts to introducing the logic into your tests. First, the test batch starts off with a Run Keyword Test operation, followed by an If… Then operation, which has a child Run Keyword Test operation. This parent/child relationship is duplicated for every test you have, with the exception of the first test in the batch. Below are the steps for creating your test batch:

 

  • Add a new Keyword Test to the KeywordTests node in the Project Explorer by right-clicking the node and selecting Add | New Item. You can name it anything you want, but I prefer Test Batch.
  • Click the Run Keyword Test operation from the Test Actions section and drag it onto your workspace.

    • Choose the first test you want to run and click OK.

  • Click the If… Then operation from the Statements section and drag it under the last inserted operation.

    • Click the Ellipsis button in the Value1 field.
    • Click the Mode drop down and select Last Operation Result.
    • Click OK.
    • Leave the Condition field as: equal to.
    • Click inside of the Value2 field and type true.
    • Click OK.

  • Click the Run Keyword Test operation from the Test Actions section and drag it on top of the If… Then operation, making it a child of this operation. If you drop it into the wrong place, don’t worry, you can use the green arrow buttons on the Keyword Test toolbar to position the operation correctly.

    • Choose the next test you want to run and click OK.

  • Keep repeating the last two bullets until all of your tests are set up.

 

Using the steps above, I grouped some example tests together in one test batch. The Keyword Test that contains the test batch with the Run Keyword Test and If… Then operations looks like:

As you can see, my test batch contains sample tests called: Login, Verify_Account, Buy_Stock, Verify_Order, Verify_Quantity, Lougout and Close_IE and I have them organized in that order and each test is dependent on the preceding test. The If… Then operation signifies that if the test before that operation fails, then the child test will not run.

As I mentioned in the beginning, this is only one piece of the puzzle when creating test batches. We still have to configure each Keyword Test contained in the batch run so that it can essentially report its outcome to your test batch.

What to Include in Individual Keyword Tests for Batch Runs

Each Keyword Test used in a batch run needs to be able to tell whether it encountered errors or not. This is what determines if the following test will be run. To do this we’ll build a little bit of logic into each test.

Taking from my previous example for test names, let’s say you have a Login test. Every login screen usually throws an error message if the wrong login credentials are used. Knowing this, you can use the If Object operator in a Keyword Test to check if the login was successful and then write the appropriate message to the log using the Log Message operator. This is the only test that requires these operations in my example and it doesn’t require anymore explanation since it is pretty simple to do.

The remainder of the logic is built with: Else, Call Object Method, Set Variable Value, If… Then and Return operations. Two variables also need to be defined in every test. The names of the variables for every test are the same, ErrorCount and WarningCount and both are of the Integer type and have a default value of 0. I’ll explain what these are used for in a bit. To create those, you just need to click the Variables tab at the bottom of your test steps and right-click within the workspace, and then click New Item from the context menu. Pretty simple, right! Don’t worry, the steps to do all of this is not far behind…

Configuring Individual Keyword Tests for Batch Runs

Now it is time to configure your individual tests with the logic that tells your test batch whether it failed or passed. It’s not hard, because once you do it to one test you just copy the operations from that test to the others and make minor tweaks. Let’s get on with it!

First, don’t forget to create two variables in every one of your tests:

 

  • Click the Variables tab at the bottom of your Keyword Test.
  • Right-click within the workspace area and select New Item. Perform this step twice to create two entries.
  • Click inside each field and make the values:

    • Names: ErrorCount and WarningCount
    • Type: Integer (Both)
    • Default Value: 0 (Both)

  • Click the Test Steps tab at the bottom of the workspace area to go back to your Keyword Test.

 

You have just created your variables and they should look like:

Click for larger image

These variables will hold the number of errors and warnings that have been logged in your test. Their values will be set by the logic described below.

Now let’s create the logic at the bottom of our Keyword Tests:

 

  • Click the Call Object Method operation under the Test Actions section and drag it to the bottom of your test.

    • In the Call Object Method dialog type Log and click Next.
    • Select the ErrCount [Get] method in the Specify Method or Property portion of the wizard and click Finish.

  • Click the Set Variable Value operation under the Statements section and drag it under the last operation in your test.

    • Select the ErrorCount variable and click Next.
    • Click the Mode drop down and select Last Operation Result and then click Finish.

  • Click the Call Object Method operation under the Test Actions section and drag it to the bottom of your test.

    • In the Call Object Method dialog type Log and click Next.
    • Select the WrnCount [Get] method in the Specify Method or Property portion of the wizard and click Finish.

  • Click the Set Variable Value operation under the Statements section and drag it under the last operation in your test.

    • Select the WarningCount variable and click Next.
    • Click the Mode drop down and select Last Operation Result and then click Finish.

  • Click the If… Then operation under the Statements section and drag it to the bottom of your test.

    • Click the Ellipsis button in the Value1 field.
    • Click the Mode drop down and select Variable.
    • Click the Value drop down and double-click ErrorCount and then click OK.
    • Click the Condition drop down and select greater than.
    • Click in the Value2 field and type the number 0.
    • Click the And button.
    • Click the Ellipsis button in the Value1 field.
    • Click the Mode drop down and select Variable.
    • Click the Value drop down and double-click WarningCount and then click OK.
    • Click the Condition drop down and select greater than.
    • Click in the Value2 field and type the number 0.
    • Click OK.

  • Click the Return operation under the Statements section and drag it on top of the If…Then operation, making it a child of this operation. If you drop it into the wrong place, don’t worry, you can use the green arrow buttons on the Keyword Test toolbar to position the operation correctly.

    • To specify the return value: change the Type field to Boolean and select false in the Value field.
    • Click OK.

  • Click the If… Then operation under the Statements section and drag it on top of the first If…Then operation, making it a child of this operation. Again, you can use the green arrows on the Toolbar to position it correctly.

    • Click the Ellipsis button in the Value1 field.
    • Click the Mode drop down and select Last Operation Result.
    • Click OK.
    • Leave the Condition field as: equal to.
    • Click inside of the Value2 field and type false.
    • Click OK.

  • Click the Log Message operation under the Logging section and drag it on top of the second If… Then operation, making it a child of this operation. Don’t forget about the green arrows on the toolbar if you need to move the operations around.

    • Click the Error message type.
    • Click inside the message box and type any informative message you want to see in the log. For example, I used my test name, and then stated that the test failed and all remaining tests in this batch will not run.
    • Click Finish.

 

Almost there! Just two more operations to go… See I told you this was easy!

  • Click the Else operation under the Statements section and drag it to the bottom of your test. This is a parent operation and should not be a child of the last operation.
  • Click the Return operation under the Statements section and drag it on top of the Else operation, making it a child of this operation. Remember, you have the green arrows on the toolbar if you need to move it into the correct position.

    • To specify the return value: change the Type field to Boolean and select true in the Value field.
    • Click OK.

So now, when the tests run, they check to see if errors or warnings have been logged. As long as there are no errors or warnings, the test is considered successful, and the next test in the batch will run. The operations should now look like this in your test:

Click for larger image

That’s it! Now all you have to do is copy all of these keyword operations and paste them at the bottom of every test you want to include in your batch run. Oh, and don’t forget that you need to have the variables in every test included in the batch run; otherwise your tests will fail because these are not present. Also, if you included the name of your test in every Log Message operation, then you will have to alter this so that the correct test names are used. Just double click this operation to change it.

Conclusion

I hope you enjoyed learning about creating test batches in TestComplete as much as I enjoyed blogging about it. Let me know how it works for you and as always, don’t hesitate to comment on this blog post. And, one more thought, if you haven’t tried TestComplete 7, download it and give it a shot, I think you’ll be presently surprised at all of the new features.



Close

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox

By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading