Automating Cloud Testing

The variety of platform/browser/versioning combinations that the present day Windows applications must be tested on, along with the favorable cost equation for renting computing resources in the cloud, drives the expansion of the larger test labs into the cloud. This post reviews how SmartBear customers can leverage cloud computing resources to set up and run tests using Automated Build Studio and TestComplete.

Contents

Why Automating Cloud Tests?

To test your applications thoroughly, you need to check the application’s functioning on computers with different software and hardware environments. An easy and cost-effective way to obtain different environments is to use cloud computing resources provided by Amazon Elastic Computing Cloud (or Amazon EC2).

Amazon EC2 rents out cloud computers (or instances, according to the Amazon EC2 terminology). The cloud computers are virtual machines that are run by the cloud engine. You can lease instances of various hardware configurations. You can then easily install the needed software on these instances and run your tests on them.

A testing process that involves using cloud resources is called cloud testing.

To run a test in a cloud, you may need to perform lots of steps:

• Create (lease) and configure cloud computers.
• Start them.
• Upload tested applications, test projects and test data to the cloud computers.
• Run your tests there.
• Get the test results.
• Stop cloud computers after the test is over and remove unneeded files.
• Etc.

The entire procedure seems to be tedious and error-prone, especially if you go through it over and over again. Though it may be quite easy to run several tests on a few cloud computers, the situation changes when the number of tests or computers increases (have you thought about running 20 test computers manually? 50 computers?). Automating these cloud tests simplifies both set up and test execution and provides considerable time savings.

Automating Cloud Tests With TestComplete and Automated Build Studio

Automation of cloud tests comprises two essential areas:

• Automation of tests running on individual cloud computers (instances)

  In other words, you need to create automated tests that will run on remote computers without human intervention. The tests should be “smart” enough to handle various unexpected situations that may occur. For example, a test should be able to handle unexpected message boxes in the appropriate manner.

• Automation of operations on the instances in the cloud

  This means you should also automate the actions you perform on the instances: launch, start, stop instance, logging in to them, uploading data to and downloading it from the instances and so on. Ideally, you specify only the number of desired instances and the test starts the instances in the cloud and runs tests on them automatically.

To automate test execution on individual cloud computers, you can install and use TestComplete there. TestComplete is an automated testing tool for 32- and 64-bit Windows, Web, .NET and Java applications. It supports visual tests as well as scripts in several scripting languages. It also supports event tracking and includes enhanced library of script functions. All these features help you create tests that control unexpected situations that may occur during a test run on remote machines, so even inexperienced users can create error-proof tests running on cloud computers with ease.

To automate actions against instances in the Amazon cloud, you can use Automated Build Studio. It includes special commands (or operations in Automated Build Studio terms) for performing various actions with the cloud services and computers. For example, using these operations you can easily:

• Create Amazon machine images
• Launch, start, stop and terminate cloud computers (instances)
• Enumerate available instances
• Attach hard drives to the instances
• Save the instances snapshots and create machine images from them later
• Upload files to and download them from the instances, and perform other actions

These operations let you automate the actions that you perform in the AWS Management Console manually. The following image displays the Automated Build Studio operations and the Console commands:

All the operations are configured visually via special wizards and dialog boxes. So, you can automate the cloud-related actions with ease, without writing scripts or program code.

By combining cloud-related operations with the For or While loops you can easily create scalable macros for automating cloud tests. For example, if you put the operation that starts a cloud computer in a loop, you can easily launch the desired number of instances.

Also, Automated Build Studio closely integrates with TestComplete and includes special means for running operations on remote computers. So, using these two products together you can quickly and easily bring your automated tests to the Amazon cloud.

Example

We created a sample Automated Build Studio macro that demonstrates cloud test automation. For detailed description and download link, see another article on our web site – Cloud Test Automation Tutorial.

Conclusion

We have described how you can use Automated Build Studio and TestComplete to automate tests running in the Amazon cloud. We hope you find this information useful when making a decision on bringing your automated tests to the cloud.

If you don’t have Automated Build Studio or TestComplete, here is where you can access a trial or a production version and begin automating cloud operations and tests yourself:

• Download Free Trial of Automated Build Studio »
• Order Automated Build Studio Now »
• Download Free Trial of TestComplete »
• Order TestComplete Now »