How Test Automation Fits With Agile & DevOps

Many software teams aren’t satisfied with their test coverage or the associated maintenance costs. At the same time, it’s hard for business teams to justify the time spent writing tests without actionable business metrics. The good news is that moving from test automation to continuous testing can unlock the benefits of automated tests.

Let’s take a look at the difference between test automation and continuous testing, as well as how these concepts fit with Agile and DevOps processes.

Many development teams aren’t happy with their test coverage or the associated maintenance costs — a shift to continuous testing could help.

What is Continuous Testing?

Most software teams are familiar with test automation. After pulling in a user story, test engineers write tests to determine if the code is correctly implements the user story’s requirements. These tests usually include low-level unit tests that test individual methods and higher-level integration tests that ensure an entire workflow functions as expected.

Continuous Testing is the process of executing automated tests as part of a software delivery pipeline designed to obtain feedback on business risks as quickly as possible. Whereas Test Automation checks user story requirements, continuous testing spans the entire DevOps process and delivers insight into tangible business risks.

Continuous testing is necessary for many reasons:

• Application architectures are becoming increasingly complex, which makes it hard for developers to track down bugs. For example, a single web application may include several microservices and various APIs that interact with various client apps.
• Release cycles have been much shorter and there’s no time for adequate manual tests or unorganized automated tests. Many companies are moving towards continuous delivery, which means deploying new software versions in near real-time.
• The interconnected nature of software and business means that an application failure is a business failure. When a business depends on software, any downtime can be extremely costly in lost business and reputation.

In short, continuous testing ensures that business and technical teams are on the same page about requirements and that the software functions as expected by operating throughout the software development lifecycle. Automation engineers can be more confident in the code they deploy and stakeholders can be sure that the software contains minimal business risk.

How it Fits with Agile & DevOps

Automation engineers may not fully understand how the tests fit with the Agile process and DevOps teams — especially if they’re only involved with one piece of the puzzle. While they may not be involved with the full spectrum of DevOps processes, it’s helpful to understand what’s happening in order to better deal with any bugs or issues that may arise.

Let’s look at a common workflow to see how everything fits together and provides real value to the business:

• A developer, test engineer and stakeholder meet to discuss a user story and come up with concrete examples of behaviors from a business standpoint. These behaviors are converted into executable behavior-driven development (BDD) tests.
• Developers pull a user story from the backlog and write unit and integration tests (in conjunction with test engineers) as part of the test-driven development (TDD) process on their local machines. The tests and code are deployed to a shared code repository when everything passes.
• DevOps engineers develop continuous integration (CI) servers that run code in a shared repository, execute all of the tests (TDD and BDD) and ensure everything passes. They may also setup different branches for pre-production or staging.
• QA or automation engineers write additional tests, as required, and run other quality assurance processes. For example, they run performance tests to ensure that an application doesn’t contain any bottlenecks under an expected load.
• Ops engineers reuse the tests in the shared repository for acceptance testing and ongoing post-production monitoring.

Continuous testing involves many different parties, including developments, test engineers, DevOps engineers and even stakeholders — if using BDD in the development process.

How TestComplete Simplifies DevOps

TestComplete is an automated UI testing tool that makes it quick and easy to create, maintain and execute functional tests across desktop, web and mobile applications. With its intuitive interface and extensive integrations, you can increase test coverage and ship high-quality software by moving beyond test automation and toward continuous testing.

The platform ships with native support for Gherkin’s Given-When-Then scenarios, enabling non-technical and technical teams to quickly convert business requirements into automated tests. With an IDE, Gherkin interpreter, test running and reporting insights in a single tool, you don’t have to worry about complex technology stacks to build and convert feature files.

Automation engineers can quickly build end-to-end tests using record and replay capabilities powered by an AI-supported customizable object repository. Rather than learning new tools and languages for each platform, you can use the same tool to test web, desktop and mobile applications. The Intelligent Recommendation System also makes maintenance a lot easier.

Finally, a lightweight execution engine and distributed testing capabilities make its say to execute tests on remote or virtual computers, as well as integrate with top continuous integration systems, including Jenkins, and SCMs, like Git. It’s easy to incorporate all of your tests into a continuous integration and deployment process.

Integrations with Popular Tools

Automated tests rely on continuous integration tools to automate repeated tasks, such as retrieving source code, building the project and running automated tests. TestComplete integrates with the most popular build tool — Jenkins — via its out-of-the-box plugin. You can easily configure TestComplete tests to run within a Jenkins Pipeline alongside other your other tests to maintain a streamlined testing process.

Software teams using Azure DevOps, a collection of development, build, test and deploy tools by Microsoft, can sync TestComplete tests in real-time using the TestComplete Test Adapter. You can also link manual tests created in Azure to automated tests in TestComplete using simple drag and drop functionality. Rather than wasting hours integrating tests and generating reports, Azure DevOps and TestComplete help dramatically cuts down on the number of steps and provides a single source to view reports, enabling test engineers to focus on higher value tasks like test design.

When it comes to maintaing traceability and managing tests, TestComplete makes it easy to send the results to Zephyr for Jira, a popular tool for QA teams to plan, manage and monitor tests in real-time. These metrics help teams better understand test coverage, identify high-risk areas and recognize trends with test automation to improve their workflows.

Sign up for a free trial and discover how easy it is to get started with TestComplete — you can even import your existing functional tests to build an instant test suite!