Continuous Testing Starts with Test Automation, But Ends with Continuous Learning
Test and Monitor | Posted November 16, 2018

QA teams are rethinking their testing processes and tooling in order to move toward faster, more reliable, and higher-quality software development and delivery. The key to accomplishing this is by embracing a continuous future. Continuous testing is the ability to automate your tests to obtain immediate feedback on business risks. It is the foundation of a continuous delivery engine that has not only changed the way teams structure their workflows and systems, but also has also led to a mindset shift for the development, testing, and operations teams. Teams can optimize benefits of a continuous engine by designing a continuous testing strategy that is in lock-step with their development strategy.

Quality Becomes Everyone’s Job with Behavior-Driven Development

Test automation must exist throughout the SDLC for continuous testing to exist. But it can’t just be about implementing automation. To truly leverage continuous testing in an organization, software teams need to embrace tools to accelerate collaboration and provide instant feedback for continuous learning to occur and ultimately, for continuous testing to be successful.

To many, this process sounds perfect for agile environments, but many times this is easier said than done. How do we inspire disparate teams to collaborate? How do we advocate iteration during the entire process? Behavior-Driven Development (BDD) a modern approach to software development where teams come to a shared understanding on how an application should behave to foster continuous learning. Adopting a BDD approach enables software teams to align, creating the perfect cross-section of needs and desires between business stakeholders, developers, and testers.


What is BDD vs What it is Not

BDD should not be confused with the tools that support BDD. It’s a simple approach that any software team can adopt. Below are key differences around how requirements are written in BDD and who is involved so BDD is successful.

User Perspective – BDD focuses on the end users’ view on how the application should behave.

Developer Perspective – Other approaches often focus on the developers’ view on how the application should work. For example, in this approach, requirements often start with “ability to…”

Approach – BDD is a modern approach to software development and should not be confused with tools that provide BDD support

Tools – Tools can support and accelerate BDD adoption, but BDD is not a tool. Cucumber, Specflow, TestLeft are all test automation tools that support the BDD approach, but these tools are not synonymous to BDD.

Examples – BDD is written based on examples on how a user uses a product or application and can be written in everyday English or any language. The more human, the better.

Gherkin - Many folks implement BDD using Gherkin, a language to describe a user requirement in Given, When, and Then format. However, Gherkin is not synonymous to BDD.  

Development – For BDD to be successful, it involves all stakeholders in a software development lifecycle to come to an understanding on how any application should behave. This means that business analysts, designers, developers, testers, and operations team are all responsible for understanding the users’ requirements on how an application should behave

Developers – BDD is not only used to drive development to ensure developers are building against the right requirements, but also designers to ensure their design is correct, testers, to ensure the feature is behaving correcting, and operations teams to ensure what they are deployment is accurate

An End-to-End BDD Workflow with Hiptest + TestLeft

Three key practices define an end-to-end BDD workflow that require a mindset shift for people, processes, and tooling.

  • Discovery: Shared understanding of the user requirements through collaboration (People)
  • Formulation: Examples of system behavior that are documented in business terminology and feature stories (Process)
  • Automation: Feature stories from Formulation are automated to verify the system’s behavior (Tool)

Hiptest + TestLeft, two Smartbear tools, facilitate collaboration natively by reusing assets throughout the entire development process and supporting an end-to-end BDD workflow.

  1. Within Discovery and Formulation, business goals and requirements can be define in Hiptest
  2. Then, feature files are created based off the shared understanding of requirements in Hiptest
  3. Exporting Hiptest feature files to your favorite IDE to be built is the first time toward Automation
  4. Using TestLeft, step definitions can be created automatically to test new features
  5. Then, tests can be execute in your continuous integration environment such as Jenkins as the last step in Automation
  6. And in effort to foster continuous learning, results are then exported to Hiptest, which can then be trace to a defect/requirement management tool i.e. Jira


Collaboration, Feedback, and Iteration for Continuous Learning

Once BDD workflow is integrated into a team’s development lifecycle, tools can help with providing instant feedback and outlets for integration. For example, at the discovery and formulation phase, teams can go back and forth between business requirements and feature files to come to a share understanding in HipTest. And at the automation phase, teams can review and refactor continuously between feature files and step definitions using both Hiptest and TestLeft.

A test automation framework is the building block to a continuous testing approach. When you build off automation with collaboration and instant feedback, continuous learning begins to surface and teams can begin to capture the true benefits of continuous testing.


Learn more by watching a live demo in the webinar, The Complete Continuous Testing How-To Series Part 1: How To Speed Up Your SDLC with Continuous Testing From Design to Development to Deployment