When transitioning from strictly manual testing towards automated testing, I’ve seen several ways for the transformation to fail or stall. One of the most common mistakes is treating automation as an exact replication of manual tests. It’s helpful for us to quickly review the reasons many people begin the journey to automation and what they hope to achieve.
If manual testing is considered a constant effort over time, many groups think of automation as a method to reduce that effort. As a graph, I’ve seen variations of a chart I like to call the “Shark Fin.”
As you can see, there is a large but short initial effort to learn the tool, a more long-term phase of building tests, a sharp drop off with some refactoring, and then the effort in maintaining the automated tests is less than the equivalent manual testing effort. In other words, a return on your investment in automation.
However, what actually happens is something like this:
Manual effort is never actually constant over time, and the automation has some unexpected substantial refactoring. In the end, automation is rarely less effort than the equivalent manual testing and the team gets frustrated at the amount of effort needed to automate.
So why is there this gap between the expected results of automation and the reality? Well, experience has shown that one very common reason is that automated testing is viewed as a replacement for existing processes.
Testers and business analysts are used to thinking in workflows, and manual tests are commonly written as such. The natural inclination when automating those tests is to automate the entire workflow from start to finish.
This is a mistake however, because automation does not have the same return on investment for every step. Some steps in a test are harder than others to automate, and some steps have more reusability. This is an easy trap to fall into, because the simple workflows testers usually start with actually do have a high return, and that early success breeds repetition. But it’s not long before you run out of those simple workflows.
The way out of that “entire workflow” trap is to break your existing tests into tasks. A task is a single action or small group of actions that are the pieces of your overall workflow, often listed as steps. This requires a small shift in mindset from Test Automation to Task Automation – you aren’t automating an entire test, but a set of tasks. Once you have that list of tasks, you can rate them on two criteria: how much effort it takes to automate and how much effort it is currently taking to execute manually. You’ll quickly identify a group of tasks that have a high return on the automation investment.
There are 4 concepts to keep in mind during this approach.
1) Testing is hard work, the goal of automation is to free up resources for that hard work
2) All workflow or processes are made of tasks
3) Tasks do not always equal Tests (for instance tear-down, set-up scripts)
4) Tasks have different returns on automation investment
Common high-return task automation candidates are:
- Environment Setup / Teardown
- Data Entry
- Form Filling
- Varying data inputs in a repetitive process
- Exposing backend data (APIs, DB tables, etc….)
- Repetitive / boring tasks that are prone to inattention errors
- Tasks with high reuse value across many workflows
- Tests with timing or screen responsiveness as a criteria for success
- Many non-functional test types, such as Performance Testing
- Smoke Test Suites – key user workflows as a screening tool
- Result Capture
But remember to also ask your team which tasks they hate doing! Removing tasks that your team dislikes is the surest way to get them on-board with any automation effort.
After automating some of these tasks, the end result looks something like this:
This leads to the challenge of combining the execution of manual and automated tasks into a single test, which can be difficult without the right tools. There are two approaches, and your team’s skills, the application under test, and the percent of tasks you believe will be automated inform which approach is better for you.
- Automation tools that pause and wait for manual testing either in a form or via a button
- Test Management or Manual Testing tools that run scripts on demand
Good luck with your automation transformation, focus on efficient tasks, and please feel free to reach out for advice.
To learn more, take a look at the slides below: