Going Agile Using Test Automation
by Ian McLeod,
I am a big proponent of short iterations in Agile development. Most common software development problems go away. Doing iterations well means you get immediate feedback from users, you don’t have a big backlog of defects at the end, and you gain an objective measurement of progress.
So what’s not to like?
Ask your testing team. Testers don’t always like the very short test and regression cycles driven by iterative development.
Regression testing is cumulative. As features are added to each iteration, there is more to test to make sure the current iteration did not break anything in previous iterations or releases. I have seen this every time I introduce iterations to a development team. As the product grows, so does the testing effort, and all the pressure to get things done in those short iterations is on the testing effort.
This reality is usually the most difficult part of making the transition to iterative development. Last place I introduced iterations, we seemed to be able to fit less and less new development in each successive time boxed iteration. Why? Because after a few iterations it took the whole iteration to run the regression, even just selective regression.
How do we fix this?
Should the department hire more testers? That’s expensive, and probably won’t make you popular with your CFO (especially if you just sold him on Agile because it would lower development costs). Test less? That means defects aren’t discovered and fixed in the iteration, which defeats one of the key objectives of good iterative development.
The solution is automation of regression and unit tests. Ideally, automation runs every build, so the team knows right away if something is broken, and can fix it while the programmer still remembers what he was thinking when he wrote the code. Automating the regression tests also frees the test team from repetitious, tedious testing – the kind that makes the day long and boring and allows them to spend time thinking about creative ways to test the product.
But, but – there is no time to automate! We have a schedule! We have a marketing team breathing down our necks, asking about our progress like a three-year-old whining in the back seat, "Are we there yet?!"
It’s true that automation is not free. But it still can be accomplished. Here are a few things I have done to find time to automate when there is no time to automate.
- If there is a backlog of tests to automate, add an automation iteration into the release. This should be one iteration, with no new product features. Everyone (developers too!) works on automation of the backlog of tests. It’s planned and implemented just like any other iteration, with working tests as the objective. Interestingly, I’ve found, this process often exposes quite a few product defects as well.
- Adopt a new viewpoint: A feature is not done till it works and there is an automated test to validate that completion. Yes, this might mean you have fewer features in each iteration, but with the accumulation of automation it probably results in greater velocity overall. In the long run, we probably spend more time regression testing features than we spend during the initial development.
- Create a position of an automation architect, someone who is responsible for the overall automation framework. Test automation design is no different than software design; good architecture means easier maintenance and a better foundation for future development. With someone focused on this, maintenance of the automation will be much easier. This can also be a great career growth path for members of the team.
- Recognize that you don’t have to automate absolutely everything. Focus on the most commonly run tests or procedures, or those that take the most time. A relatively small percentage of automation can still generate a big ROI.
Well executed iterations, with test automation that allows immediate discovery of defects accelerates development and frees the team to do the interesting, creative work that makes projects successful. It takes some work to do it well, but the payoff is extraordinary.