Selling the Test Automation Story: Are We Selling the Right Thing?
  July 13, 2011

Many organizations consider the idea of test automation with fear and loathing. It might have nothing to do with the technology or tools, however, but that techies are promising the wrong things.

I confess that I may not be the most expert automation engineer, so I probably have my limits with suggesting all the things that you could do to succeed at test automation. But I am an expert in false starts, brief gains, and frustrated miring, so that is a topic I feel I can talk as an expert about.

We’d all love to hit the ground running with some amazing tool that could easily, simply, and elegantly automate all of our tests. Ideally it would have a minimal learning curve, require a minimum of effort, and be supple, robust, and flexible for the long term. However, try as hard as we might, we’ve yet to meet that fabled place… well, most of us have yet to meet that fabled place.

It’s because of this that many organizations may look at the idea of automated testing with a combination of fear and loathing. It’s not because automation is a bad idea. In fact, it’s a very good idea in many circumstances. Test automation can be a tremendous blessing in certain organizations and can save hundreds of thousands of manual testing man-hours over the long term. So why are there still organizations resisting it?

If my own experience and that of other organizations is any indication, it’s because, as Bret Pettichord put it so succinctly at the Selenium Conference in San Francisco this past April, “We are selling the wrong story.” When we talk about test automation, there is a set expectation, that of turn-key simplicity. Our listeners assume that the tests start, they run for a period of time, and then a nice formatted report comes out telling us everything we need to know. In this vision it’s effortless, seamless, and, again, elegant in its simplicity.

That’s the dream. In reality, software test automation requires a lot of effort, up-front investment of considerable time and resources, and solid toolsmithing skills to make an effective framework. It’s true that anyone can record and play back a test one time. But to make tests that are robust, flexible, and error-proof for long-term deployment, those who will be writing and maintaining test scripts need genuine software development skills and understanding of development principles.

While this may seem a bleak picture, let me assure you it is not. I am a firm believer in the ability of test automation. It has its place, and when deployed in the right way, with the right expectations, it can be a tremendous boon to an organization. To experience that boon, however, we as testers need to realistically approach the subject, and we need to be armed with what we really can and can’t do.

I suggest that you not even use the word “automation” when you begin the conversation. That’s a loaded term that comes with a tremendous expectation. Instead, sell your organization on “computer aided testing” first. This is a more realistic and attainable goal. Whether we are testing compiled applications, web services, or mobile apps, the best early wins are finding ways to automate our most repetitive tasks and to use them to our advantage.

Next, focus on numerous small tools that can be used over and over again. The “computer aided testing” that provides the most value are those steps that help us remove the most tedious work from our plates so we can concentrate on the higher level issues and challenges. You and I might automatically apply a label of “test automation” to this process, but our listeners may not – and thus won’t carry along the expectation baggage.

From there, I find it helpful to take a page out of the Behavior Driven Development approach of “Red, Green, Refactor.” Even with simple tests, we can use this approach to understand what our tests really need to do. We get the tests to pass and then re-examine the tests to make them more efficient where we can, and get development’s buy-in to help us do so where we can’t. Over time, these small steps can build into an environment where a more traditional automation approach can grow and thrive.

Whether you are new to the world of automation or a seasoned veteran, helping set the expectation up front and taking small steps to prove the value of your computer aided testing opens doors to further and more in-depth automation opportunities. Just as a child needs reassurances that a situation isn’t as scary as she thinks it is, organizations need to be convinced that what’s being offered isn’t as scary as it sounds. Set an appropriate and realistic expectation and then deliver on that. While I won’t say the approach thereafter will be easy, I can say that, with this approach, it will be easier than over-promising and under-delivering. Deliberate and steady wins this race.