Trends in Software Testing: A Report from the CAST 2012 Conference

  August 16, 2012

Professional software testing is evolving, just as are other software development endeavors. Matt Heusser learned about the coming trends that will affect your career, from philosophical approaches to the job skills you need to acquire to assure continued employment.

Funny thing. We care desperately about software quality when we need that software. If an e-commerce website goes down for more than 30 seconds, or has a data breach, it’s front page news. Yet, so many companies downplay the importance of software testing. Quality assurance gets little time, energy and attention until something goes wrong. That dichotomy is a frequent frustration for professional software testers, and the issues contributing to it – at a personal career level, x, and y – were among the topics discussed at the Conference for the Association for Software Testing (CAST) in July 2012.

A Matter of Approach

The very definition of software testing is up for debate. As Michael Larsen, a tester at, explains, most companies adopt one of three testing approaches:

  • Programmers write developer tests that are tightly coupled with the code; they may not see testing as anything more than “part of the job” of programming.
  • Testers use tools like Selenium or TestComplete to “drive” automation, often in a record/playback way.
  • The third school of thought emphasizes the thinking, human tester in the moment, doing exploratory testing. That’s valuable, Larsen says, but tends to be very expensive.

Larsen’s “Balancing exploratory testing with test automation” presentation


Instead of using one of the three approaches, Larsen suggests blending them all. Automated scripts can generate test data, parse I/O, log actions, perform clearly defined specific actions, and search for patterns more quickly and cheaper than a human can, he says. What automation scripts cannot do is be curious, try new ideas based on what they learned during test execution, notice something truly different and unplanned, or make a judgment call.

One way to blend approaches, according to Larsen, is to use automation to drive to a specific place, then stop and explore from that place. He calls it a “taxi cab” approach.

Want More QA Skills? Learning Programming. Redux.

The folks at CAST draw strongly from the third group, which holds up the thinking, exploring tester as an example. And, according to many people, they are in danger of becoming irrelevant, if not a footnote.

In her keynote address, Elisabeth Hendrickson, principal consultant of a studio in the Bay area, also played on this theme that the view of testing is changing. In her case, she posits, the market is requiring more technical skills. Using research from Craiglist for 2010 and 2011, Elisabeth analyzed the nature of advertised testing jobs. She found that open jobs increasingly require programming skills, such as SQL, Selenium, Java, or Ruby. In fact, 80% of the jobs she examined on Craigslist required programming skills.

            Elisabeth Hendrickson on “The Thinking Tester: Evolved”

Hendrickson was not alone in this observation. One recurring comment I heard at the conference was, “This is the last generation of non-programming testers.”

This kind of rhetoric isn’t new; the first time I heard a line like that was ten years ago.

The two sides of the argument seem lined up against each other: One suggesting that testers will all become programmers, the other that testing skills and programming skills are distinct and unique.

My belief is somewhere in the middle. 

Many companies known for styles of testing that use heavy automation, like Microsoft with its Azure cloud, are also known to let bugs slip though. The Azure leap year bug, for example, looked remarkably similar to the leap year bug that took out the Zune three years prior.

In my experience, it takes a mix of programming skills and other “breaking” skills to have a successful test effort. This diverse team might include some toolsmiths, but might also include subject matter experts, risk management experts, and, well, professional breakers. While I do see all of society becoming more technical, I am skeptical of “all or nothing” arguments about the future. 

At the end of her keynote, Hendrickson agreed with this, and told the audience not to pick up a Ruby or Python book if they really had no interest in the subject. If testers have value and focus on increasing that value, she says, they can do just fine.

Choosing QA Skills To Focus On

I talked with Nandini Javagal about the immediate “job skills” scene one night at a meetup she organized for people interested in Continuous Delivery.

Javagal was frustrated. She was trying to hire strong Automation Engineers, people to create the build, test, and deploy systems she needed to enable her developers. It’s one reason she attended the conference. But she could not find anyone (available) with those skills at CAST, not even anyone with interest in building those skills.

This was no huge surprise for me. As much as Nandini knew and appreciated exploratory testing, she was looking for more hard technology skills. Not the person who could serve as product owner for the randomized testing in Anand Ramdeo’s talk, but the one to build it. 


Attending the board of directors' meeting then getting on a plane to head home was a bit of a letdown. It felt like the great debate, the one to get all these issues out in the open, never really happened. Even though the conference was scheduled in an area to attract a lot of startups, the Silicon Valley crowd never showed up. The rhetoric came out that all testers should be certified, or learn to program, but the two sides never talked to each other. The most popular blog post to come out of the event may just be a blog post that identifies the issues by listing controversial tweets – but it doesn’t resolve those issues.

My overwhelming desire, my one biggest takeaway form CAST, is that we need a multi-discipline dialogue about what testing is and how it contributes to the value delivery process.

Where do you think the trends are in job skills for software testers? Let’s talk about it in the comments.


See also: