QA Testing For What Your Users Really Want

  August 06, 2015

The software world in general is moving toward increased shared responsibility and individual ownership. From developers to operations to testing, the microservices world has us focusing on projects not products and watching teammates going from firmly defined roles to everyone doing a bit of everything. Agile programming pushes team members to work together, while scrum rearranges it so frontend and backend developers are often performing their own quality assurance testing.

So why wouldn’t your quality assurance team become more customer-facing? Why wouldn’t a quality analyst mimic user experience design best practices? There’s no doubt that QA affects usability and user experience, but now it’s time to get more deeply involved with it.

Usability Testing and User Experience for QA

Usability testing is when you test directly with potential users to see how they can or cannot accomplish the tasks required. It goes past the testing for functionality and brings the human user into it all, to see if the app, process or application programming interface or API actually meets the so-called average user’s requirements for ease of use and functionality. Basically, before throwing it out into the wild, you get real people to see not only if it works, but if it works well as a solution to their needs.

Why try usability testing? Guaranteed, people will use your product in ways you never could have imagined. For the entire team, usability testing is a great a way to uncover unexpected issues, find what is unnecessary or unused before going any further, and just get bold, unadulterated opinions from an outsider.

Problem is that usability testing is usually perceived as expensive and time-consuming. But it doesn’t have to be!  Your first users and testers should be your own people. Friends, family and especially colleagues—you all need to eat your own dog food and such. But then it’s time to get out in the real world.

Why not try guerrilla usability testing? David Simon, experience designer at ThoughtWorks, explains how to do guerrilla testing in the simplest way. Once a week or so throughout a key project, his team works on testing different prototypes—from napkin sketches to fully functioning products—on strangers.

Organize your usability test by answering these four questions:

  1. What will we test? You want to test as much of your product as you can, but particularly focus on the aspects you and the design team are less confident of and those that you know are most important to the user.

  1. Where will we test? Simon gives the example of if you are working on a mobile app for a store, go to that store and spend time with a couple customers, as well as a few employees.

  1. Who will we test with? Go ahead, pounce on random strangers in cafes or, if your user is a developer, coworking centers, MeetUps and hackathons are always good places to find willing guinea pigs.

  1. How will we test? Watch how your testers use the app and ask them to talk aloud, describing what they’re doing and how, what would they do next, etc. Try to note both the opinions they give and how they are actually behaving. After you do this, perhaps ask a specific open-ended question or two that addresses a specific area of concern to your project.

As Andy Budd wrote: “The object of this is to gain powerful insight into the thought process behind the user’s actions.” Just keep the test short or their minds will wander!

Don’t forget to end by asking if they would recommend this product or service to their friends! (And make sure to look them in the eye in case they’re just being too polite.)

You aren’t trying to sell anything, so don’t feel shy about inviting people to a coffee or pizza in exchange of their thoughts. Be as transparent as need be without giving away the cow of your secret project. Try to keep it casual, quick and simple, by focusing on a specific part to test and getting their gut reactions, jotting down any hiccups.

And if it’s the right setting to do it, think about recording the session. To seem less creepy, you use this camera hack so to record all sound but only video of the screen.

How often should you get involved in these tests? You should work to be customer-facing about every couple weeks because otherwise you’ll get out of the habit and you’ll lose that empathy you’ve been building up.

Look at it as a fun way to get out of the office for an hour to stretch your legs—it’s more fun than you think, we promise!

Why Should QA Be Performing Usability Testing?

It’s clear why website and app designers should perform guerrilla usability testing—it saves money in the long term while limiting risk in the short term. But why should you as a tester or quality analyst care?

It’s kind of like a Seurat painting. We often spend so much time looking at the details that we don’t back up and look at the big picture. By taking time to interact directly with users, you are ensuring you’re becoming a better quality analyst. Even if you’re focused on just testing, it makes you test differently.

It makes sure that when you’re testing, you’re also considering how long is should take people to do something, in how many steps, and constantly taking into account how the user would rate the system.

Former director of project management and quality assurance, turned agile coach David Dame says he usually drove his organizations to experiment with cross-functional collaboration like this.

“In large organizations, the development teams get pushed distant from the customers. As the company grows, they have different layers of bureaucracy from the dev team.” QA becomes so focused on the product not the customer that the whole experience is being distilled down to do the requirements, losing the human interaction.

Quality assurance involves following predefined test cases that meticulously outline the required functionality, action to be taken and the desired outcome. This is an effective way to identify any specific implementation issues, but it does nothing for measuring the quality of the user experience. Usability testing trains you to expect the unexpected because we all know that users use our systems completely differently than we thought.

“By getting the QA to work with the user experience and work with the usability, they gain empathy. They can be another channel of communication on how customers use the system. Then they begin testing the system not only as a set of requirements but for use.”

Dame says QA absolutely “should be involved in working with the customers, seeing how they’re using the set and looking for ways of bridging what the product can do and making it available to the customer.”

He even goes as far to advocate that the QA team should be educating the customer on what the system could do, which further checks for usability and helps learn the best way to engage those users to use those features.

Dame advocates for QA being a part of usability testing and even going through prototype mockups. He says you can run the tests or at least listen in on the tests while the UX people run them. By putting QA in front of the customer they are able to break through natural bias to see what the real requirements the clients have.

Plus, learning “how the customers use those features will help the QA testing as well. They use it in ways we didn’t imagine.” He talked about cases varying from power user moves down to never thinking that an administrator would drag and drop all folders into one spot, which led to working with UX to help users change file names more easily.

It’s an increasing trend for team managers, like Dame, to encourage all team members to get involved with usability testing and gathering user experience feedback. “What motivated me to do that was that we kept getting bugs that didn’t match our test cases. ‘How do we keep missing this?’ It was hard to see what the challenges were and the UX people we couldn't’ get to them because they were always on the customer side. There was a gap between what they were doing for requirements and what was being passed down” from the customer-facing side.

Plus getting QA involved in usability testing doesn’t just make you a better, more customer-focused quality analyst. It makes everyone involved better team members. As the service-oriented and containers movements has given each of us a sense of greater ownership, it’s important to work as a team in certain projects like this. By working with the other departments to get a grasp on customer user experience, you are inherently reminded that you are all in this together.

“They get to see the customer and to build a relationship with the UX side, so, if they can’t be on the customer side, the UX relationship is made,” so you’re better kept in the loop.

In exchange, the UX team can see features that are coming up the pipeline, so they can respond more quickly to what they think their customers want or need. You can even go full circle by bringing the customer support team into these usability tests.

What’s your experience with usability?

Have you sat in on or joined a usability test? How do you work with the UX team? Share your experience below!