Ministry of Testing announced their latest 30 Days of Testing Challenge, and for the month of March, they’re talking all about software testability.
But what is software testability and why does it matter? Why should we care about testability at all? Let’s start with the basics.
What is Software Testability?
Isn’t every application testable? Most probably are, but based on the system design and architecture, it could actually be easier or more difficult to conduct tests and find bugs.
There are many ways to define testability. At the most basic level, testability considers how easy software is to test. Another way to look at testability is how likely testing will be to expose faults in an application.
This makes sense if you think about the nature of a tester’s job -- sometimes, it seems no matter how many tests you run and how much coverage of the application there is, bugs still seem to slip through to customers.
In this way, testability is a product of effective communication between development, product, and testing teams. The more the ability to test is considered when creating the feature and the more other team members ask for the input of testers in this phase, the more effective testing will be.
Additionally, testability is heavily dependent on the requirements set and the understanding of what needs to be tested. For example, a requirement might be that the application should easily allow users to go from the home page to check out.
However, this doesn’t consider determinants such as the product(s) we want to check out, when the test is considered to pass (at the checkout page vs the order confirmation page), or even what’s considered “easy” -- should it take a certain amount of time or should there be a maximum number of actions, or something else?
Of course, there are many factors that could affect testability, and there’s no good way to give an exact measurement to it, but the more knowledge you have of the systems under test, the more straightforward testing will be, and the more valuable the results will be to the entire team.
Why is Testability Important?
You may not feel like you have much say in the testability an application, but the idea isn’t just to make your life easier as a tester, it’s also to provide more value to your team.
Making a case for software testability and advocating for how it affects the end product means more consideration will be given to this idea in the crucial phases before the product hits test, such as planning, design, and code review.
Testability is determined in the design and development phases, but it often gets overlooked for other requirements such as usability and functionality since the application is usually built for the user rather than for the QA team.
However, creating an application that’s highly testable will actually benefit the user in the end, as well. Testability impacts deliverability. When it’s easier for the testers to locate issues, it gets debugged more quickly, and the application gets to the user faster and without hidden glitches.
Additionally, testability will help product and development teams as well. By having higher testability, those teams will benefit from faster feedback, which will allow more frequent fixes and iterations.
We often talk about shifting left and thinking about quality sooner in the software development life cycle. Rather than waiting until test, having a whole-team approach to testability means giving your application thoughtful consideration during planning, design, and development, as well.
This includes emphasizing multiple facets such as documentation, logging, and requirements. The more knowledge a tester has of the product or feature, its purpose, and it’s expected behavior, the more valuable their testing and test results will be.
There’s also something to be said about the need for communication between various team members and discussing what would help increase testability.
By advocating for testability during earlier phases as well as after the product is released, testers can do their jobs without wasting the time figuring out what a feature or function is supposed to do, what changed in the application, or how users will be handling it.
Testability vs Ability to Automate
Sometimes when thinking about testability, we might confuse it with ability to automate or “automatability.” Rather, the ability to automate asks whether there would be a benefit in running the test more than once.
While the two are certainly related and will likely influence each other, something might be highly testable but not be a good candidate for test automation, and vice versa.
When we talking about software testing and testability, we care about what new information humans are able to derive from the application during a test. With automation, you are often “checking” that a function is working with another machine.
Deciding what to automate is a conversation it itself -- prioritizing tests and what the value of automating them would be is important to do before including them in an automation suite.
Just because you receive a “pass” or “fail” after an automated test, doesn’t mean that the information is valuable to the team. Additionally, some tests may be difficult or impossible to automate.
It’s important to understand whether or not automation is the right fit for a certain test before you spend the time creating and maintaining the test in an automation suite.
However, it’s also important to take a step back and determine the testability of an application first. While testability will usually inform whether or not the test is automated, teams may make the mistake of skipping testability and going straight to automation or automatability.
Making Time For Testability
At its core, testing aims to verify whether or not something works. However, many things can interfere with how effective that test is -- requirements, conditions, environments, tools, knowledge, data, documentation, and ego are just a few.
While testability might mean something different to every team, it’s important that you determine what measures will help your organization improve feedback and feature releases.
Additionally, while the word “testability” might infer that this is the job of your QA team, the reality is that testability is a whole-team approach to quality that requires consider before, during, and after the release.
As we discuss shifting left and making quality a priority sooner in the software development life cycle, testability should be part of the conversation. By designing an application to be inherently testable, there are fewer obstacles to releasing a functional, high-quality application.