Defining Efficiency in Software Testing
As Keith Klain explains in the video above, there’s no magic bullet that will help you make your software testing more efficient. Efficiency in testing is accomplished by asking the right questions and applying the correct techniques to answer those questions.
Take Pair-Wise testing. This is a mathematically-based technique that can greatly reduce the number of tests needed to adequately cover a set of combinations.
Let’s take a look at some font controls as an example.
This requires 48 tests for exhaustive testing (for anyone playing at home, you can’t have sub and super-script on at the same time).
But only 8 pair-wise tests are needed, which cover all possible combinations of two (hence the term Pair-Wise) of these variables.
Now, this can be a way to efficiently test specific questions regarding combinatorial problems - it reduced my test load by 40 tests! Unfortunately you can‘t just apply this across the board to all of your tests and call that efficiency, because for most tests you would not be answering the right questions. Other tests may require different techniques to answer your question.
Testing against a user story? Have a manual tester follow that story, perhaps schedule a session or try some exploratory testing to deviate from that story a bit. Nothing, yet, beats the on-the-fly analysis of the human brain.
Testing a log-in page? Set up an automated test to quickly log in using a large list of possible user names and passwords. Populate the list with SQL injections and names with multiple apostrophes and unusual capitalizations. Each individual test is so fast you can test with far more data than you would ever want to manually type.
Building your own testing technique knowledge base, analyzing what questions your tests are answering, and then applying the correct technique to that question is the best way to efficiently test your software.