If you were asked to write a test case, would you know what to do? What about a test script, or a test scenario? The first step is learning what these terms mean. Each of these terms implies a different level of detail and is used for a different purpose. Once a tester knows what each of these terms mean, they can figure out how to use them to describe the testing work that is done on a daily basis.
Test Scripts
This story begins with the most detailed way to document testing, the test script. When people talk about test scripts, they usually mean a line-by-line description of all the actions and data needed to perform a test. A script typically has ‘steps’ that try to fully describe how to use the program — which buttons to press, and in which order — to carry out a particular action in the program. These scripts also include specific results that are expected for each step, such as observing a change in the UI. An example step might be “Click the ‘X’ button,” with an example result of “The window closes.”
When a tester first starts a new job, they might not know much about the product, the business domain, or even software testing. Scripts can help bridge that gap. If the tester carefully follows the directions — enter the string ‘abc’, click the submit button, make sure the form submitted and the value was saved — the test idea will be covered enough to consider it ‘tested’.
There are a few drawbacks to consider before going all-in with detailed scripts. Active software projects change often — pages get redesigned, user experience changes, and new functionality is added. To be effective over time, testers have to make a continuous effort to update the scripts to match the new product. This can take time away from testing. Another drawback is that scripted tests are often designed to test one specific thing repeatedly, using the same steps and the same data each time the test is executed. This means that if there are bugs that lie outside the directions given in the test script, they will not be found unless the tester strays from the script. Scripted tests do not always encourage testers to use the creativity and technical skill required to find hidden bugs.
Test Cases
The second most detailed way of documenting testing work is to use test cases. Test cases describe a specific idea that is to be tested, without detailing the exact steps to be taken or data to be used. For example, a test case might say “Test that discount codes can be applied on top of a sale price.” This doesn’t mention how to apply the code or whether there are multiple ways to apply the code. The actual testing that will cover this test case may vary from time to time. Will the tester use a link to apply a discount, or enter a code, or have a customer service rep apply the discount, or will they feel compelled to test every way to add a discount that they can think of? Test cases give flexibility to the tester to decide exactly how they want to complete the test.
This flexibility from test cases is both good and bad. Flexibility is beneficial when the tester is familiar with testing and familiar with the software under test and the current set of risks in the software. If the tester clearly understands what has already been tested, what has changed recently in the program, and how users typically use the program, they will choose an approach in their testing that will exercise both the most common user paths, and the less common paths that are most likely to reveal bugs.
On the other hand, if the tester does not have a good understanding of how the program is used, the recent risks to the program, and how to evaluate those risks as a tester, they may not have the information or skill they need to assess the actions required to reveal important bugs.
Test Scenarios
The least detailed type of documentation is the test scenario. A test scenario is a description of an objective a user might face when using the program. An example might be “Test that the user can successfully log out by closing the program.” Typically, a test scenario will require testing in a few different ways to ensure the scenario has been satisfactorily covered. Just based on that light description, the tester might choose to close the program through the menu option, kill it through the task manager, turn the computer off, or see what happens when the program runs out of memory and crashes. Since test scenarios offer little information about how to complete the testing, they offer the maximum amount of flexibility to the tester responsible for them.
Like test cases, the flexibility that comes with using test scenarios creates similar benefits and drawbacks. Testing skill and domain knowledge make it easier for the tester to break test scenarios down into the relevant test ideas, select the approach that makes most sense, and perform tests that find important problems. This work is fun and challenging for a skilled tester, but it may be difficult or impossible for a novice unless they are able to collaborate with others to get the needed skill and perspective.
So How Do You Choose?
So, which one is the best to use: test scripts, test cases, or scenarios? Luckily, testers don’t need to pick just one. Test scenarios, test cases, and test scripts can be used in tandem so that testers can tune test management to the varying skill levels and domain knowledge across a group.
Have some tasks that need to be repeated regularly and a pair of hands available? Try using a test script. A robust software testing tool, like TestComplete, can use these test scripts to create accurate and repeatable automated tests across multiple devices, platforms, and environments easily and quickly. Thought of a number of ideas that need testing and have some skilled testers available to determine the best way to test them? Use test cases. Need to think broadly about the actions that will be taken by users as they traverse a new workflow? Create some test scenarios and unleash your best testers to flesh them out. All of these forms of documentation may provide value, given a clear understanding of their benefits and their limitations.
Get Full Visibility Into Your Testing Process
As you scale your testing efforts, you may find it increasingly difficult to track your growing list of test cases, as well as the defects that are identified through your testing efforts. You may also face challenges managing the tests you are running at different layers of your application — including API and UI-level tests. A test management tool, like QAComplete, provides the real-time insights into your testing state through built-in traceability across requirements, tests, and defects. A test management tool enables you to manage a complex testing process by getting full visibility into changes and their impact with versioning.
The exploratory skill set can take moment to learn, but a lifetime to master.