If you’re anything like me, you’ve seen a different process for managing tests everywhere you’ve worked. Usually, I look at a development process as being flexible to the organizations and the personalities in the organization because process is only effective if people believe in it and adhere to it, and no organization is exactly like another.
But there are some areas of your process that need some rigor if you are going to produce quality software on a regular basis. One of those areas is test management. It doesn’t matter how deeply and how well you’ve tested your software if you cannot repeat the tests or accurately describe what they were.
How often have you heard a developer tell a quality engineer that they can’t reproduce a reported bug and ask for the steps they used when they found it? And how many times have you heard a QE reply with words like “I think I…”? And, as a manager myself, I have often looked at a production bug and asked “Did we miss this in testing?”
Here are some tips for managing your tests so those conversations happen less frequently.
1. Clearly itemize each step in the test.
Nothing could be more tedious than this part. But it’s critical to write tests that can be repeated exactly by any member of the team, including the developers. Often a bug surfaces when you follow one path through a feature but not another, so having the path of the testing clearly outlined eliminates any communication problems between team members.
2. Send your tests through a review cycle.
Avoid wasting precious time and resources testing a feature inaccurately. The developer who wrote the code is best qualified to know whether a test makes sense and will actually exercise the code thoroughly. And don’t forget the designer and product owner, both of whom know the intent of the feature better than anyone. I have witnessed many times when a test plan becomes the vehicle for discovering that a feature’s intent was not clearly communicated or thought out.
3. Clearly specify pass/fail for each step in the test, not just the test itself.
By breaking your test into discrete and detailed steps, you not only ensure that the test is repeatable but you also create an audit trail for exactly where failures occur. If the tester updates each step in the test run with its pass/fail status, there is a clear record for the developer to identify the code that is called between the last passing step and the step that failed.
4. Store your tests in a common area where everyone can see/use/share them.
Ah, well, that audit trail I just mentioned is only good if the developer can see the tests and their steps. This is a common mistake – nobody looks past the bug report to the test execution report. Hopefully, the tester creates clear and detailed bug reports but why not leverage the tests themselves by sharing the execution report with the developer? Another benefit to keeping your tests in a shared repository is that each member of your testing staff can re-use and/or re-run those same tests for subsequent releases or commonly shared functionality across features.
5. Save those tests and their execution reports.
Quality reporting doesn’t end when you deploy your code to production. In fact, those test results turn to gold when something unexpected occurs in production or a user reports a bug. Reviewing the tests that were executed and their audit trail of failures can tell you if you missed a path through the feature or promoted code that didn’t pass the test execution.
We might as well admit that test planning and management are the most tedious, and sometimes grueling, part of the development cycle. But if you take the time to find the right tool and the right process for documenting, reviewing, and assessing your tests on a regular basis, you can significantly improve not only the quality of your code but also the quality of your testing.