Karl Wiegers Q&A: Peer Review, the Gift that Keeps on Giving
Collaborate | Posted December 25, 2012

Merry Christmas everyone! Last week, we published some of Karl Wiegers's answers to questions that were asked during his Dec. 5 webinar, “Peer Review—An Executive Overview.” Since everyone is in the spirit of giving, we thought it was the perfect time to give you the second installment of insight from Karl.

I have seen (many times) failures due to bad requirement gathering. Moreover, the requirement document becomes absolute during coding phase! Any best practices on business requirement documentation?

A: I’m not sure if you’re asking about best practices for documenting the requirements in the first place or best practices for reviewing requirements documents. If the former, there’s no succinct answer. There is a huge body of knowledge on good practices for requirements elicitation, analysis, specification, validation, and management. One good resource is my book Software Requirements, 2nd Edition.

If you’re asking about best practices for reviewing requirements documents, chapter 8 of my book More about Software Requirements addresses exactly that issue. The first point is to educate the reviewers so they know how to effectively participate in a peer review. Don’t overwhelm the reviewers with too much information to be covered at once. Performing incremental and even informal reviews on small portions of a growing specification is the best approach, combined with a more rigorous inspection of the completed requirements deliverable for a particular release or build increment.

Invite the right reviewers who can represent various perspectives: author of the requirements; users who provided input to the requirements; developers and testers who will have to do work based on the requirements. Have different reviewers use different portions of the checklist or different analysis techniques to look for errors. Have different reviewers start at different portions of the deliverable instead of having everyone start at line 1 of page 1. These approaches should give you better results than just passing out a requirements document to a reviewer and asking him to tell you what he thinks of it.

One of my regular statements to developers is that QA's job is not to find defects/bugs but to validate quality. Isn't that what the Q stands for?

A: In a pure sense, the role of quality assurance is to define processes that will lead to building high-quality products, and to monitor and measure the performance and effectiveness of those processes to ensure that they prevent defects and lead to products that are suitable for their intended purpose. The role of quality control is to assess the quality of products using techniques that identify defects.

Peer reviews, static code analysis, and all forms of testing are quality control activities. However, many software organizations (imprecisely) use the term QA to refer to all of these kinds of activities. Sometimes they even, heaven forbid, use QA as a verb: “Now Fred is going to QA the release.” Most of the time, when software people say “QA” they really mean “QC.”

Speaking of requirements documents, one of the important sources of downstream problems is "over-designing." This causes a lot of problems and lost productivity downstream.

A: I’m not sure exactly what you mean by “over-designing.” I can think of two interpretations. One is that requirement specifications include unnecessary and inappropriate design constraints, often imposed by people who aren’t the best judge of whether those constraints are reasonable and necessary. That is, solutions are presented in the guise of requirements. Any business analyst will tell you that this is a very common problem. It’s certainly a risk because too often those solution ideas just represent one possible way to solve the problem, and they mask the actual problem or need.

The other interpretation of “over-designing” is over-engineering, where a more complex solution is specified than is necessary to achieve the customer’s objectives. This is most common with nonfunctional requirements, such as excessively stringent performance or security requirements that lead to unnecessary effort and cost to satisfy them. Peer reviews can help detect both sorts of problems.

Should code be tested, and then reviewed, or the other way around?

A: The available literature data suggests that it is less expensive to find coding defects through peer review than through testing. Therefore, my suggested programming sequence is:

  1. Write the code and make sure it compiles.
  2. Write the unit tests that will tell you if the code is implemented correctly (you could actually do this first). Simply thinking through tests like this will likely reveal errors in the code.
  3. Execute a few simple smoke tests to judge whether the program at least appears to run.
  4. Run the code through a static code analyzer to find problems that tools are good at finding.
  5. Perform a peer review.
  6. Execute your full suite of unit tests, automated as much as possible so you can do regression testing whenever you change something.

What approach do you recommend for peer reviews of software architecture models?

A: You need to judge whether the architecture will allow for implementation of all the requirements, so be sure to have the person who wrote the system requirements participate in the review. You need to make sure that system functions have been properly allocated to components in the architecture, so have a systems engineer or another architect make this assessment. It’s important to assess whether the architecture will fulfill all specified quality attributes, performance goals, security, and reliability considerations and that it conforms to all pertinent constraints. Both internal and external interfaces are important parts of architectures to be assessed. You might need to invite as reviewers people who are familiar with any external systems or devices to which the architecture must interface. A more detailed checklist for architecture reviews is available at ProcessImpact.com

See also:



By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox