Building Quality Software and Processes with CMMI and Peer Reviews
Develop | Posted April 26, 2011

Customer Guest Blog

Glenn D. House, Sr.

President and CTO of 2Is Inc.

As a serial entrepreneur in the Boston Metro area and a member of the Entrepreneurship Council at Washington University in St. Louis, I am often peppered with pleas for advice, investment opportunities, or just polite scallop-on-a-toothpick mixer conversations about the latest fabulous, social-networking, cold-fusion Google-making software project. Two of my favorite questions to ask, however, always draw blank looks: First, “How are you producing your product to meet market demands?” Second, “Once you’ve produced the product, how do you know it’s what you intended to produce?”

After they cock their head and I re-frame the question as “What’s your software process?” they smile and nod their head in manner reserved for their aging grandparents.  "Software process, that's for big companies”; or “Software process is for the older generation, we are the Internet generation -- we are smarter now”; or my personal favorite “We don't need a process, we are just generating a phone application and we have to get this done now." The hits just keep falling from their lips.  This thinking extends not only to new ventures; but also to small and midsize software teams led by seasoned veterans. The sad fact is that ad hoc process is still the too common practice. I cringe listening to the pronouncements of misguided psyche because I have attended way too many follow-up meetings with aforementioned founders and teams and heard the “news” that a critical defect, requirement or other act of God has caused the project to derail, cascade hopelessly over budget, or worst of all, anger the customer.

Many are discouraged from software process by the perceived overheads of cost and time, but we have found that these factors are substantially reduced when a process model is paired with a complementary set of Commercial-Off-The-Shelf (COTS) tools.  Two of our favorites are from SmartBear Software: TestComplete, which we use for automated testing, and CodeCollaborator, which is an excellent tool for supporting an essential component of any software process, and one that we especially value: peer reviews. We have successfully integrated CodeCollaborator into our development model, which is based on the Software Engineering Institute's Capability Maturity Model Integration (CMMI) process. Few COTS tools contribute to so many of the required CMMI process areas as CodeCollaborator.

I am not a consultant; we make our money by delivering software that delights our customers.  I strongly believe that software process separates the "wheat from the chaff." We are a small development team and could not achieve our level of success and quality without a serious, documented and an auditable evolving software process. According to a CMMI study[1] we are only one of approximately fifty development teams in the US to achieve CMMI Level 3 based upon our team size.  We are actively working to achieve Level 5 maturity by 2013. This is not to say that CMMI is the only appropriate software process; there are numerous processes with benefits that support one organization’s goal better than another, and the CMMI model has benefited our organization. 

Contrary to popular folklore, CMMI does not tell you how to develop software; it merely provides a rigorous, auditable framework to support your choices in development methodology, whether it is classic waterfall, spiral, agile or another style. However, the most well- intentioned and well-documented software process is just a heap of worthless paper and electronics bits if the group dynamics of the development team don’t embrace it, and this is where peer reviews are significant. In my experience, no amount of heavy-handed dictums can compel a team to follow a process; the team must see the value, experience it and develop a desire to follow it, thereby making the process a living, breathing presence. There are many advantages and myths that have built up around peer reviews and CMMI[2] (I recommend that you read the white paper referenced here) but I will focus solely on the group dynamics of peer reviews.

How do you get busy and jaded developers to want to perform peer reviews; how do you expand the number of artifact types beyond source code; and last how do you get stakeholders (other than developers) to participate?

How do you get busy and jaded developers to want to perform peer reviews?

Developers must see the inherent value of and be easily able to participate in peer reviews. CodeCollaborator is an ideal platform because it lets the developer, whether author, participant or observer, make contributions to the review at the time and venue of their choice.  Developers have flexible hours and workplaces; getting a development team physically together often means imposing some draconian meeting time and place or leaving out a key individual. Then there is the fact that developers don’t like to be told what to do!  With CodeCollaborator, though, the author publishes the artifact(s), writes advice, invites participants and observers and kicks off the continuous virtual meeting.  CodeCollaborator allows for full interaction and pushes notifications without being overbearing.  It can be customized to allow you to use the same defect designations from your defect and feature tracking system (see Configuration Management in CMMI) which means the developers don’t need a separate peer review vocabulary.  Finally, the author can review the submitted comments as they are made or all at once at their in their own time.  They can even make changes and incrementally load new versions so those who have not yet performed the review can benefit from those who have. CodeCollaborator makes peer reviews convenient, allowing developers to benefit from the peer reviews without going through great pains to execute them.

How do you expand the number of artifact types beyond source code?

It is well documented that finding errors in requirements has a greater impact on quality than finding an error in source code.  CodeCollaborator supports text documents, PDFs, word documents and spreadsheets; this means that performing a peer review on a Concept of Operations (ConOps) or requirements document is just as easy as performing one on source code.  Because CodeCollaborator records all of the comments and defects, metrics can be established that allow for process optimization required by later states of the CMMI model (see Measurement Analysis [MA], Causal Analysis and Resolution [CAR], and Product Quality Assurance [PPQA]); the beauty of it is that this collection and organization is an automated side-effect of the review, eliminating the traditional scribe role.  More documents and more data (which are easily manageable through CodeCollaborator) equal a better controlled process. 

How do you get stakeholders (other than developers) to participate?

External stakeholders (customers, executives, sales, and marketing) have valuable inputs for the development team. We have encouraged customers, subject matter experts and others to peer review ConOps Documents[3] as well as Use Cases and requirements. CodeCollaborator is easy to use and can be taught to a non-technical participant in a matter of minutes.  Given that CodeCollaborator allows you to expand your document set and that peer reviews are collaborative and have temporal and venue flexibility, it is possible to easily incorporate the feedback of a wider audience into the peer review process.   Collecting feedback and identifying defects and omissions has paid for our investment in CodeCollaborator many, many times over.

In conclusion, it is my opinion that the leveraging of an integrated set of COTS tools is essential to the successful and efficient implementation of a software process for a small to mid-sized development team.  CodeCollaborator is one tool that has contributed to our success. In addition to its many other CMMI-friendly features, CodeCollaborator's overall flexibility in document types, timing, and review location allows for increased participation in peer reviews, making it an important tool for process-oriented development.


Glenn D. House, Sr. is currently President and CTO of 2Is Inc. 2Is is an audited CMMI Level 3 software organization developing enterprise supply chain Business Intelligence Solutions.  Glenn is an IEEE Computer Certified Software Development Professional, Certified Data Management Professional, Certified Business Intelligence Professional, a Senior Member of the Association for Computing Machinery and a member of the IEEE.  His experience in being Sr. Vice President of Strategy and Operations for Mentor Graphics (NASDAQ:MENT), President and CEO of Webspective Software and President and CEO of Allpoints Systems has given him a unique perspective of software processes in both the large and small company environment. Follow him on Linkedin.

Join the CodeCollaborator User Group on Linkedin

[1] CMMI for Development SCAMPI Class A Appraisal Results 2010 End-Year Update, SEI Carnegie Mellon Institute, Pittsburg, PA, March 2011

[2] Debunking 3 Common CMMI Peer Review Myths:  How Peer Reviews Help Achieve CMMI Compliance and Higher Performance, Paul E. McMahon,

[3] IEEE Guide for the Information Technology – System Definition – Concept of Operations (ConOps) Document, IEEE Std 1362-1998, IEEE Computer Society, The Institute of Electrical and Electronic Engineers, Inc.


By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox