Not All Metrics Are Evil
Develop | Posted August 05, 2009

Tom DeMarco recently published an article called Software Engineering: An Idea Whose Time Has Come and Gone? (pdf). It caused Jeff Atwood's head to explode.

My head did not explode, but I did post a tweet on the SmartBears Twitter account that called out what I thought was the most obvious part of the article:

This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value.

That makes complete sense to me - after all, the less potential value a project has, the more important it is to control costs.

The more interesting portion, though, was this:

The book for me is a curious combination of generally true things written on every page but combined into an overall message that’s wrong. It’s as though the book’s young author had never met a metric he didn’t like. The book’s deep message seems to be, metrics are good, more would be better, and most would be best. Today we all understand that software metrics cost money and time and must be used with careful moderation.

Agreed - however... there are many software development metrics that can be collected with minimum cost: code size/complexity, unit test code coverage, team velocity, etc. Even though collecting them is cheap, you still have to use caution when applying what the metrics tell you. For example, don't use the latest code coverage report to berate the developers, instead use it as a guide for where other types of testing might be done instead.

This is why our peer code review tool collects metrics automatically: so that developer time is spent on doing the work instead of measuring the work.  The same guideline I stated above applies: don't use code review metrics for the wrong purposes; instead use them to learn about the state of the source code as it changes (KLOCs reviewed, defects detected, defects/KLOC, etc.).

So not all metrics are evil. If you can automate the collection of the metrics and thereby reduce cost/intrusion, then the only
concern is to make sure you use moderation when interpreting those


By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox