LOGIN | REGISTER NOW | SUBSCRIBE
 
 
News on Monday
more>>
SharePoint Tech Report
more>>


   

 
 
Download Current Issue
ISSUE 2/1/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?


 
blogs tab
Visual Studio 2010 Release Candidate Available Today
A Visual Studio 2010 release candidate is available on MSDN.
02/09/2010 09:45 AM EST

Is Microsoft eyeing Office subscription pricing?
Microsoft may be preparing to offer a new Office pricing option called "union," which charges the same for cloud as on-premises.
02/01/2010 09:38 AM EST

Facebook rewrites PHP runtime
Facebook is about to open source its own PHP runtime, written from scratch for speed.
01/30/2010 08:53 PM EST

 

Events calendar tab
Macworld 2010
2/9/2010 to 2/13/2010
San Francisco
IDG World Expo

SPTechCon 2010
2/10/2010 to 2/12/2010
San Francisco
BZ Media

PyCon 2010 (Python Conf.)
2/17/2010 to 2/25/2010
Atlanta
Python Software Foundation

Southern Calif. Linux Expo
2/19/2010 to 2/20/2010
Los Angeles
SCALE

IBM Pulse
2/21/2010 to 2/24/2010
Las Vegas
IBM


 
print Printable version 
Most Read Latest News Blog Resources
Facebook brings HipHop to PHP
HipHop compiles PHP to speed up the Web layer at world's most popular social networking si...

Adobe moves LiveCycle application platform to the cloud
Release gives developers just what they need to create targeted business process-oriented ...

Syncfusion announces component suite for .NET 4
Essential Studio 2010 has business intelligence, reporting and UI controls, ready for .NET...

Digg!  Digg
Reddit  Reddit
Slashdot  Slashdot
Share on facebook  Facebook
Share on Twitter  Twitter

Debunking Cyclomatic Complexity


By Andrew Binstock
March 15, 2008 — 
There can be little doubt that metrics are emerging as a new dimension in the management of code quality. Whereas five years ago, few people except for software engineering wonks cared to run metrics on their code, now many managers are starting to view metrics dashboards as a key tool for knowing where a project stands. I suspect, but don’t know for sure, that the door was opened by unit testing: the ability to have a visual display of test results and then of code coverage stimulated the desire to obtain additional quantitative data about codebases.

The current infatuation with metrics has led to the creation of many new metrics—an explosion of measures that perfectly parallels the explosion of sabermetrics in baseball. With so many emerging metrics, it’s hard to know what is useful. So, almost a year ago, Enerjy—a company that specializes in metrics dashboards—decided to undertake extensive research into which metrics most track the likelihood of defects. They examined more than 50 open-source projects. They combed through code release-by-release and matched bug reports back to the individual modules; from this, they built up a statistical model that would identify which metrics were the best predictors of problems in code. The No. 1 predictor—I doubt this will surprise anyone—is the amount of code in a given module. The more code, the greater the odds of a bug. This seems kind of obvious: If all code has bugs, the more code in a module the more likely it will have bugs. However, as unremarkable as this correlation is, it testifies powerfully to the benefit of small, discrete methods, which is a keystone of object-oriented programming.

What the survey did not show, however, is that code complexity does not correlate directly to defect probability. Enerjy measured complexity via the cyclomatic complexity number (CCN), which is also known as McCabe. It counts the number of paths through a given chunk of code. Even though CCN has limitations (for example, every case statement is treated as equal to a new if-statement), it’s relied on as a solid gauge. What Enerjy found was that routines with CCNs of 1 through 25 did not follow the expected result that greater CCN correlates to greater probability of defects. Rather, it found that for CCNs of 1 through 11, the higher the CCN the lower the bug probability. It was not until CCN reached 25 that defect probability rose sufficiently to be equal that of routines with a CCN of 1. This is an important discovery, because it essentially states that there is no correlation between CCNs of 1 through 25 and bug expectancy. By no correlation I mean here that for half this range, a higher CCN indicates a lower chance of defects, while for the other half of the span, it implies a higher likelihood.

This range of CCNs with values 1 through 25 is important. The majority of routines written in object-oriented languages today (in fact, I would surmise, the overwhelming majority) have CCNs in this range. This means that for most code you write, CCN does not tell you anything useful about the likelihood of your code’s quality. (Above 25, CCN does correlate to greater bug probability.)

This discovery has implications. For example, a recent metric that has garnered some attention in the press is Agitar’s Crap4j, which measures unit-test coverage versus CCN. This metric was not the result of hard statistical analysis but a means of stating quantitatively that complex code needs more unit tests. This rule has been a precept at Agitar—a vendor of unit-testing software—for a long time. The code coverage Crap4j recommends to avoid crap code rises linearly with CCN, but curiously starts with a recommendation of 0 percent coverage for code with CCN of 1 through 5. However, we now know this code is equally bug-probable as code with a CCN of 20—which requires 71 percent coverage per Agitar to avoid being called crap. To be fair, part of Enerjy’s results could be explained by the fact that small routines in the CCN range of 8 through 15, which have the lowest CCN, are cleaner because of unit tests. But, in that case, Crap4j is wrong to suggest that code with CCNs of 1 through 5 need have zero unit tests to avoid being crap. They should be tested to avoid being crap.

This greater clarity is the insight that real research can bring. It shows us the danger of adopting rules based on our experience and universalizing them without checking the reality. Until this rigor is more widely accepted, I fear, we will hear about more new metrics of uncertain value.

Andrew Binstock is the principal analyst at Pacific Data Works. Read his blog at binstock.blogspot.com.


Share this link: http://www.sdtimes.com/link/31820
 

Add comment


Name*
Email*  
Country     


  • Comment
  • Preview
Loading