Integrate, Then Mutilate, Your Code

By Andrew Binstock

June 15, 2007 —

One of the most important trends emerging in software development is the adoption of continuous integration. While developers, especially agile developers, have known of the benefits of continuous integration since the late 1990s, only now is it starting to gain real traction.

Part of the long delay, I believe, is the result of the misleading name, which communicates little about the actual process. Continuous integration (CI) is neither continuous, nor does it really integrate anything. In its most basic form, CI is the practice of building the entire project whenever code is checked into the SCM. This baseline definition is a little limited as all CI products also can run unit tests on the completed build. They then use some notification system to alert developers if the build has been broken or if unit tests have failed.

The key benefit of CI is that broken code is flagged right away. This enables the developer who checked it in to get immediate feedback, rather than waiting for the nightly (or weekly build), when the details of the code’s behavior have already begun to grow dim. By immediate detection of the problems, resolution can be implemented at the earliest possible point, which is also the least expensive moment to do so.

There are many CI packages available on the market today; most of them are free, open source tools. The most well known is CruiseControl (cruisecontrol.sourceforge.net), although it has many open source and commercial competitors. (For an extensive comparison of the features of the major products, go to damagecontrol.codehaus.org/Continuous+Integration+Server+Feature+Matrix.)

Recently, the interest in CI by commercial vendors has begun to pick up steam. Agitar bundles CruiseControl in its flagship AgitarOne product. And Borland just released Gauntlet, which has numerous important features that I’ll touch on in a later column.

Unlike most server-oriented software, CI products are fairly easy to install and configure, and they integrate easily with most major SCM systems in use today. Moreover, CI servers run happily on low-horsepower hardware. After all, they monitor code repositories, kick off builds and various scripts, and then report results: nothing terribly demanding.

If you install a CI package, you’ll quickly see not only the benefits I just mentioned, but you’ll also discover that most products offer additional functionality that’s pretty compelling. For example, many CI servers run code verifiers and static code analyzers, and can reject code that falls short of shop standards. When the CI servers run unit tests, they also collect coverage data. This data and numbers garnered from other tools can be presented in either an integrated dashboard-style Web interface or a series of reports hosted on the server. The packages also offer extensive e-mail notification systems.

CI servers rely on plug-ins to do much of their work. They generally support build systems such as Ant and Maven, so it’s easy to get them to build software using the processes you already have in place. You can add more steps either through a CI plug-in or through your preferred build system. Because of this feature, I think, builds will move increasingly to a centralized CI server, rather than being distributed among participating developers or hosted on traditional build servers that do little more than compile and link.

The ability to run unit tests as part of the integrity validation of the codebase is, of course, an important feature. However, there is little that can tell the developer or the manager whether the tests are any good. Even high levels of code coverage are no proxy for test quality (quantity, yes—but not quality).

One tool that helps identify weaknesses in unit test coverage is Jester (jester.sourceforge.net). It represents a malicious idea that beneficially mutilates your code. Once your code runs JUnit tests cleanly, Jester will change one meaningful thing in your code, recompile it and rerun your unit tests. If the tests still run cleanly, Jester points out that a unit test is needed. If the tests fail, Jester moves on to another tweak. Given a long enough run, Jester will find lots of seams in your code that are not covered by tests. This will enable you to spot naked code that was not tested for an unforeseen eventuality. If your code can stand up to multiple Jester runs, chances are good you’re correctly testing all the behaviors you need to be checking. Versions of Jester exist for Python (Pester) and C# (Nester).

The great joy of writing code today is that we have wonderful tools that previous generations never dreamed of. CI and Jester are two fine examples. Make them part of your site’s build cycle.

Andrew Binstock is the principal analyst at Pacific Data Works. Read his blog at binstock.blogspot.com.

Share this link: http://www.sdtimes.com/link/30740

Zeichick’s Take: Interact with your source code Alan recounts a time when a problem was solved beyond reading code, but also interacting with it

Code Center update allows code approval grouping Building on Code Center's existing tracking features, Black Duck has added more flexibility in approving open-source components, and entire stacks of software can be approved from a business, coding and legal standpoint.

‘Your Code Looks Like My Code‘ Businesses look to legal counsel to protect homegrown code. But for open-source projects, calling in the lawyers is rarely the...

Add comment

Name*
Email*
Country

Comment
Preview

News on Monday
more>>
SharePoint Tech Report
more>>

Download Current Issue
ISSUE 3/15/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?

Google Code turns 5
Google Code Turns 5, and adds a Paxos Algorithm to make the system more stable and reliable.
03/17/2010 11:16 AM EST

Test your Visual Studio 2010 know-how
Microsoft is offering free beta certification exams for Visual Studio 2010.
03/17/2010 11:08 AM EST

Microsoft lifts the hood on IE9
Microsoft is previewing IE9.
03/16/2010 01:10 PM EST