Characterization: Beyond Pure Unit Testing - Software Development Times On The Web

Characterization: Beyond Pure Unit Testing
By Andrew Binstock

March 15, 2007 — If asked to explain, practitioners of unit testing generally recommend the practice because it enables them to be sure their code does what they expect it to. It is a mechanism that gives some reasonable belief that the base being built is solid—that is, it works correctly given a variety of inputs and under varying conditions.

For me—a strong believer in unit testing—the practice has other advantages that are less discussed but certainly acknowledged by developers.

One is that I can reasonably extrapolate when I will be done with a project. The aspect that historically made guessing completion milestones so difficult was that the coding cycle generally opened onto an extensive debugging cycle whose extent and duration were unknown. This was so because developers preferred doing the minimum amount of testing necessary to move forward with feature creation. So, when the package was pushed to QA, simple tests with out-of-range values frequently revealed design flaws and weaknesses that required rewriting large sections of code. Unit testing removes this problem. If I write good unit tests, this kind of refactoring almost completely disappears. So, the estimation I am left with is guessing how long it will take to write the original code and the accompanying unit tests. In general terms, I can estimate this fairly well. And so unit testing gives me the key ability to map out work in a sensible way and proceed according to a known, reasonable schedule.

Another benefit of unit testing is my central theme. By writing lots of unit tests, I create hundreds, even thousands, of seismographs that are embedded in my code. If new code disturbs some functionality or undoes an invariant of my previous code, one of those seismographs will generate a fault message in the form of a failing unit test. This is invariably a valuable insight (and one, I might add, that is possible only if all unit tests are run regularly.)

The concept of using unit tests as seismographs that monitor existing code has been given new life recently in work by Michael Feathers that first appeared in his excellent book “Working Effectively with Legacy Code” (Addison-Wesley, 2004). In it, Feathers discusses “characterization tests.” These are unit tests written explicitly for the purpose of capturing the current functionality of code. The tests are not intended to prove the quality of code, but merely to record how it operates. (Remember the book’s theme is dealing with legacy code, which Feathers defines as existing code with no unit tests.) The benefit is that if you’re refactoring legacy code, you can tell if you’ve disrupted it when any of these characterization tests fail. When you think about

it, this might be about the only way of recording functionality in a faithful and actionable way. Clearly, deriving UML diagrams or flowcharts of the code is nearly pointless in this regard, because those artifacts cannot automate the process of telling you what you’ve unhinged and what its effects are.

However, characterization tests do have one important practical drawback: If you are actively changing the legacy code, every change is going to require you to fix or rewrite the characterization tests—in addition to the new unit tests you’re writing for your modifications. That’s a lot of tests. And at some point, especially after a big change, even developers who are sold on characterization tests give up the effort and slip back into the old ways—hoping to get by on the new unit tests, some ill-defined functional testing and their wits.

Comes now an interesting solution from Agitar. A new project, currently in beta testing (but visible at www.junitfactory.com), automates generation of characterization tests. An Eclipse plug-in sends your Java code to Agitar’s servers, where dozens of tests per class are generated and shipped back to your desktop.

These tests are not intended to find errors in your code. On the contrary, they are all designed to pass. Now then, when you make a change and a few tests fail, you do not fix them. You look at them and make sure you understand what failed and why. If you’re OK with the failures (that is, changes were noted where you expected), you simply have www.junitfactory.com regenerate the characterization tests. The new tests now reflect your modified version. You can always save the original tests if you want to maintain the original snapshot. I recommend you do this so that you can document exactly what functionally changed.

What makes characterization testing really work is the automatic generation. On greenfield projects, if the generated tests pick up a change that escaped my hand-written unit tests, I cut and paste the test into my suite of unit tests. And suddenly my code and projects are of even better quality. Nifty, huh?

Andrew Binstock is the principal analyst at Pacific Data Works. Read his blog at binstock.blogspot.com.

EMAIL THIS ARTICLE

SEND FEEDBACK

MORE COLUMNS

SUBSCRIBE TODAY!

E-Newsletters:
  News on Mon/Thurs.
  Test & QA Report
  EclipseSource

SUBMIT

JOB BOARD

PDF & PRINT EDITION

* Requires Resource Account! LOGIN or SIGN UP

* Download Current Issue!
ISSUE 8/1/2008 PDF

* Need Back Issues?
DOWNLOAD HERE

Receive The Print Edition?
SUBSCRIBE HERE

EVENTS CALENDAR

SHARE 2008
8/10/2008 to 8/15/2008
San Jose
SHARE

ACM SIGGRAPH
8/11/2008 to 8/15/2008
Los Angeles
ACM SIGGRAPH

Intel Developer Forum
8/19/2008 to 8/21/2008
San Francisco
Intel

Business of Software 2008
9/3/2008 to 9/4/2008
Boston
Red Gate Software

VSLive New York
9/7/2008 to 9/10/2008
New York City
1105 Media

MORE EVENTS

GET NOTIFIED!
About all of the latest Resources

SD TIMES 100

6th Annual SD Times 100

It's time once again to
recognize the organizations
or individuals that have
demonstrated leadership in
their markets.

HOME SITE MAP RSS CURRENT ISSUE BACK ISSUES SUBSCRIBER SERVICES ABOUT US CONTACT US BZ MEDIA