The Finesse of FitNesse

By Allen Holub

September 15, 2006 —

Last time I heard Robert Martin talk, he was waxing poetic about a testing framework called FitNesse (www.fitnesse.org), which I finally got around to checking out this week. FitNesse is an interesting amalgam of a test-suite driver and a wiki. You define your tests as tables, using standard wiki notation (vertical bars to separate columns, for example) embedded into a standard Web page. If you don't speak wiki, you can cut and paste the tables from a spreadsheet. You can also put normal English on the page, so you can easily write about what the test is actually testing, provide links to other tests and so forth. The page also includes a "Test" button, which when clicked, runs the tests and colors various rows in the tables to indicate where the test succeeded or failed.

These tables are front ends for methods in test classes (called “fixtures”) that you write, and different fixtures handle the tables in different ways.

For example, the “ColumnFixture” treats the columns as either inputs or expected outputs of methods in the fixture class (which you write). The first row of the table identifies the actual fixture class to instantiate to run the test, the second row contains either a method name (which represents a desired return value) or argument names. You specify sample arguments in the argument columns, and the expected return value in the column labeled with the test-method name. When you push the Test button, the system calls the method as many times as there are rows, passing it the arguments you specified and checking the expected return value against the actual return value. If they don’t match, the cell is colored red; otherwise it’s green.

(That’s one thing that’s both good and bad about FitNesse. In a test-driven-development environment where you, the programmer, are running test suites manually every few minutes, this sort of visual feedback is great. In a large fully automated test battery that’s running unattended overnight, I’d rather have just a list of failed tests than pages of visual candy.)

Other fixtures handle things like testing a database (assuring that a result set or a list of returned objects are what you expect) and testing UI code (by simulating the effect of a “submit” button push, for example).

FitNesse is actually just a front end for Ward Cunningham’s Framework for Integrated Testing system (fit.c2.com). FIT is described nicely in Rick Mugridge and Cunningham's book "Fit for Developing Software" (Prentice Hall, 2005). FitNesse surrounds FIT with a wiki and provides page updates that show results, etc. FIT does pretty much everything else. FitNesse supports Java and .NET out of the box. With a little work (downloading a special version of the server), it also supports C++, Delphi, Python, Ruby, Smalltalk and Perl.

Strictly speaking, FitNesse is a framework for acceptance tests—tests that are written by the user to test that a system behaves the way that it should. Though you could use it that way, many naive users will nonetheless be confused by some of the things you need to do to hook FitNesse to the code (like specify class paths). On the other hand, any user can go into a wiki page that you’ve set up and tweak the values used for testing.

Using FitNesse only for user-supplied acceptance tests would be a waste, though. It’s a great tool for easily exercising your unit and integration tests as well. FitNesse is not a unit testing framework, however. It’s more of a test driver that exercises the unit tests in an organized way that makes the results of the tests easier to see. You still have to code the actual tests, and that’s real coding, not something that you outsource to a bunch of junior programmers or delegate to a bunch of monkeys who pound randomly on the keyboard.

There are bunches of things about FitNesse that I really like: There are no hideous XML files or special magic functions to write. (Everything’s done with the magic of introspection.) The classes you do write are very lightweight, easy to maintain and easy to organize. The tests are not embedded in the code, which encourages you to test the objects in your system as black boxes. (If a test exploits knowledge of how an object works internally, if you need to change the object’s implementation, you need to change the test, too. That’s a bad thing, because, should the test fail, you don’t know whether you just introduced the bug into the object itself or into the test. It’s best for the actual tests to not change, ever.)

The main thing about FitNesse is that it’s stunningly easy to specify or modify a set of tests by just typing into the wiki. There’s really no programming at all (unless you think of FIT as a table-based programming language, but it doesn’t feel like programming). Any system that makes it this easy to specify and run tests is a wonder, and with testing this easy to do, there’s no excuse not to do it.

Allen Holub is an architect, consultant and instructor in C/C++, Java and OO Design. Reach him at www.holub.com.

Share this link: http://www.sdtimes.com/link/29545

Add comment

Name*
Email*
Country

Comment
Preview

News on Monday
more>>
SharePoint Tech Report
more>>

Download Current Issue
ISSUE 3/15/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?

Google Code turns 5
Google Code Turns 5, and adds a Paxos Algorithm to make the system more stable and reliable.
03/17/2010 11:16 AM EST

Test your Visual Studio 2010 know-how
Microsoft is offering free beta certification exams for Visual Studio 2010.
03/17/2010 11:08 AM EST

Microsoft lifts the hood on IE9
Microsoft is previewing IE9.
03/16/2010 01:10 PM EST