I can’t deny it, I’m officially skeptical about the Agile movement. I’ve read a lot about the Agile philosophy over the past year, and the more I read, the less impressed I am. The principles at the heart of the Agile movement are self-evidently true, but they haven’t translated into a useful set of methods for software developers. As I wrote in a recent post: “This is no way to achieve reliable, repeatable results. It’s de-evolution in action, a return to the days of late-night hack attacks and reliance upon the heroic contributions of uncommonly talented superprogrammers.”

I admit that my criticism is disarmed somewhat by the first statement of values listed in the Agile Manifesto itself: “Individuals and interactions over processes and tools.” So Agile hasn’t translated into a heavyweight set of processes and tools? Duh. It was never intended to.

This morning I stumbled across a blog post titled “Agile is a Sham.” It turned out to be an invitation to an entire subculture of Agile skeptics. I thought for a moment I’d found some likeminded thinkers to spend time with. But I couldn’t have been more wrong.

I followed a trail of hyperlinks that led me through a rising tide of dissatisfaction with Agile. Not because Agile hasn’t translated into a useful set of tools and methods, but because Agile imposes too many tools and methods! This anti-Agile community is contemptuous of consultants and trainers who are peddling Agile as the new cure for everything that ails software development. They despise Scrum and Kanban as snake oil that managers buy in an effort to get superior results out of inferior programmers. The blogger summed up his Agile skepticism like this: “[T]he success of the coding part of a project is dependent on the caliber of the engineers doing that coding and not the process they follow.”

Maybe you recognize this line of thought. Our industry is vulnerable to a very unattractive elitism. There are lots of programmers who believe the best way to ship working software on time is to get rid of onerous development processes, sweep out the below-average programmers, and leave the competent programmers – the real engineers – alone to do their thing. Just get out of their way, and they’ll make the code work.

Members of this elitist clan argue that project management and formal development processes are just obstacles to their brilliance: “If you need to follow a prescribed process in order to be in any way an effective coder then you are mediocre at best and so is your work and your project.  If you are a good programmer, you know yourself how to communicate and create your ‘process’ as you go along.”

I find myself totally at odds with this community of Agile skeptics. Their cynicism about Agile consultants and trainers is probably well-founded, I admit. But their insistence that raw competence makes methods obsolete – and that reliance on methods is an admission of mediocrity – sets my teeth on edge. Their elitist views are demonstrably incorrect and patently offensive.

So yes, I’m an Agile skeptic. But I’m the other kind – the kind that would like to see Agile principles instantiated in more discipline, more tools, and more methods. That’s how you translate good ideas into working processes. Our field needs more discipline, not less.

Web recommendation: The open-source Go language has achieved a milestone: Version 1.0 is now available. I like what I read about this lightweight language. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He is hopeful that a certain video clip he saw today, featuring himself, never reaches the public eye.

The best conference presentation I ever attended was at the Software Development Conference in 1989. I was editor of Computer Language magazine, the sponsor of the conference, and I had major responsibility for signing up speakers and assigning topics for the keynotes and technical sessions.

Larry Constantine spoke at our conferences, and Grady Booch. Bjarne Stroustrup and P.J. Plauger and Adele Goldberg. Bill Gates spoke several times. Ed Yourdon was a regular. Our conference attracted the industry’s best-known thinkers.

The session I’m talking about wasn’t delivered by one of software development’s legendary figures, but by a technical manager from The Whitewater Group, a small company that published an object-oriented programming language called Actor. Jim McCarthy had proposed a seminar on project management and slipped deadlines that he called “Slipping without Falling.”

Jim’s talk was both extremely entertaining and extraordinarily insightful. Drawing on his experiences in project and process management, he delivered a series of memorable, colorful aphorisms, each of which expressed in shorthand a profound truth about the messy process of shipping software. Jim focused on techniques for building healthy teams, a black art that is still almost universally mismanaged.

One of Jim’s most memorable bits of advice was this: “Beware of the guy in a room.” Many projects, he explained, have at their core a challenging bit of code that is assigned to the development team’s acknowledged superstar. This programmer takes custody of the crucial module and retreats to his office. The team spends the next weeks tiptoeing past his door so they don’t interrupt his intense focus. No one asks for forecasts, bug-tracking data, or progress reports. The team knows that its success hinges on this single programmer and his brilliance. His code will be ready when it’s ready.

This dynamic is common and pathological. It is anathema to teamwork. The pattern reinforces harmful stereotypes about the relative value of programmers and superprogrammers. It reduces meeting deadlines to a game of chance. As McCarthy said, “the results are uniformly fatal to the professional development organization.”

McCarthy’s observation hit home. I had seen organizations held hostage by a guy in a room. Worse, I recognized that on a few occasions, I had been the guy in a room. I had helped erode healthy teamwork in companies where I had taken sole custody of creative challenges and everyone else just waited for me to deliver.

Jim’s insights into team leadership led him to Microsoft, where he led the team that developed Visual C++ – a team that is legendary for its productivity and innovations. He expanded his Software Development Conference talk into a book called Dynamics of Software Development, which I commend to you as essential reading. Jim now runs The McCarthy Show, which offers leadership training. If you visit the Web site, which you should, be sure to check out the Episodes page, which currently offers 115 podcasts on elements of McCarthy’s leadership principles.

Curiously, Jim’s warning about the danger of “a guy in a room” is not unique in our industry’s literature. In 1971, Gerald Weinberg wrote an extraordinary book called The Psychology of Computer Programming. Like McCarthy, Weinberg focused on effective teams. His book included a short section called “The Ten Commandments of Egoless Programming,” which listed rules for functional teams. Item number nine on the list was this: “Don't be ‘the guy in the room.’ Don't be the guy coding in the dark office emerging only to buy cola. The guy in the room is out of touch, out of sight, and out of control and has no place in an open, collaborative environment.”

I don’t know if Jim picked up the “guy in a room” concept from Weinberg or it occurred to him independently, and I don’t really care. It’s an important insight with wide applicability.

Does your team rely on the efforts of a guy in a room? Are you the guy?

Web recommendation: McCarthy’s still got it! You can catch a video of his Agile 2008 presentation, “11 Commitments for a Shared Vision,” by clicking here. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He had such a nice day today.

I ran across an interesting Web page today: Agile @ 10: Ten Authors of The Agile Manifesto Celebrate its Tenth Anniversary. The editors at The Pragmatic Bookshelf contacted the 17 signers of the original Agile Manifesto and asked them to contribute their thoughts about developments in the Agile world over the succeeding 10 years. Ten of the 17 signers contributed publishable responses, which are collected in the article. The contributors – Andy Hunt, Kent Beck, Ron Jeffries, Jon Kern, Ken Schwaber, James Grenning, Arie van Bennekum, Stephen J. Mellor, Ward Cunningham, and Dave Thomas – shared reflections on the extent to which their bold statement changed the development world.

The article led me to read lots more about Agile, including the Agile at 10 series of articles here at SD Times.

For the most part, these articles stick to a predictable formula. With varying degrees of humor and self-effacement, the signers express parental pride in the path the Agile philosophy has traversed from offbeat mission statement to mainstream acceptance.

But as I read more closely, I noticed something else. At one point or another, in one article or another, almost every one of the signers tacitly admitted that the Agile movement has not lived up to his hopes. This quote from Andy Hunt is typical:

Last summer, I had the good fortune to visit a “very advanced” Agile shop. These folks really did embrace agile methods with a discipline, completeness, and a zealous fervor that would be hard to match. In many ways, they could have been a poster child for Agile methods...But these weren’t productive developers freed from mindless process dogma. They were Agile slaves. The dogma they followed was ours, and they followed it well. And as with many organizations in a similar position, they saw some promising results. Continuous integration, refactoring, unit tests, pair programming—all these techniques yielded some benefit. But they weren’t thinking, they weren’t reacting, they weren’t being agile. When problems came up, they addressed them with all the grace and elegance of a deer caught in the terrifying blaze of alien headlights. They knew how to do Agile; they didn’t know how to be agile.

Ron Jeffries told The Pragmatic Bookshelf, “I had imagined more. I had hoped that many people would adopt these ideas, and I had imagined a significant step upward in project success among those who did adopt them. I had imagined the industry really moving up a notch. What happened is that many have adopted the ideas, at least in name, but that few of them have attained anything like the benefit that is possible.”

In the same article, James Grenning said, “Many developers are still unaware of Agile, or only know the misconceptions. The idea that more upfront work is needed is deeply ingrained in the software development mindset. Code bases are a mess, dragging teams and products to a standstill.” He concluded, “Now we have organizations that have a few Scrum Masters and proclaim themselves Agile, but that continue to spend months in analysis and design, and similar amounts of effort in test and fix. They have stories and iterations, but ignore relative effort estimates and velocity. Code is deteriorating. Tests are not written. And they wonder why Agile is not working for them.”

I find Hunt's wistful observation insightful and moving. At its heart, the Agile Manifesto was a declaration of independence from the shackles of traditional development methods. The signers advocated not new processes, but new values and a new attitude.

The sad irony is that instead of overthrowing slavish commitment to orthodoxy, Agile has spawned a new orthodoxy – complete with certified tools, recommended texts, and a thriving culture of consultants to descend upon your organization and tell you how to do Agile right.

More and more shops now think of themselves as Agile. They're investing in Agile training and attempting to adopt Agile methods such as Scrum and Kanban. But they've missed the point. Instead of becoming enlightened freethinkers, they've drunk the Kool-Aid and joined the Cult of Agile. Dogma and orthodoxy are alive and well. At the cutting edge of our profession are the people Andy Hunt calls “Agile slaves.”

As Dave Thomas told The Pragmatic Bookshelf, “There’s this growing tendency to treat the word 'Agile' as a noun. People say, 'we’re doing Agile.' But friends, Agile is not a noun. It’s an adjective, meaning to be able to move quickly and easily.”

Is your organization agile? Or is it just “doing Agile”?

Web recommendation: The Supreme Court of the United States has agreed to hear a case in which police tracked a suspect's location by placing a GPS device on his car and querying its location every 10 seconds for 28 consecutive days – without a warrant. The question, of course, is whether the government should have the power to use technology in this way. The stakes are high and the right answer should be obvious to all of us. (Isn't it?) The case is summarized well in this GigaOM article. J.D. say check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He likes raisins and walnuts in his oatmeal cookies.

You probably can't tell, but I spend four or five hours of research for every hour I spend writing these posts of mine. I consider the research overhead – as a longtime computer-industry journalist, I know that I need to stay on top of multiple trends and technologies so I can comment intelligently on new developments. Heck, I have to stay informed just so I'll recognize a comment-worthy development when it happens.

I do this research the way you do, I suppose. I make massive use of my browser's “Open in new tab” feature. When my computer starts wheezing, I know it's out of RAM and swapping pages to disk, so I stop and read a few pages, closing the 95 percent of the tabs that don't merit follow-up. Then I return to new-tab mode.

This probably isn't the optimal way to do research. If my deadlines were more pressing, I'd be more scientific about it. But I enjoy the odd side-trips my research method affords me. As the marvelous John Crowley points out in an old book I love dearly, sometimes the snake's hands in a story are the best part.

But I am digressing. What I want to write about is hiring.

Today's Web-meandering led me to an article called “How do startups hire the right people?” Before the page had finished loading, I had mentally composed a rebuttal to serve as today's SDTimes blog post.

It was the phrase “the right people” that got me riled up. I have a visceral reaction to the idea that we can divide the world's people into desirable and undesirable. I abhor the widespread belief that native talent accounts for the (very real) productivity differences among software developers.

My experience is that the right environment can coax superprogrammer results out of developers who would do average work in an average environment. I passionately believe that we are much better off improving our workplaces – the physical plan, the philosophy, the training, the systems – than searching for coding superstars. If we build supportive workplaces, organizations that encourage flow, then we don't have to ask the HR department to do magic. We can create programming stars instead of hoping to stumble across them in a stack of CVs.

Well. It appears I have stumbled over a soapbox. Occupational hazard, I suppose.

I'll say much more about this in a future post.

As for the article that inspired this little rant, it turns out to be innocuous. It wasn't the elitist diatribe I was expecting. It's an entirely defensible article that makes a good point. I'm not linking to it because it's way off-topic. Or, more precisely, I am way off-topic. Whatever.

Web recommendation: Dr. Anna Akbari's presentation at a recent TED conference hit me very close to home. She examines, in a studious way, the relationship between the technologies that are consuming much more of our waking hours and the effects on our quality of life. I love it that she moves past the usual hand-waving and provides some real data and on-target recommendations. Good stuff. J.D. say check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He hopes his last words are the same as those of Steve Jobs, as reported by his sister: “Oh wow. Oh wow. Oh wow.”

Writing code is hard. Luckily, the world is full of resources to help us with the task. Books, conferences, seminars, university courses, magazine articles, blogs, Web sites – all promise to teach us how to write programs. But the luxury of writing programs represents, in my estimation, the smallest part of the software developer's career. Far more commonly, we inherit responsibility for someone else's program code. And on this topic, the development experts are lamentably unhelpful.

It's a shame, because this job – maintaining, correcting, documenting, updating, completing, and extending existing code – is much more difficult than writing new programs. It is, after all, harder to read code than to write it.

My first programming job centered around an inherited codebase. I was first called upon to write little patches for the bugs that halted the existing application now and then. Eventually I began adding features. After some months of this, I had a vision of what the app could be if it were fundamentally restructured in a new release. I sold the vision to my employers and got permission to rev the app. It was only after that rewrite was complete that I won permission to create a completely new app.

I imagine this sounds familiar. This is what a career in software development looks like.

It's all wrong, of course. Maintaining and extending a body of existing code is much more difficult than writing new code. We should assign our most experienced and efficient programmers to maintenance. But that would lead to a revolution, wouldn't it? Because we all want to exercise more creativity on the job, and there are more opportunities to be creative in writing new apps.

Search the Web for advice on maintaining an inherited code base and you'll find two sorts of information. You'll get articles that look promising, but turn out to be advice on how to write new code that is more maintainable and extensible – useful, important advice, but not relevant to the task at hand. Or you'll get advice on refactoring.

Don't get me wrong, I'm all for refactoring. But how many of us have the luxury of spending months making small improvements to the underlying design of inherited codebases, without making a single change in the software's functionality? Where does one find such an unlikely confluence of management enlightenment and budgetary largesse?

Years ago, CASE tools promised to read source code and create navigable graphic representations of the code's workings. You could look at a high-level diagram to explore the application's main functionality and zoom in to see the individual modules, classes, functions, and lines of code that instantiated that functionality. I thought those tools had real promise.

Alas, the tools have not kept pace with the reality of modern apps. They worked well enough for applications that were built of a single source-code file. But today's apps, as you well know, are built in modular fashion of libraries, subsystems, Web resources, and code files written in multiple languages. I don't know of a visual tool that can cope with such complexity. (I would be delighted to learn of one.)

So here we are, stuck. We arrive at the workplace full of great knowledge about how to write new programs and find ourselves challenged with a harder task that no one warned us about.

It's just one of the reasons I think our so-called profession is a mess. I'll share more in future posts.

Web recommendation: The good folks at whitehouse.gov – yes, the President's Web site – have added a feature called We the People. It's a page where any user can launch a petition, and where visitors can virtually sign petitions if they find themselves moved to. Officials promise that petitions receiving 5,000 or more signatures will get replies, and perhaps even move the administration to action. I learned about We the People because I got excited e-mail from correspondents who wanted me to sign a petition calling for a ban on software patents in the U.S. I'm not going to post a link to the petition because the proposed ban is not sufficiently detailed, and it is wrongheaded to boot. Software patents do more good than harm. I do appreciate the Obama administration's little experiment at populism, however. We the People may not be the most efficient or well-implemented scheme of getting people involved in politics, but I think its creators have their heart in the right place. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He recently relocated to a small town outside Belgrade – stop by if your travels take you through Serbia.

Over the past year, I have written about how some developers are gaming IT certifications by obtaining "braindumps" through BBSs, torrents, and Web sites. Braindumps are files that have verbatim answers from certification exam tests. Now, the cheaters have sunk to a new low: They may not even be taking the tests themselves.

It's like a bad episode of 'Saved by the Bell' - a company is now offering imposter test-takers for hire. Apparently even memorizing answers is too hard for some people. The Web site accepts payments via PayPal, and promises to refund all fees if a test isn't passed.

I'm told that Microsoft certification centers (at least in the US) ask for two forms of ID. So a company that offers imposters clearly must be skirting this issue somehow, raising questions about the legality of the operation. This is more sad news for people who work hard to get their certifications, and employers that value training.