Softwares Go-To Guy

Sophistication, complexity of applications require ‘generalist’ developers who do many things

By Geoff Koch

December 1, 2007 —

There’s a subtle but persistent undercurrent in technology today about the waxing of smart-but-flexible generalists and the waning of hyper-focused specialists. Sports fans out there might best understand it by way of a baseball-inspired metaphor, if you’ll forgive a nod to the boys of summer here in the early days of winter. Software’s designated hitters who do little more than, say, maintain the mail server, are very much out. Switch-hitting speedsters—who can administer databases, build and maintain Web sites, write client-side apps and move easily up and down the LAMP stack—are decidedly in.

Other readers might consider this relatively recent and certainly more coherent rant on the topic from Philip Nelson, a Wisconsin-based developer who maintains the blog “An Active Coder.”

“Most of us have had the misfortune to work on teams that were structured in specialist silos,” wrote Nelson in an April 30, 2006, posting. The antidote, he said, is for managers to make sure coding teams remain relatively diverse and for individual programmers to cultivate at least basic familiarity with technologies—including databases—well beyond their comfort zones.

“The dreaded DBA [database administrator] pops up in my mind right away,” Nelson complained. “Having to deliver projects where you need to coordinate a DBA, install team, Web designer, systems analyst and security experts is painful and inefficient.”

Luckily for developers working in either Microsoft or Java environments, database programming and administration is becoming slightly less opaque. Details vary, but common trends in each of the big three integrated development environments—Visual Studio, Eclipse and the surprisingly resurgent NetBeans—include making it simpler to deal with different data types, to establish connections to a variety of SQL databases, and to abstract the complexity of arcane SQL syntax and instead work visually with tables and rows, or even with higher-level entities.

What’s behind these changes?
“The environment in which we are all developing applications has clearly gotten more sophisticated,” said David Straus, senior vice president at Corticon Technology in Redwood City, Calif., citing increased business dependence on IT, more distributed usage and more data sources as examples. Corticon offers a business rules modeling environment that exists as an integrated tool within a variety of IDEs, including Eclipse.

“In the old days, applications were often standalone,” Straus continued. “Today we are trying to develop applications into component services which are orchestrated by some [software] layer. We want these components to be reusable and well orchestrated.”

Microsoft and the more dispersed communities that maintain the open source development tool sets aren’t just responding to a roiling business and technology environment. They’re also trading punches with each other.

“Both Eclipse and Visual Studio are competing feature by feature, tool by tool, including the ‘standard’ IDE features, language features, modeling tools, SOA tools, built-in testing tools, reporting tools, database tools, etc.” said Jeff Bocarsly, a division manager at RTTS, a software consulting and quality assurance company in New York City. “Each of these organizations, Eclipse and Microsoft, appears ready to match the other with the most elaborate development environments that computing has seen.”

MICROSOFT MOMENTUM
Any discussion of development environments must start with Microsoft, despite observations that the Redmond-based giant is in some trouble. “Microsoft is beset with competition from all sides, unlike any it has seen in decades,” wrote New York Times senior writer John Markoff in a July 27 article on Bill Gates’ slow transition out of the company he founded. Even when Microsoft bests Google, the company that is undeniably its current chief rival, the press accounts aren’t exactly triumphant. “Microsoft clearly thought that it had no choice,” reported The Economist after the Oct. 24 announcement that Microsoft would pay US$240 million for a 1.6 percent share of Facebook. “As the number of potential partners dwindled, Microsoft got desperate.”

The public relations front is far removed from one inviolable fact: Every day, teeming throngs of programmers still develop in and for Microsoft environments. Minneapolis-based Gartner analyst Mark Driver said that Microsoft’s current worldwide developer community numbers more than 7 million, which is roughly the combined population of Los Angeles and Chicago. Within this community, he added, the penetration of Visual Studio is well over 90 percent.

Visual Studio 2008, which Microsoft promised to release by the end of November along with .NET Framework 3.5, includes a host of database-specific features. One is the introduction of Language Integrated Query (LINQ), intended to help developers better deal with data of all types by way of extensions to the C# and Visual Basic programming languages and the .NET Framework. “These extensions provide integrated querying for objects, databases and XML data,” said Francois Ajenstat, director of product management for SQL Server. “Using LINQ, developers will be able to write queries natively in C# or Visual Basic without having to use specialized languages, such as SQL and XPath.”

Another promised feature will surface in Visual Studio Team Edition for Database Professionals. Microsoft has been hawking the tool set as a way to bring database professionals into the application life cycle since the middle of 2006. Not surprisingly, the product is optimized to work with Microsoft SQL Server. However, the release will include at least a cautious nod to openness.

“We are opening up the API to allow partners to build connections that talk to other databases,” said Matt Nunn, group product manager for Visual Studio Team System. “We continue to add new features—refactoring, data generation, database testing, database design tools—that make it easier to manage change to the database and to test and integrate the DB into the rest of the application life cycle.”

OPEN SOURCE OPTIONS
“I think that the biggest advantage of Eclipse over the other IDEs is that it is very modular and configurable, [which is] not so much [the case] in Visual Studio,” wrote Peter MacIntyre in a sprawling hour-long interview conducted via Google Talk. MacIntyre is an 18-year tech industry veteran who owns consulting firm Paladin Business Solutions IT in Charlottetown, on Canada’s Prince Edward Island.

One sure sign of MacIntyre’s status as a true believer in open source is his choice of reading material—at one point bringing up “The Cathedral and the Bazaar” by Eric S. Raymond, an undeniable centerpiece of the open source literary canon. “That book was my ‘road to Damascus’ for switching to the open source concept.”

MacIntyre can back up his somewhat offbeat zeal, however, particularly when it comes to his expertise in Zend, an ongoing bottom-up rewrite of the PHP scripting language. He is co-author of “Zend Eclipse PHP,” due out in early 2008, and claims to be the first person in Atlantic Canada to have passed the PHP 4.x Certification Exam.

“Eclipse’s database interface, just speaking about the Zend Studio for Eclipse product, is quite functional,” he said. “I remember having quite a time wrestling with Visual Basic and ASP trying to get them to even talk with [Microsoft’s] own database offerings.”

With the exception of the minor user interface-related annoyance of having to flip back and forth between the database and code perspectives, MacIntyre declares Eclipse to be mostly complete when it comes to working with databases, though he added an important qualifier: Eclipse “is not meant to be a fully operational database development tool; it’s more meant to be a ‘window’ into that house [of data] so that you can keep on with the development, and maybe reach in through the window from time to time.”

John Graham, a Sybase staff software engineer with an acronym-laden second job, provided a more close-up look at the evolution of Eclipse vis-?-vis database development. Graham also serves as Project Management Committee (PMC) chair of the Eclipse Data Tool Platform (DTP) project, a top-level Eclipse project led by Sybase, in conjunction with Actuate, IBM and Japan’s NEC Soft.

The ongoing work of the DTP project represents the most vibrant interface between the Eclipse tool set and databases. Three of the project’s four core development activities—honing domain models related to a variety of data sources, addressing connectivity issues and boosting SQL development tooling—are squarely focused on database development while also being vendor-neutral and extensible. Only within the last core DTP activity, which carries the generic “enablement” label, do the four sponsoring companies work to customize the core framework for their particular commercial products.

The current milestone release, DTP 1.5, came out in June and contained code that Graham believes goes a long way to building out a solid foundation that spans most of the mainstream SQL databases. “We’re getting many fewer comments like, ‘Hey, I downloaded DTP, and it doesn’t work with the database I use,’” said Graham. At present, DTP uses a hodgepodge of “glue code” and wizards to make it easier for developers to access various database unit testing frameworks. DTP 1.5 also included a new driver for accessing XML-based data sources and new support for the Sybase SQL Anywhere mobile database.

The next release, scheduled for June 2008, will include a more highly visible addition—a SQL query builder that will provide a visual metaphor for building SQL statements. Developers will be able to construct queries by drawing lines between various rows and columns of displayed tables. No doubt the feature is an example of innovation in the Eclipse community. But, coming many months after a similar visual query builder was added to NetBeans, it also serves as a reminder that Eclipse is by no means the only open source IDE available to Java developers.

NETBEANS: THE COMEBACK KID
Gartner’s Driver is among those who, until recently, considered NetBeans to be on life support. But the open source IDE supported by Sun Microsystems has rebounded of late, increasing its market share among Java developers to at least 20 percent from a position that barely registered on analysts’ radar screens, Driver said. While Eclipse has much broader OEM support and a richer ecosystem of third-party tools, NetBeans wins points, Driver continued, for its out-of-the box functionality for end-user developers and for aggressively going after dynamic scripting very early on. “It’s one of the nicest Ruby IDEs available,” Driver said.

Sun’s Gregg Sporar, a NetBeans technology evangelist, was nice enough to run though the various features of version 6.0, which is currently in beta and set to be released Dec. 3. “We do understand that software developers need access to relational databases,” Sporar said, clicking to expand a database tab and revealing preloaded drivers for most of the mainstream open source SQL databases: MySQL, PostgreSQL, Apache Derby, also known as Java DB, and more. Connecting to other relational databases, including all the big commercial offerings, including DB2, IBM, Microsoft SQL Server and Oracle, is as easy as downloading the relevant Java device drivers and then adding the drivers to the IDE via one simple right-click command, he added.

In a 25-minute online demonstration, Sporar walked through several features—one-click ability to connect to essentially any database with a JDBC API; an easy means to actually start any database server, such as Derby, whose runtime code is included in the IDE; and a straightforward means of seeing foreign key information within datasets to better understand relationships between various categories of data. But undeniably the crown jewel of NetBeans 6.0 is a graphical means of query building that not only completely hides the complexity of SQL syntax, but also appears far simpler than the checklist-driven method of constructing a SQL statement that is so prevalent today.

Not to be outdone, Microsoft plans to similarly mask SQL complexity for its developers when the ADO.NET Entity Framework is released sometime next year. “Rather than programming against tables and columns in a database, developers [will be able to] use high-level entities such as ‘Customer’ or ‘Order’ to represent the underlying data,” said Ajenstat. “The ADO.NET Entity Framework enables developers to program against relational data in terms of such entities.”

In the end, however, despite all the enhanced and drag-and-drop ability to work with databases, it’s unlikely that job categories such as “database administrator” and “Web developer” will ever completely merge into each other. Desktop publishing and cheap laser printers didn’t supplant professional graphic designers and printers, nor has Dreamweaver or its new open source content management cousins, Drupal and Joomla, destroyed the market for Web designers. And howl as they might, freeware-armed bloggers and vloggers haven’t come close to replacing the mainstream or trade press.

While there’s at least modest sniping between the Java and Microsoft IDE camps, most interviewed were in broad agreement that the era of utility infielders that can play any position on a development team will likely never arrive. The logic associated with adding a software service that automatically approves a new account, for example, “needs to be built and maintained by people who are business-savvy, not IT-savvy,” said Straus. “These people should not have to deal with the complexity of databases.”

“While broader database and middle-tier skills are a big plus for a developer, in addition to expertise in the presentation layer, the DBA as a specialty is still a necessary ingredient for architecturally complex projects,” agreed Bocarsly, who went to his own baseball metaphor: “It might be good to have players who can cover either shortstop or left field on the team, but your closer is still going to be your closer.”

Share this link: http://www.sdtimes.com/link/31405

Lattix Extends Architecture Analysis Software’s Abilities The new version of Lattix LDM—which can analyze complex software systems to determine interdependencies—focuses on handling more complex applications and...

Project Estimation Does Not Have to Be Software’s ‘Black Art’ “It was a heck of a topic to write about,” said author Steve McConnell of his latest book: “Project Estimation:...

Testing Software’s Goal: Make Bugs Go Splat Software is getting more complex. Service-oriented applications pull together modules that weren’t necessarily meant to be joined. Multithreaded environments can...

Add comment

Name*
Email*
Country

Comment
Preview

News on Monday
more>>
SharePoint Tech Report
more>>

Download Current Issue
ISSUE 3/15/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?

Google Code turns 5
Google Code Turns 5, and adds a Paxos Algorithm to make the system more stable and reliable.
03/17/2010 11:16 AM EST

Test your Visual Studio 2010 know-how
Microsoft is offering free beta certification exams for Visual Studio 2010.
03/17/2010 11:08 AM EST

Microsoft lifts the hood on IE9
Microsoft is previewing IE9.
03/16/2010 01:10 PM EST