Which Functional Testers Do the Best Job?
Stories Columns Opinions Resources
Preflight builds spread wings for smoother projects
Developers are increasingly turning to preflight builds, allowing them to experiment with ...
|
Coverity creates program to enforce code adherence
The Architecture Analyzer uses mapping technology from the company's Software DNA static a...
|
QCon 2008 features domain-driven development
This year's QCon invites speakers like Eric Evans and Dan North to talk about domain-drive...
|
.NET similarities prove golden for Silverlight
Microsoft has focused on making Silverlight 2 symmetric with the .NET platform, and that h...
|
SOA Watch: New economic realities
In the current economic downturn, agile programming and SOA are attractive options that bu...
|
Integration Watch: A new twist on threads
The key to raising the efficiency of multiprocessors is to shrink the overall workload by ...
|
Integration Watch: The Return of NetRexx?
Java scripting languages are seeing a surge in popularity, with NetRexx looking particular...
|
Windows & .NET Watch: Transaction crowd gets a boost
With multicore chips becoming the standard for processors, the need for a flexible, usable...
|
From the Editors: Election should shake up JCP
Rod Johnson has the right ideas for opening up the Java Community Process, and he may be a...
|
Letters to the Editor: Sun gives REST, SOAP choice
A reader takes issue with a headline on our story about Sun working with REST along with S...
|
Guest View: Be smart and lazy
The optimal solution for problems is the simplest one, so always aim to streamline your ap...
|
Zeichick's Take: From EXEC to EXEC 2 to REXX to NetRexx
Andrew Binstock's column last week, "The Return of NetRexx," brought back some fond memori...
|
Advanced Corda CenterView™ Data Visualization for the BusinessObjects™ Intelligence Platform
Corda Technologies presents a white paper on pervasive BI. The BusinessObjects business in...
|
From Mobile to SOA: A Guide for Optimized Application Deployment
Customer need has driven the emergence of multiple computing tiers. Today’s application d...
|
e-Kit: Web Application Security
Is your network secure? What about your web applications.
If IT security is your top p...
|
Practical tips for saving money on code maintenance
If software design is expensive, well, code maintenance is even more so. When you look...
|
By Edward J. Correia
July 15, 2008 —
Since its acquisition of Mercury Interactive in 2006, Hewlett-Packard continues to lead the pack in terms of tools for software testers, according to a report from Forrester Research released last week. But is QuickTest Pro the best functional testing tool available? Is it worth the price? How does it compare with IBM's Rational Functional Tester and others from Borland, Compuware, Empirix and Seapine?
The report, titled “The Forrester Wave: Functional Testing Solutions, Q3, 2008,” looked at those leading suppliers of solutions that include manual and automated testing as well as test management capabilities. Notable exclusions were Microsoft's Visual Studio Team System for Testers, which offers little or nothing in those areas, and MKS Integrity, which doesn't include automation. Companies with less than US$10 million in annual revenue also were excluded.
The evaluation was based on 96 criteria in the areas of the companies' current offerings, their strategies and presence in the marketplace.
Forrester reports that HP is still the default choice for large organizations, as was Mercury prior to its acquisition. "HP's high scores in our evaluation confirm that this is a rational strategy," said the report, who co-authored the study. But what weaknesses in HP's products might cause companies to look elsewhere? "For years, the main answer has been price, but that reason is now joined by concerns over the quality of support—a major bugbear that HP is scrambling to address," she said.
Other industry powerhouses come from Borland and IBM, whose solutions are popular with "technical testers" who prefer to work with scripting languages rather than a GUI, she said. "These vendors have attempted to broaden their appeal by beefing up on their support for manual testing. But Compuware, like HP, appeals to testers who have less of a technical bent," the report said. Many IT organizations adopt those tools to work alongside the same vendor's development tools, the study found.
Tools from Empirix and Seapine reigned supreme in terms of price but largely fell short on functionality, Forrester said. Empirix in particular was lacking; it tests only Web applications, and "just Internet Explorer at that." Seapine's TestTrack Pro, however, "stacks up fairly well against the competition."
But HP's QuickTest Pro and Quality Center were not without their downsides. While both scored well for manual and automated testing, as well as test management, they were faulted for limitations in operating system support (both are Windows-only) and their use of VBScript for scripting.
The report characterized IBM's solutions—ClearQuest, Rational Functional Tester and Rational Manual Tester—as a bit of a sleeper. "Rational Manual Tester is a great tool that hasn't attracted nearly the attention it deserves," she said. "And IBM's test automation tool is a good pick for users with programming experience; it supports scripting in either Java to VB.NET."
But the suite falls short on test management. Perhaps IBM will get it right when it completes the porting of ClearQuest to Jazz, the second platform migration the tool has undergone in three years. "The Jazz platform is a great base on which to build, but all this churn makes it difficult for IBM to develop functionality specific to test management." Still, IBM in recent years has integrated its function testing tools toward the ultimate goal of enabling keyword-driven testing.
Through its acquisition of Segue Software and subsequent investments in its Silk line of test products, Borland now has "an extremely usable new manual testing client" and test management tool in SilkCentral, the report said. And its SilkTest automation tool now has a Java scripting engine that integrates with Eclipse. It also maintains ties to its manual testing roots through the retention of 4Test scripting engine.
Compuware shined in the area of test management, and the company's solution comprises a "wider range of products than other evaluated solutions," according to the findings. This, the report said, was in part because Compuware is incorporating technology from its Changepoint project portfolio management product into its test management solution. "These excellent test management capabilities are now available to all customers, whereas they were previously reserved only for customers who purchased Compuware's CARS services offering," she said.
The company's test automation also has been enhanced, giving non-technical users the ability to record and manipulate test cases. Compuware's function testing solution lost ground for its weak integration with related tools and lackluster reporting.
Empirix and Seapine offer "less capable alternatives," than others in the study, according to Forrester. In any race, someone always comes in last. In this case, it was Empirix's e-Manager Enterprise and e-Tester function testers, which turned in the lowest scores in every major category. Empirix test automation tools lost points for generating proprietary scripts inaccessible to end-users and testing only IE apps. However, the company later this year plans to introduce a so-called Open Scripting Engine, an extensible scripting platform.
What's more, the company's test management tool "has no workflow capabilities and does not integrate with any testing or life-cycle tools besides Empirix e-Load," Forrester said. Empirix announced in March that it agreed to be acquired by Oracle.
Seapine, on the other hand, is a "small but rapidly growing" ALM tools vendor that undercuts competitors on price by about half, "but doesn't match them on features," the study reports.
QA Wizard, its test automation tool, targets "a narrow range of technologies ... and has no specialized support for rich Internet apps, Web services or packages applications." Scripting is offered through a GUI and VBScript. TestTrack Pro TCM, for test management, does not integrate with any third-party apps.
Next week, I'll look at some open-source alternatives.
Share this link: http://www.sdtimes.com/link/32536