Lightening The Load-Testing Load

By Jennifer deJong

January 1, 2007 —

John Camp remembers it like it was yesterday.

Just as 34,000 students at Wayne State University were getting ready to begin a new semester, the Web application that lets them sign up for classes froze. “It locked up for a full two hours, and students resorted to registering by phone,” said Camp, CIO for the Detroit-based university. “[Administrators] were concerned that enrollment would fall.”

The culprit turned out to be the database that drove the application. “No one had exposed it to a full load of 1,000 simultaneous registrations,” he said. “We had tested 10,000 functional transactions, but we had not tested a true simulation of everything.”

Camp’s experience is not uncommon. Load testing—making sure an application will perform well under expected usage—is a tricky business. It requires intimate knowledge of everything, from how an application works and how users are likely to interact with it, to how the production environment is set up, according to analysts, development managers and tool makers. “You have to peer into all the tiers. There are so many moving bits and pieces, so many places where problems can occur,” said Mark Eshelby, product manager for Compuware, which sells testing and other application life-cycle tools. “If you don’t [load test thoroughly enough], you are going to have problems when you go live,” added Camp, who learned that lesson the hard way.

Part of the problem for many shops is that load testing remains an afterthought in the development process, even though the impact of poor performance on the business is well understood, said Voke analyst Theresa Lanowitz. Instead of simply running load tests just before applications are deployed, development teams should architect apps from the get-go with “what-if” performance scenarios in mind, she said. They should also allocate enough time to fix the flaws they find, she added. “You should [set aside] four weeks in June to test the Christmas Web site.”

A planned approach lets testers find out a number of things: “What happens when a Web shopper places an item in a shopping cart? What happens when you decrease the bandwidth? What happens when you have 10,000 [such transactions] coming down the line?” said Eshelby.

Another reason apps fall short under high-usage scenarios is that accurately predicting concurrent use is still an emerging discipline, said Forrester analyst Carey Schwaber. It’s hard to get that right, especially for a new application, she said. “What kinds of projections can you arrive at that are near reality?”

Many shops turn to tool makers that provide on-site load-testing services to help answer that question. “That lets you take advantage of their institutional knowledge,” said Lanowitz. Service providers conduct load tests every day, but most development teams do so only occasionally, making the skill set expensive to maintain in-house, she said.

Bill Piotrowski, executive director of technology and information services for Leon District Schools, in Florida, agreed. His team runs load-testing tools in-house from time to time. But to prepare for peak periods, he calls in expert help. “Not many places have the luxury of peeling off people and dedicating them full time to the [load-testing] task,” he said. “Developing the level of expertise that [the load-testing tool makers have] is not realistic.”

‘THE ULTIMATE DEADLINE’
What constitutes a “peak period” varies widely across different types of businesses, said Jonathan Rende, vice president of product marketing for testing tool maker Mercury Interactive, recently acquired by Hewlett-Packard. Much of it is seasonal. “For chocolate maker Hershey, the peak period is Halloween,” said Rende. For Web retailers, it’s Christmas, added Dan Koloski, director of strategy for testing tool maker Empirix. But other events impact usage, too. “A TV ad that airs during a major sporting event will also drive traffic to the advertiser’s Web site,” he said. Whatever the origin, preparing for peak periods is all about dealing with the “ultimate deadline,” he said.

Rende said Mercury has found a growing sophistication among line-of-business executives in planning for such deadlines. “It’s part of how they compete online.” A few years ago, there were problems, he said. “But now [the business] has gotten the religion.”

Forrester’s Schwaber said she doesn’t disagree that executives understand the impact poor application performance can have on the business. “But they aren’t always clear on the sacrifices involved.” They often ask developers to add more features to apps without understanding the impact that decision may have on performance, she said.

WHAT DO THEY FIND?
Load testers isolate all manner of bottlenecks, and they are not just problems that extra hardware can address, said Koloski. “Sometimes an ISP’s routers and switches aren’t adequately provisioned, which means [users can’t access] the Web application.” Another common problem occurs when dynamic content is refreshed at insufficient intervals, he said. “But more often than not, it’s not a given component that is poorly tuned,” said Koloski. “It’s that the working whole is less than the sum of the parts.”

The registration and other applications that Wayne State students, faculty and administrators rely on are running fine today, said Camp. But he has taken drastic measures to ensure problems like the database freeze-up don’t happen again. Wayne State is establishing an application testing laboratory, designed to be used not only by the university, but also by outside clients, in order to defray costs.

“No one is doing testing thoroughly enough,” said Camp. “But the old days are over, and you cannot skimp on preproduction testing,” he said. “There is so much dependency on electronic services today, and there is no longer any tolerance for disruptions.”

Share this link: http://www.sdtimes.com/link/29966

iTKO virtualizes load testing Lisa 4.5 has the ability to virtualize shared components that might have interdependencies and otherwise be off limits to QA test teams outside of specified hours.

SoftLogica Updates Load Testing Tool Beating up your Web applications is a fundamental part of the testing process. Now, SoftLogica hopes to get in on...

AccordSQA Adds Load Testing To SmarteSQA Suite AccordSQA, which makes a performance testing tool that automatically gleans information about objects’ characteristics on a page, this month will...

Add comment

Name*
Email*
Country

Comment
Preview

News on Monday
more>>
SharePoint Tech Report
more>>

Download Current Issue
ISSUE 3/15/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?

Google Code turns 5
Google Code Turns 5, and adds a Paxos Algorithm to make the system more stable and reliable.
03/17/2010 11:16 AM EST

Test your Visual Studio 2010 know-how
Microsoft is offering free beta certification exams for Visual Studio 2010.
03/17/2010 11:08 AM EST

Microsoft lifts the hood on IE9
Microsoft is previewing IE9.
03/16/2010 01:10 PM EST