Zero tolerance for bugs
Stories Columns Opinions Resources
Preflight builds spread wings for smoother projects
Developers are increasingly turning to preflight builds, allowing them to experiment with ...
|
Coverity creates program to enforce code adherence
The Architecture Analyzer uses mapping technology from the company's Software DNA static a...
|
QCon 2008 features domain-driven development
This year's QCon invites speakers like Eric Evans and Dan North to talk about domain-drive...
|
.NET similarities prove golden for Silverlight
Microsoft has focused on making Silverlight 2 symmetric with the .NET platform, and that h...
|
SOA Watch: New economic realities
In the current economic downturn, agile programming and SOA are attractive options that bu...
|
Integration Watch: A new twist on threads
The key to raising the efficiency of multiprocessors is to shrink the overall workload by ...
|
Integration Watch: The Return of NetRexx?
Java scripting languages are seeing a surge in popularity, with NetRexx looking particular...
|
Windows & .NET Watch: Transaction crowd gets a boost
With multicore chips becoming the standard for processors, the need for a flexible, usable...
|
From the Editors: Election should shake up JCP
Rod Johnson has the right ideas for opening up the Java Community Process, and he may be a...
|
Letters to the Editor: Sun gives REST, SOAP choice
A reader takes issue with a headline on our story about Sun working with REST along with S...
|
Guest View: Be smart and lazy
The optimal solution for problems is the simplest one, so always aim to streamline your ap...
|
Zeichick's Take: From EXEC to EXEC 2 to REXX to NetRexx
Andrew Binstock's column last week, "The Return of NetRexx," brought back some fond memori...
|
Advanced Corda CenterView™ Data Visualization for the BusinessObjects™ Intelligence Platform
Corda Technologies presents a white paper on pervasive BI. The BusinessObjects business in...
|
From Mobile to SOA: A Guide for Optimized Application Deployment
Customer need has driven the emergence of multiple computing tiers. Today’s application d...
|
e-Kit: Web Application Security
Is your network secure? What about your web applications.
If IT security is your top p...
|
Practical tips for saving money on code maintenance
If software design is expensive, well, code maintenance is even more so. When you look...
|
By Deb Radcliff
August 1, 2008 —
If you’re in the business of developing high-assurance programs, any bug is a bad bug, says Phyllis Schneck, vice president of research integration at Secure Computing Corp.
“It’s no longer like the old days, when I was reviewing FAA code and we just had to be sure the application worked,” notes Schneck, former chairman of the national board of FBI InfraGard, a public-private infrastructure protection partnership. “Nowadays, everyone is equally worried about security defects, particularly as software code runs more and more of our infrastructure.”
As risks have increased, traditional bug-finding tools have risen to meet new security challenges. This is particularly true with static analysis testers, which can scream through a million lines of code in 90 minutes and tell developers not only what is wrong with the code, but also explain the security risk and the fix, as well as track repairs.
“Once a developer starts using static analysis tools, that developer won’t be able to imagine life without them,” says Theresa Lanowitz, founder and CEO of analyst firm Voke. “As a result of using the tools, coding gets better over time, and they save money by making the repairs before the code goes into release, instead of finding them after.”
Despite such enthusiasm, today’s static analysis tools aren’t perfect. They rely on a database of known “signatures” of security-related bugs, so they are prone to the same false positive vs. false negative trade-off you would see in other signature-based technologies like intrusion detection. Nor are they a replacement for dynamic testing, also known as vulnerability analysis of an active application, and so many vendor offerings consist of both static and dynamic testing options.
The other issue is that tools are all over the map—command line freeware, system-based tools (such as Microsoft’s Static Driver Verifier, which looks at drivers), and commercial tools that cover specific languages and platforms to varying degrees. In addition, there are services like Cigital (with 100 employees using a variety of analysis tools) and Veracode, which is a service that scans binary output rather than source code. Most other static analysis tools can scan the latter.
Meanwhile, the tools are undergoing an identity crisis of their own. For example, Adam Shapiro, Microsoft’s senior program manager for static analysis, is emphatic that Microsoft’s offerings are not security tools; they are “quality” tools. Meanwhile, there is a whole movement of scientists and professionals who are just as emphatic that static analysis testing is as much about security as it is about quality.
“MITRE has a list of 550 things that can go wrong with code. At least half of those bugs are security-related,” said Paul Black, who’s overseeing the Software Assurance Metrics and Tool Evaluation project (SAMATE) under NIST ITL (Information Technology Laboratory). “Static analysis can catch these security bugs through libraries of known vulnerabilities and by making pre-defined assumptions about how these bugs will interact from a security perspective.”
At the source
To prove that using static analysis tools can improve overall application security, one need only look at benchmark results produced on SAMBA, the open-source Unix-Windows interoperability effort.
Two years ago, Coverity, a leading commercial provider of static analysis tools for C, C++ and Java, offered up a free scanning Web site for SAMBA’s developers, which led to the immediate discovery and repair of 216 defects in the first nine days.
Since then, developers have fixed 8,500 security defects, and overall security of codebase has improved from 0.3 bug per thousand lines of code to 0.025 per thousand. That figure seems inconsequential until you realize that it implies there are still 18 or 19 holes in the project’s 750,000 lines of code at any moment. It’s even more consequential when considering much larger applications with lines in the tens of millions.
Jeremy Allison, one of two SAMBA developers assigned to fix the bugs found by Coverity, says that Coverity scans the team’s applications every other day, because contributing developers are constantly changing the codebase. Coverity catches bugs with every code change, he adds, albeit there are fewer as developers change their ways.
“The report output drills down and gives us a category of a bug like a null pointer indirection. It tells you, ‘Here you made this call,’ and, ‘Here you indirect it,’ ” he said. “After a while of using it, you get to know the kind of things it looks for, and your code gets better.”
Learning how to avoid common mistakes is a huge benefit that static analysis tools can provide, says Gary McGraw, CTO of Cigital and author of two books on secure software development by Addison-Wesley.
That learning, he said, comes in three forms: “Education in the use of the tools and rules themselves, education in the security risk of the bug found, and education in best practices on how to write secure code. That’s a huge difference from most security tools, which just point out you have a problem over here somewhere,” McGraw notes.
According to Tarek Nabhan, products division manager for Egypt-based software services firm ITWorx, the education chain works like this: In one place, developers see the bug, the line of code, its interdependencies with calls and functions, the fix and what they’ve been learning about coding best practices.
“As a result, my people know more about good coding techniques, and the quality of code is improving,” says Nabhan, whose organization has 100 of its 700 developers trained on Ounce Labs’ Ounce5 static analyzer, with plans to train the rest on it over time.
At binary
Unlike most static analysis tools, which test source code at compile time, Veracode SDLC looks at code after it’s been compiled, by doing binary scans. The same cannot be done in C and C++ environments, many people assert, because machine code is too abstract from which to gather sufficient security data.
Chris Wysopal, founding CTO at Veracode, says that one can enhance binary output to test it as or more thoroughly than source. And even if it is tested as source, he adds, unanticipated interactions with compilers can cause vulnerabilities that don’t show up until after it’s compiled.
As an example, he points to a case in which the password wasn't getting zeroed out because of a compiler bug, something that could be discovered only by inspecting the binary. The bug exposed a secret to unlocking full privileges to a Windows application and, ultimately, the underlying system.
“Source and binary analysis are much more similar than they are different,” he says. “We’re trying to solve the same problems, tackling the same set of bugs. Both are statically analyzing the application code, one before and the other after it’s compiled.”
Market madness
Static binary, static source, dynamic testing: It all goes to show there is no single correct way to statically analyze a complex codebase. As vendors react to specific problems, they cover some bases, but none cover all the things that could go wrong and lead to compromise.
Take, for example, a June SAMATE meeting of NIST computer scientists, where teams ran eight tools and the Veracode service against three Java and three C programs consisting of 50,000 to 100,000 lines of code apiece (for full list and tests, visit samate.nist.gov/index.php/SATE#Participating_teams). There was little to no overlap in the tools coverage, according to an internal memo written by Black. Each tool found different bugs that the others weren’t catching.
Results also showed that reporting mechanisms would sometimes produce multiple repeat reports for a single bug (for example, not taking into consideration things like the combination of source and sink). They did, however, pass with flying colors in the education aspect of telling developers the security consequences of the bugs they find.
Like repeats, false positives are problems for static analyzers because they simply can’t be tuned to anticipate every complex association and innuendo in the code, says Brian Chess, co-author of “Secure Programming with Static Analysis” and founding chief scientist at Fortify. For example, Coverity—which sets the standard for the lowest rate of false positives, according to analysts—averages 13%.
False negatives are also problematic, he adds, pointing to a bug he examined in late June that compared the variable read from a Web page to a program string that converted input into all capital letters. That may not seem like a problem, he explains, but if that application runs on a server in Turkey, it would be vulnerable to an input injection attack because in Turkish, lowercase letters don’t map to uppercase letters the same way they do in English.
For most tools, rules can be tweaked as false positives arise. However, in some cases, development organizations find it easier to change the way they code. That was the case not long ago, when the SAMBA group kept getting false positives on code developed by a “particularly clever” coder, Allison says. He adds, “I found it easier to rewrite the clever parts of the code that were tripping Coverity up.”
For reasons like those, static analysis tools for now are only seeing general acceptance in military, financial, critical software and other high-assurance environments, according to analysts. For example, GrammaTech is making inroads into healthcare, with its C and C++ analysis tool working on a large number of medical and other embedded systems, including Symbian. In addition, the company has a DARPA contract to develop static analysis for multicore systems, an area in which Coverity is already strong, according to Lanowitz.
“When you’re looking at multiprocessor technology pulling from all these applications, there are so many things that can go wrong from a security perspective. Applications may not close sufficiently, memory allocations may be too small, deadlocks in the security chain,” said Lanowitz. “This type of forward thinking in the market is helping us solve new problems before they become too large.”
Related Search Term(s): Security, testing & troubleshooting, Coverity
Share this link: http://www.sdtimes.com/link/32548