High-Security Systems Need Open-Source Software

By Robert Dewar

January 15, 2007 —

Many applications depend on software in a very critical way. Think about safety-critical applications, such as those used to control the avionics of commercial and military planes. Every time you get on any modern plane such as a Boeing 777, you literally place your life in the hands of a complex software program. A bug in this program could easily cause a fatal crash.

And yet we have had no deaths on commercial aircraft that can be attributed to software errors. We actually know pretty well how to build reliable software when we need to. It takes special techniques, which for sure are not used in most of the unreliable software that we see around. These techniques involve use of appropriate languages, as well as specialized testing and certification protocols. In some cases, mathematical “formal” methods are used to demonstrate that the code is error-free.

This surprises some people, who think that it is normal for large systems to be full of errors, and certainly we are surrounded by unreliable software. The 777 actually is a very nice illustration: The one component of the software that was not designed to meet safety-critical requirements was the cabin entertainment system, and as many experienced travelers know, this system is full of glitches and often breaks down.

Another case arises in the high-security area, where we worry not only about bugs, but about external attacks, including possible cyberterrorism. Just as an architect designing a building has to take into account not only structural soundness in “normal” conditions (including extreme weather), but also intentional attacks from terrorists or others, there are applications where being bug-free is not good enough.

THE VOTING MACHINE CASE
A case of software security that has center-stage attention these days is the code that controls voting machines. Voting machines are not a safety-critical application in the classical sense; no one dies if the application fails. Well, more accurately, no one dies directly, but indeed the possible consequences of inaccurate recording of votes are staggering, and in the worst case it could undermine critical public faith in the voting process. This is arguably an even much more critical application than the 777 avionics.

Consequently, you would assume that the software is written with the same kind of care. Well, if you assume this, you are sadly mistaken. The code is proprietary, and undergoes no kind of formal certification or testing of the kind we use for planes. Furthermore, as evidenced by reports emerging in news stories, the software is full of errors. As one example, the recent disappearance of thousands of votes in a Florida district in the 2006 election confirms our worst fears. Just suppose that control of the Congress depended on that outcome—we would be in a very serious situation.

The voting machine case also raises the security issue in an extreme form. Any number of parties both internal and external could be interested in influencing our elections. Computer science professionals have repeatedly demonstrated that the voting machines in current use can be hacked into changing results, and far too many people are in a position to duplicate this kind of feat.

So what shall we do to prevent this? One proposal is that we should make the code in voting machines open source. This discussion has heated up in recent days (see The New York Times front page story from Dec. 8, 2006, “Changes Are Expected in Voting by 2008 Election”). The Federal Election Assistance Committee is requiring that voting machine code be made available for federal inspection (currently it is a closely held proprietary trade secret, and we rely on the companies to test their own code). A bill proposed by Rep. Rush Holt, D-N.J., would go further and require the code to be made public, a step toward the open-sourcing of such code recommended by many scientists in the field.

CLARIFYING OPEN SOURCE
Let’s look at exactly what we mean by open source. It is indeed a somewhat general term, with several different components. First, there is the important fundamental quality that the source code can be examined by anyone. That’s probably the most important aspect in the voting machine case. Second, there is the issue of avoiding normal proprietary restrictions that prevent redistribution and modification of the code.

In the case of voting machine code, it is important that if flaws are found, they can be fixed without the manufacturer standing in the way. Finally open source refers to a free environment in which code is created by an open community cooperating.

This open development can be useful in achieving rapid development, but must of course be somewhat restrained if we want to build highly reliable systems. We can’t have anyone anywhere making arbitrary changes to the software. In fact, in practice, open-source software is often very carefully controlled by its community.

At first glance, the idea of making the voting machine code public seems like an obviously advantageous idea. Not only do we now have thousands of skilled pairs of eyes examining the code for flaws, but we can also assess the quality of the code.

If we see a pile of ill-organized uncommented code written in C++ using all the tricks of that complex language, we may conclude that such code was not built in a manner consistent with requirements for security. To produce reliable systems, we have to use rigorous development techniques, analogous to how an architect designs a building. We don’t let just anyone design and construct a building in a haphazard manner, leaving it to building inspectors after the fact to guarantee safety, and we can’t produce reliable software in a haphazard manner either.

In the software arena, we use appropriate languages for safety- or security-critical software development—for example, a rigorously controlled language subset designed for such purposes—and we follow standardized development techniques.

If/when the code and its supporting project documentation are opened up, we can check that the system was developed using appropriate techniques, including perhaps the formal testing materials we would expect for avionics code.

Furthermore, if we open-source the code, the hackers of the world (and we use this term in its original non-pejorative sense) who delight in finding errors in code, will be quick to point out problems they find, and report them.

A LOOK AT THE OBJECTIONS
All in all, open-sourcing voting machine code seems a clear win, so what’s the objection?

There are two objections. First, the manufacturers claim proprietary rights, and object to their trade secret code being revealed.

Frankly, I find such objections disingenuous. If the designers of the new building replacing the World Trade Center refused to share any details of their design with the public, and simply promised us that it would be safe, we would not let them break ground. The security of our republic depends on reliable voting in which the public has confidence. The proprietary rights of a few companies cannot take precedence over this. There are many ways in which such companies can be properly compensated for their work, not least of which is the revenue from selling the machines.

A more fundamental argument is that open-sourcing the code will reduce security by giving hackers (in the more modern evil sense), terrorists and others who would subvert our democracy access to the code so they can look for flaws. Michelle Shafer, a vice president at Sequoia Voting Systems, said that while the industry was willing to give the source code to a small number of specific individuals with a “need to know,” making it universally available would give it to people with malicious intent, making the system less secure.

Is this argument valid? Well, certainly if the code is made public, there will be those looking for flaws to exploit. So are we better off relying on keeping the code secret? Perhaps we might think so, if there were such a thing as a secret (something no one knows). But in real life there are no secrets. If we rely on secret code that is buggy and attackable, then we are relying on the integrity of those who know the secret.

The “No More Secrets” phrase from the movie “Hackers” is really what governs here. We just cannot rely on secrets to protect knowledge of serious flaws. Even if the secret is held carefully, competent hackers can reverse-engineer software components to find out what is inside them. (This happens all the time with Microsoft’s Windows, where security flaws are discovered using this technique.) I once heard a talk by Markus Kuhn, now at Purdue, at that time at Cambridge University, in which he described the techniques that had allowed them to reverse engineer all smart cards they had ever seen in the lab, and determine the code they contained, despite the fact that these cards were designed to make this impossible.

Instead of relying on secrecy, we must achieve code that is demonstrably error-free and safe from attack. The only way to achieve this is to allow general public inspection, and the only way that the public can have confidence is if the professional community can examine the code and proclaim that it meets the highest standards. The current situation is that this community of experts, relying on reverse engineering, finds lots of errors and undermines any confidence we could have that the code is correct.

RESTORING CONFIDENCE
Voting machines are a pillar of our democracy, and just as democracy belongs to the people, all the details of this pillar must also belong to the people. If we lose faith in our electoral process, the damage done is irreparable. In the coming months, the debate on the Holt bill will make this issue central to the public debate. I see only two viable possibilities. Either we return to paper and mechanical devices we trust, or we open up the technology so that the new electronic machines can earn the same level of confidence. Perhaps the public debate over the voting machine case will serve to educate the public on this important general issue.

Robert Dewar is president and CEO of AdaCore, and is also emeritus professor of computer science at New York University.

Share this link: http://www.sdtimes.com/link/29989

News Briefs: August 1, 2008 Microsoft and ASG team up to create an API connecting their products, DevExpress has a beta of its AgDataGrid suite for Silverlight, and TotalView's source debugger has been updated to allow it to run on the Cell Broadband Engine architecture.

News Briefs: January 15, 2009 Cigital and Coverity partner together to improve each other's products, Scio Consulting creates services for SaaS environments and AmberPoint integrates its SOA governance and management precuts with Microsoft the Microsoft platform.

SOA Software lets developers check out candidate services Portfolio Manager visualizes interrelationships between business imperatives and candidate services to help in planning

Add comment

Name*
Email*
Country

Comment
Preview

News on Monday
more>>
SharePoint Tech Report
more>>

Download Current Issue
ISSUE 3/15/2010 PDF

Need Back Issues?
DOWNLOAD HERE

Receive the print Edition?

Google Code turns 5
Google Code Turns 5, and adds a Paxos Algorithm to make the system more stable and reliable.
03/17/2010 11:16 AM EST

Test your Visual Studio 2010 know-how
Microsoft is offering free beta certification exams for Visual Studio 2010.
03/17/2010 11:08 AM EST

Microsoft lifts the hood on IE9
Microsoft is previewing IE9.
03/16/2010 01:10 PM EST