Source-Code Assessment Tools Identify Security Holes Resulting from Shoddy Development
By Jeff Forristal
Infosec practitioners worth their Red Bull know that perfect security is an ideal worth striving for but extremely difficult to achieve. No application of any size and complexity can be perfect in its first implementation; bugs will be present, and some will affect security.
Some software vendors seem to operate on the premise, If bugs are a fact of life, why expend time and resources to find and remove them before an application is released? Why not just issue patches and updates as holes are discovered? Seasoned software makers know that fixing bugs after the general release is more costly than finding and removing bugs during development (check out Applied Software Measurement: Assuring Productivity and Quality, by Capers Jones [McGraw-Hill, 1991] for supporting evidence).
If the financial bottom line isn’t reason enough, how about this: More and more attackers are reverse-engineering application security patches to uncover vulnerabilities. It is far quicker and easier to recover technical details based on what is fixed by a security patch than it is to discover a vulnerability. Sabre Security even offers a movie showing how reverse-engineering is used in its BinDiff product to analyze the vulnerability fixed by Microsoft’s MS05-025 patch. And the still-present gap between the time a vendor releases a patch and the time the patch is widely deployed means attackers have enough time for exploitation. The concept of 0-day (unknown/unpublished) vulnerabilities is becoming moot when 1-day (recently patched) vulnerabilities are just as lethal and far more research-efficient. Of course, this phenomenon is brought to us courtesy of vendors that follow the “release now, patch later” security cycle.
We understand secure application design isn’t easy. A high level of programmer education and awareness are required, as are continuous security assessments and process evaluations. Depending on humans for this analysis work is risky–people make mistakes and don’t scale well to handle large, complex applications; therefore an element of automation is practically a necessity. But the source-code security assessment market has been growing, with a slew of vendors offering products that support different programming languages and take different analytic approaches. We set out to determine how mature the market is, how the various tools work and how good a job they do. Which features matter, and how do the free and open-source options compare with the commercial products? This primer and the corresponding product writeups are based on our market analysis and examination of 12 source-code assessment products that focus on the security ramifications and remediation of discovered defects. (We didn’t include general defect-discovery products that don’t identify a defect’s security implications.) We examined Compuware DevPartner SecurityChecker, Fortify Software Source Code Analysis Suite, Klocwork K7 Development Suite, Microsoft Visual Studio 2005 Team Edition, Ounce Labs Prexis, Parasoft Jtest, Secure Software CodeAssure, SPI Dynamics DevInspect & SecureObjects, as well as the open-source Cigital ITS4, Flawfinder, Secure Software RATS (Rough Auditing Tool for Security) and Splint 3.1.1 in our Portland, Ore., Neohapsis partner lab. Our features chart summarizes the offerings.
Notable vendors missing from our list include Coverity and Application Security. To let us evaluate its product, Coverity sought to impose legal restrictions that could have conflicted with us publishing our impressions of its product in this article. Application Security never responded to our e-mail messages or phone calls. We didn’t consider other open-source projects, including CQual, MOPS, BOON, PScan and FindBugs, because of their niche focus or the nonproduction/beta quality of the current version.
Old Idea, New Offerings
The concept of using an automated tool to find defects in an application’s source code is nothing new. Many products discover general application defects and programming policy violations. Even the notion of discovering security-specific defects isn’t unique, as security defects are a specialized subset of general defects. Purists would argue that most application defects are security defects–anything that can cause an application to crash or consume an excess of resources affects availability, and thus can be vaguely classified as a denial-of-service attack.
We weren’t surprised then to see some vendors taking general defect discovery products and repurposing them with a security spin. A perfect example is Parasoft’s Jtest, a Java defect-discovery and -testing tool that offers a subset of security-specific analysis rules. On the other end of the spectrum are commercial products designed to tackle the challenges of security analysis. Products from Ounce Labs and Fortify Software, among others, exist for the sole purpose of performing application security assessments.
The origins of these products also vary widely. Coverity’s Prevent was spawned from Stanford University’s xgcc/Metal research, and Splint hails from the University of Virginia. Microsoft gained Prefast’s predecessor through its 1999 acquisition of Intrinsa, and Klocwork was spun out of Nortel Networks. The collective driving motivator is the desire to discover, quantify and remove security risks in an application at as early a stage as possible.
It’s not just developers who can benefit from source-code security assessments. All the tools we examined assume (exclusively, in some cases) a developer audience–which makes perfect sense because developers are writing the insecure code–but QA managers and engineering VPs who oversee the quality aspect of the application-development process will find these tools provide valuable insight. Project leads and development liaisons can analyze code created by outsource development firms to judge their overall attentiveness to and understanding of security. Global security teams and CSOs can discover what general security problems are likely to exist before deploying a software package. Application security auditors can use these tools to find security bugs faster and more effectively. Even tech-savvy legal departments can leverage security analyzers to determine the risk and liability presented by a collection of source code.
Klocwork® offers a portfolio of development productivity tools designed to ensure the security, reliability and maintainability of complex code bases. Using proven static analysis technology, Klocwork’s tools identify critical security vulnerabilities and reliability defects, optimize peer code review, and help developers create more maintainable code. Klocwork’s tools are an integral part of the development process for over 1000 customers in the consumer electronics, mobile devices, medical technologies, telecom, military and aerospace sectors.
Klocwork and the Klocwork logo are registered trademarks of Klocwork, Incorporated in the United States and other countries. All other names are trademarks or registered trademarks of their respective companies.