As any static analysis or source code analysis vendor will tell you, false positives are a way of life. As any user will tell you, false positives suck! So what do you do about them? Make the tools better at finding the real issues and provide automated filtering capabilities. But I’m not here to talk about false positives where the tool is utterly wrong. What I want to talk about today is what I call “perceptual false positives”.
I’ve had discussions with customers where they tell me 80% of all their defects are false. Odd, I know static analysis tools today are much, much better than that. In fact it usually tends to be the other way around — 80%+ are real. So in situations like that, you have to analyze the defects further. After analyzing a few I can see many have been marked as “false positives” even though they are real. When I challenge the customer it fits into either one of two categories:
1. I don’t care, that will never happen in my lifetime.
2. You’re wrong.
For #1, not everyone believes in defensive programming. In some cases it doesn’t even make sense, in others I’ll still argue that it is important. For example a customer once told me that the memory leak we identified is real but they don’t care, because it is initialization code. Meaning when it starts, it leaks memory but it is closed off shortly after that. I understand that sometimes you don’t have time to address every single issue, but what if someone decides to learn from you and copies that code. Or later that code turns into something more than initialization. It will happen.
For #2, typically the customer believes we are outright wrong. Certainly in some cases we are, but in others we are just not providing the proper tools for the developers to understand the issue. The best tool for that is what’s called “traceback”. Traceback provides you clear instructions on the conditions and assignments of any detected defect. One of my favorite examples is a buffer overflow. The traceback below shows you the size of the array and where you are exceeding the bounds of that array.
In this example, it says we exceed that bounds by 15 (we are accessing the array from values 0-65). The obvious question now is where on earth do you come up with 65? When we expand the traceback, we can see the full details of that calculation. The traceback proceeds through the details of every expression until we arrive at the right value, in this case 10+10+10+10+10+10+5=65.
False positives may be a way of life but when addressing them it is important to make sure you are not looking at a perceptual false positive. The traceback tool is helpful in assessing not just perceptual false positives but other identified defects as well. It will only make your life easier.