Last week, Facebook released their code analyzer tool, flint, out into the open source community. This is a good thing for programmers: you’re getting a free linting tool that’s been developed and used by a world-class software team. But it’s also causing some confusion as people are thinking “great, now all my testing by static code analysis needs are met!” Here’s why this couldn’t be further from the truth.
Flint is a linter, or a program verifier whose sole purpose is reporting violations in code against a set of pre-defined rules. Linters can operate on one file or multiple files but there’s absolutely one thing they’re not designed to do: perform whole-program analysis driven by an understanding of all the control and data flow interactions within and between software units. In other words, they help automate the discovery of defects that people could find by themselves but they’re not smart enough to find defects that are almost impossible for people to find on their own.
As described in a post by the creator of Flint, Andrei Alexandrescu, the tool is “token-oriented, as opposed to building a parse tree. The linter loads the input file, converts it to an array of tokens, and then runs various analyses against that array”. This approach limits the analysis to verifying token arrays independently from each other and is unable to resolve complex or ambiguous interactions.
The post goes on to list the types of checks performed by Flint, which basically boils down to checks against corporate coding standards and common construct errors. This certainly improves code quality within a file but does little to improve the safety, security, and reliability of the overall application being deployed. That’s where more sophisticated static code analysis comes in. Higher functioning tools are able to analyze all software units in context, checking all interactions, estimating variable values, and simulating potential runtime behavior. This targets high-performance, mission-critical systems right where it counts, finding defects that no other type of tool can. Of course, they also check for programming errors and compliance against tough industry standards.
Flint is good for what it is but it’s severely limited in terms of what static code analysis can really do. Putting it out into the open is nice but development teams rely on much more to ensure they’re delivering safe, reliable, and defect-free code.