I found 10 bugs in my own security scanner. Here's what they taught me about false positives.

I found 10 bugs in my own security scanner. Here's what they taught me about false positives.

I built a VS Code extension that scans code for leaked secrets, PII, and security vulnerabilities before you commit. A few weeks in, I sat down and did something most tool builders put off: I went looking for everywhere my own scanner was wrong.

Not "wrong" as in crashing. Wrong as in flagging things that weren't actually problems, and — worse — missing things that were.

For a security tool, false positives aren't a minor annoyance. They're the thing that gets a tool uninstalled. The first time a scanner cries wolf on loginToken because it contains the substring "log," a developer stops trusting every other finding it makes, including the real ones. So I spent a focused pass hunting for exactly this kind of bug. Found 10. Here they are, with the actual root cause for each — because "we fixed some bugs" isn't useful to anyone, but the specific ways pattern-matching security tools go wrong might be.

1. Substring matching instead of call-site matching