I benchmarked Python AI-app security scanners. Here's what each catches.

This week I shipped Python AI-app regex prefilters in getdebug 0.4.0 and benchmarked them against Bandit and Semgrep on real Python code. Here are the numbers and what each tool actually catches.

The four tools

Bandit (PyCQA) — the Python-OSS standard security linter. Hand-written rules, free, fast, Python only.

Semgrep — multi-language SAST with community rule packs. Hand-written rules, free, fast.

vulnhuntr (Protect AI, open source) — the stated category leader for LLM-driven AI-app static analysis. Python only.