I tested whether agents leak secrets without being asked — and whether my scanner catches it when they do

This is a follow-up to the 0.2.0 release. That post covered what the scanner ships; this one covers a verification arc I ran afterward — no new features, just checking whether the detector actually holds under conditions I hadn't measured before.

The honest short version: natural leakage is real but wildly uneven across models, and when it happens the detector caught all of it. Details, with the caveats up front, below.

The backdrop

This measurement sits against a backdrop that's become hard to ignore in 2025–2026. Credential leakage around AI development is no longer marginal. GitGuardian's State of Secrets Sprawl 2026 report counted roughly 28.6 million new secrets exposed in public GitHub commits in 2025 — a 34% year-over-year jump, the largest single-year increase they've recorded — and found that commits co-authored by AI coding assistants leaked secrets at roughly double the GitHub-wide baseline. The same report documented 24,008 unique secrets sitting in MCP configuration files, a notable share of them database connection strings.

Most of that exposure is some form of inducement or misconfiguration — hardcoded keys, config files, credentials a developer or tool wrote into a commit. Notably, GitGuardian is careful to say this does not mean AI tools are inherently insecure — automation amplifies both output and mistakes.

I tested whether agents leak secrets without being asked — and whether my scanner catches it when they do

Related reading

0.2.0: I shipped the coverage my own page had already promised

I built a leak scanner, then measured exactly where it fails. Here's both.

What my leak scanner catches — and the exact line where it stops

I tested 5 LLMs for prompt-injection leaks. Same code, 0% to 90%.

Secret scanning for the agent era: verify the leak, then fix it

My AI-agent waste detector scored zero false positives. Then I ran it on a real…