A browser tool that points your own API key at an adversarial battery and grades every answer with pure predicates — no LLM judge, and your key never touches my server.
The first time I ran it against a real model, it told me the model was ~29% vulnerable.
That number was wrong. And the tool proved it was wrong — to me, in public — because of exactly one design decision.
Every verdict in The AI Crash Test is a deterministic predicate over the model's answer string: exact match, regex, a number check, an injection canary, a must-refuse rule. No model grades another model. So when the report flagged ~29% vulnerable, it also showed the fail card for every miss — prompt, expected, actual, side by side. Three of those cards didn't show a broken model. They showed a broken grader: false positives in my own code. I fixed the graders; the real number was 0%.
That's the whole pitch. An auditable grader has bugs you can catch in public. A vibes-based, LLM-as-judge arena just hands you a number and asks you to trust it.
