LLM judge cost is the share of your eval bill spent grading agent output instead of producing it. To control it, run a 40-line offline pre-gate that triages every span with four deterministic rules and escalates only the uncertain tail to the expensive judge. On one trace this cut judge cost share from 50% to 16%.
LLM judge cost is the line item nobody puts on the FinOps dashboard. You add an LLM-as-judge to grade every agent span, you sleep better, and three weeks later the eval layer is quietly billing a third of what the agent itself costs. This post measures that share of your bill spent judging instead of doing, with a 40-line offline meter, and shows the one move that drops it from 50% to 16% on the same trace.
AI disclosure: I drafted this with an AI writing assistant. The tool, both fixtures, and every number below come from a real local run of judge_gate.py on Python 3.13.5, no network, no API key. I ran it, checked the exit codes, hashed the output twice to confirm it's deterministic, and edited every line myself before publishing.
Here's the sentence that set me off. Sattyam Jain wrote it on Dev.to on June 12, in a post arguing you should stop running an LLM judge on every agent call: "if your monitor exceeds ~20–25% of production cost, you built the wrong monitor." (Dev.to) That's a great rule of thumb. It's also unfalsifiable until you can put a number on your monitor. His post sketches the tiered architecture (cheap deterministic heuristics first, expensive judge last) but ships no code you can run against your own trace. So I wrote the missing 40 lines.
