Here is the kind of sentence that should keep a team up at night: "At this clarity grade, you won't...
Here is the kind of sentence that should keep a team up at night:
"At this clarity grade, you won't see a thing with the naked eye."
Our assistant produced it while explaining a diamond. The grade it cites is real — SI1, printed on
the stone's lab report. The promise attached to it is invented. A clarity grade is assigned under
10× magnification; it does not resolve what a naked eye will see. Two SI1 stones can look entirely
Everybody's eval stack has the same load-bearing assumption nobody audits: that the model-as-judge is...
Part 2 of an eval series. A 15-line LLM judge, scored against real Chatbot Arena human votes. It agreed with people on just 43%…
A design pattern for LLM-as-judge: collect evidence with binary Yes/No checks, pick one named verdict holistically, and never…
Fail-closed groundedness, deterministic corroborators, and a self-test — because an evaluator should...
model-drift grades models weekly on a frozen suite with an exact-match grader — no LLM judge, so a...
Part 3 of an eval series. I tried to fix a 43%-agreement LLM judge two ways — a bigger model (DeepSeek & Qwen via OpenRouter) and…
