I run a public board that probes 16 LLMs on a frozen 35-task suite, once a day, and keeps every...
I run a public board that probes 16 LLMs on a frozen 35-task suite, once a day, and keeps every score. When a model drops against its previous run, it opens a GitHub issue by itself and writes me a draft post.
Between 21 and 24 July it did that four times:
23 Jul Gemini 3.5 Flash -11.4 pts
24 Jul Gemini 3.1 Pro -2.9 pts
21 Jul Grok 4.3 -5.7 pts
Same outputs, same judge, two runs, two scores. The gate flickered red then green on a branch with...
model-drift grades models weekly on a frozen suite with an exact-match grader — no LLM judge, so a...
We track weekly agreement between an LLM judge and human labels (Cohen's kappa) on a sample of...
Opus 4.8, Gemini 3.5, GPT-5.5, and a $965B valuation. The LLM race accelerated into something unrecognizable from six months ago.
At 2:17 a.m., a chain of alarms yanked me out of sleep — the LLM in production had suddenly "lost its...
Your LLM provider silently swapped models under you. Here is how to detect model drift with 6-dimension contract validation.
