In May I promised — out loud, on the internet, where people can screenshot you — that I'd grade...
In May I promised — out loud, on the internet, where people can screenshot you — that I'd grade Helium's published prob_itm forecasts when the June 2026 AAPL contracts expired.
June 26 came. The contracts died. So I graded them.
This is not a victory lap. Mean Brier loss on n=2 is 0.3846. A coin flip that always says 0.5 scores 0.25 when the world resolves to 0 or 1. We did worse than a coin flip. That is information.
What we froze in May
Contract
A walkthrough of one recipe from a new open-source cookbook for the Helium MCP REST surface. Log a model's prob_itm forecast…
A browser tool that points your own API key at an adversarial battery and grades every answer with...
This is a submission for the GitHub Finish-Up-A-Thon Challenge What I Built In 2017 I...
Practical techniques for evaluating LLM outputs in production, from a full-stack AI engineer who ships AI features daily.
Two new benchmarks put hard numbers on the AI-analyst gap, and "not good enough yet" turns out to be a powerful capex…
What if your code got a grade — like a school report card — but brutally honest? I built exactly...
