We were backtesting an LLM-driven earnings signal against a field called epsActual — the kind of field everyone treats as ground truth. It isn't.
About 41.4% of those "actual" values were different from what the vendor had first reported. About 15.3% differed enough to flip a tradeable decision. When we re-ran the backtest using only the values that actually existed at each decision date, the strategy kept ~73% of its returns and ~82% of its Sharpe. The rest was look-ahead bias — and it rode in through a field whose name promised it was final.
This is a writeup of how we found it, how we measured it honestly, and the one-line invariant that turns it from a silent inflation into a loud test failure.
The setup
The signal is a post-earnings drift play: at each earnings print, an LLM scores the release and we take a position. To backtest it you replay history — for every past print, reconstruct what the model would have decided, then check what happened next.
