Most of our tooling was built for one kind of error. The expensive one is the other.
A cheap, fast model worked for over an hour on a coupled task without falling apart, and that didn't square with anything I knew about these models. They're lazy on long tasks, and the reputation is earned: you hand it an hour-long task and it delivers half, drifts, or declares victory early and leaves you with a result that looks done and isn't. This time, running inside the same disciplined workflow I already use, the usual thing didn't happen. And the task wasn't an easy one — a progress callback that had to thread through an entire pipeline up to the interface, the kind you can't slice into independent pieces, because showing progress only works if the callback crosses everything end to end. The only difference from the times I'd watched these models give up was that the plan had grown a list of deliverables: concrete states that had to exist by the time it was done.
The easy inference is that a weak model needs a list, and I went looking for whether anyone had already measured the effect of explicit deliverables — more out of a habit of never letting an explanation stand without checking it than out of any real suspicion. And it was the search that started knocking my explanations down, one by one.
