I've been tuning Codenames AI, a small web game where an LLM plays Codenames with you. Clue generation is tightly constrained: one word, a count, optional intended targets, JSON on the wire, then deterministic validation before anything reaches the board.
As the project started attracting regular players, I wanted to improve the gameplay experience without blowing out costs. Moving one model generation from gpt-4o-mini to gpt-5-mini was my first instinct.
The default reasoning setting made responses an order of magnitude slower for this workload. Minimal reasoning looked like the obvious compromise: newer model, responsive gameplay.
I expected to compare clue quality, latency, and cost while the surrounding prompt, validator, and consumer contracts stayed put.
That last part was wrong.
