A new paper introduces a method to speed up reward-based fine-tuning by having the model generate a cheap, compressed copy of itself to draft text, which the full model then verifies rather than writing from scratch. The approach, called self-speculative decoding, achieves meaningful speedups in generation with no loss in final model quality — the finished model is identical to one trained without the trick. The key insight is that the clone is re-created from the live model at every training step, so it never drifts out of sync.
Key facts
What: Teaching a model with rewards is slow because it has to write out endless practice answers. A new trick: make a cheap, shrunk-down copy of the model to crank those out faster.
When: 2026-06-19
Primary source: read the source (arXiv 2606.18967)
