In my last post, I laid out the core inequality of Speculative Decoding: a > 1 + α +...
Speculative Decoding su CPU risulta 49-62% più lenta dell'autoregressive standard; 15-30% dei draft round sono rigettati completamente, neutralizzando guadagni teorici. La tecnica è GPU-bound — su GPU α<0.15, su CPU α≈0.3 — rendendo il draft model non competitivo in ambienti CPU-constraint.
In my last post, I laid out the core inequality of Speculative Decoding:
a > 1 + α + β
Acceptance length a must exceed 1 plus the draft/target compute ratio α plus verification overhead β. If it does, SD wins. If it doesn't, SD loses.
That was theory. This post is the practice.
I ran a real A/B test on my machine. The results were worse than I expected — and more interesting.
Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding | NVIDIA Technical Blog
DFlash Speculative Decoding Drafts Whole Token Blocks in Parallel for Up to 15x Higher Throughput on NVIDIA Blackwell
DeepSeek Releases DSpark, a Speculative Decoding Framework That Accelerates DeepSeek-V4 Per-User Generation 60–85% Over MTP-1
Deepseek's DSpark boosts AI speed by up to 85 percent, a strategic win under tightening US export controls
Sonnet 5 review: I ran 64 generations to find out if it's worth it
DeepSeek unveils DSpark for 60% to 85% faster inference optimization
Introduction Speculative decoding is one of those techniques that has been "almost ready...
Speculative decoding: how a small draft model plus a single-pass target verification cuts TTFT, the variants that actually work,…
We introduce SpecExec, a new speculative decoding method that applies the classical approach of “speculative execution” to LLM…
One of the hottest topics in LLM inference acceleration right now is Speculative Decoding. DSpark...
TLDR: In this blog post, we show that using a custom speculator—trained on your own Deepseek-R1 inference traffic—can yield…
Pattern Defined Precise Definition: Speculative Decoding is an optimization pattern where a...
