NVIDIA's NarrowSpec: Halving Draft Hidden Width Still Speeds Up Speculative Decoding 1.44x

nvidia · hf · 2026-11-20

NVIDIA shows in NarrowSpec that draft models borrowing the target's hidden states (DFlash, EAGLE-3) don't need to inherit its hidden width—the draft's residual stream can run at half or quarter width while largely preserving acceptance.

Original post →

More from Infra

Infra channel →