2DSD Speculative Decoding Speeds Up Looped LMs up to 7.64x With Lower KV Memory

LucaAmb · x · 2026-11-20

Looped LMs add capability via depth rather than parameters, but each extra loop costs KV cache and per-loop latency. The team behind 2DSD (Two-Dimensional Speculative Decoding) combines speculation along both tokens and depth with cross-loop KV sharing.

Related event: 2DSD Speculative Decoding Speeds Up Looped LMs up to 7.64x With Lower KV Memory(2 posts)→

Original post →

More from Infra

Infra channel →