Training-Free Speculative Decoding Cuts Latency for Full-Duplex Speech LMs
CUHKSZ · hf · 2026-11-06
Researchers from CUHKSZ introduce Speculative Listening, a training-free, lossless speculative decoding framework for chunk-interleaved full-duplex speech language models. Key insight: for 95% of chunks, model output is identical whether input is silence or real audio, so idle compute can draft and verify upcoming audio chunks in advance. When real audio arrives, a hypothesis pool enables prefill + verification with only rare first-token corrections. The approach substantially reduces speech response and tool-call delays.
More from Research
- Measured: in-process BM25 memory costs ~4.6KB RAM per entry, linear growth — Neither-Witness-6010 · 2026-11-06
- Gradient descent explained at 3 levels: from blindfolded hill-walking to non-convex dynamics — cneuralnetwork · 2026-11-06
- LLM writes 4-digit multiplication in just 355 weights, 40,000x fewer than SOTA, no training — willdepue · 2026-11-06
- Burkov proposes pretraining transformers on LLM weight files so they learn to edit model weights — burkov · 2026-11-06
- Post-training pipelines remain highly manual and far from automatable — menhguin · 2026-11-06
- Anthropic report: full AI automation could yield 15-16x research output, labs remain the bottleneck — teortaxesTex · 2026-11-06