DLite: History-Free Speculative Decoding Beats DSpark by Up to 70% for Agentic AI
pmttyji · reddit · 2026-10-20
- SJTU researchers introduce DLite, a history-free speculative decoding framework targeting agentic workloads.
- Existing draft models suffer up to 80% degradation in acceptance length on evolving long contexts, while keeping historical KV caches adds heavy system overhead.
- DLite reuses contextual hidden states from the target model for local speculative prediction without maintaining history, with a lightweight sequential head restoring intra-block causal dependencies and a two-stage training strategy.
- It achieves 2-6 acceptance lengths and outperforms prior SOTA DSpark by up to 70%. Code and weights are open-sourced.
More from Infra
- Victor Taelin publicly begs to buy an 8-GPU Rubin machine — 'nobody is selling it' — fforres · 2026-10-21
- Nscale joins PyTorch Foundation as Platinum member after Anyscale acquisition move — PyTorch · 2026-10-21
- Decision Models launches on Antseed with an open-source auto model router — airesearch12 · 2026-10-21
- Gradium open-sources Phonon: 31M/40M edge TTS models with Rust inference stack for 10+ backends — l-m-z · 2026-10-21
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20