SwitchSD, speculative decoding that switches between neural drafting and context copying, accepted to NeurIPS
arankomatsuzaki · x · 2026-09-30
royeisenstadt announces SwitchSD has been accepted to NeurIPS 2026. SwitchSD is a speculative decoding method that uses the target model's hidden states to decide when to run a neural drafter versus simply copying tokens from context, dynamically switching between the two strategies to cut unnecessary drafting overhead. The author shares the full research story in the linked thread.
More from Infra
- Jensen Huang: AI data centers add 10-20GW a year and roughly a million jobs — victor_explore · 2026-09-30
- First SGLang Summit set for Nov 12-13 in SF, with Intel CEO and Lilian Weng speaking — BanghuaZ · 2026-09-30
- Interview with Richard Ho, leader of OpenAI's in-house chip project — bookwormengr · 2026-09-30
- Tenstorrent opens bio-model training: OpenFold3 on Blackhole Galaxy nears DGX H200 at quarter the cost — MoAlQuraishi · 2026-09-30
- On-device inference is finally practical: NPUs shift AI workloads off the cloud — lmoroney · 2026-09-30
- Cognition runs production workloads on Vera Rubin NVL72 with 4.8x token throughput — nordicinst · 2026-09-30