Free live paper discussion: why speculative decoding is fast despite doing more compute
doesdatmaksense · x · 2026-11-05
Isaac Flath and Joe Barrow host a free online paper discussion on Nov 4, 7:30–8:30 PM ET, on why speculative decoding speeds up generation even while doing more computation.
- A small model drafts tokens; a larger model verifies several drafts in a single forward pass
- They'll cover why memory bandwidth and spare GPU compute are the keys to the speedup
- How batch size and draft acceptance rate change the gains
- Registration includes the recording and notes after the talk
More from Infra
- CI costs scale directly with token prices, dev observes — aniketmaurya · 2026-11-05
- Snowflake adds Business Ontology and Cortex AI Gateway governance for enterprise agents — shashib · 2026-11-05
- Ricursive debuts from-scratch Ripple-65B: open-weight agentic coding SOTA at 1/10 cost — HazyResearch · 2026-11-05
- LexiSuite: an all-Rust local LLM stack with training, LoRA and trimming that claims to beat llama.cpp in spots — W61k3r · 2026-11-05
- Gary Marcus predicts the data center binge will collapse, with many bankruptcies ahead — GaryMarcus · 2026-11-05
- Merge's eval claims its MCP connectors beat Microsoft's native servers on tool count and reliability — shensi · 2026-11-05