Speculative decoding speeds up dense models more than MoEs, as intuition suggests
mervenoyann · x · 2026-10-20
ML engineer Merve Noyann shares a TIL: speculative decoding yields better speed-ups for dense models than for MoEs — a result that matches intuition.
More from Models
- Mistral's 209M SOTA time-series foundation model Timestral lands on Ephemeris a day after release — const_reborn · 2026-10-20
- Dev hopes MiniMax 3.1 Flash ships under 230B parameters — gnukeith · 2026-10-20
- Anthropic insider: Opus 5 is a kind but alien soul — work with it at a distance — repligate · 2026-10-20
- Anthropic vows to keep Opus 3 forever as OpenAI sunsets 4o with two weeks' notice — repligate · 2026-10-20
- Artificial Analysis launches AA-PersonalityBench: 30 personality traits and 21 communication styles across LLMs — ArtificialAnlys · 2026-10-20
- Gemini on Pixel 10 stuck replying 'I am just a language model' to every prompt — OmnomtheDoomMuncher · 2026-10-20