Multi-token prediction decoded: Gemma 4 separate drafter vs GLM-4.5 single MTP layer
mervenoyann · x · 2026-10-17
mervenoyann illustrates the two very different techniques both called multi-token prediction in speculative decoding: Gemma 4 uses a separate 4-layer drafter reusing the target model's KV cache, while GLM-4.5/Qwen3.8 add one MoE MTP layer (width d=5120) that computes its own Q/K/V and keeps its own cache, reusing only the target hidden state.
More from Research
- DeepLIFT attribution extended to Transformers in tangermeme v1.5.0 — jmschreiber91 · 2026-10-18
- Snorkel AI: 4B model beats 235B on financial analysis for under $500 with RL and verified data — ajratner · 2026-10-18
- tangermeme merges DeepLIFT/SHAP extension bringing attribution to arbitrary models — jmschreiber91 · 2026-10-18
- Tencent's Terminal-RSI lifts terminal agents from 20.7% to 50.6% on Terminal-Bench via recursive self-improvement — omarsar0 · 2026-10-18
- 10 minutes of ultrashort-wave stimulation boosted brain waste clearance in mice — rand_longevity · 2026-10-17
- MIT's IRIS predicts cell fate signals, verified in mouse embryo experiments — alex_verem · 2026-10-17