Cohere Paper Explains Why Speculative Decoding Beats Predictions on Sparse MoE

Cohere's new paper (arXiv:2610.38981) finds that adjacent tokens share 20-45% of experts in sparse MoE models, making speculative decoding cost-effective even at low batch sizes and faster than conventional predictions.

2026-11-03 ~ 2026-11-03 · 2 related posts

• Cohere paper: adjacent tokens share 20-45% of experts, making speculative decoding cheap even at batch size 1 — rohanpaul_ai · 2026-11-03
• arXiv: Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models — rohanpaul_ai · 2026-11-03