RUC's ME-Decoding uses Mahalanobis distance to preserve semantic diversity in LLM decoding
jiqizhixin · x · 2026-10-08
Renmin University of China presents ME-Decoding (Mahalanobis-Ensemble Decoding), accepted at EMNLP 2026 main conference.
Problem: Top-p and Min-p truncate candidate tokens by probability, but high-probability tokens aren't always informative — several can be semantically near-identical phrasings of the same path, while a lower-probability token opening a genuinely different semantic direction gets cut early. The candidate set looks reliable but is packed with redundancy.
Method: ME-Decoding injects semantic redundancy into decoding, using Mahalanobis distance to measure similarity between candidate tokens so selection reflects semantic diversity rather than raw probability alone.
More from Models
- Small model swarms are a mirage: 20x cheaper models burn 40-50x more tokens — pvncher · 2026-10-08
- Roko: pretraining vs RL compute split shifted massively since late 2025 — 'two different species' — yacineMTB · 2026-10-08
- Burned 300B Tokens with Nothing; Internal Model Broke Through in 3 Hours — burny_tech · 2026-10-08
- Only 162 of OpenAI's 722 math papers carry Lean-verified proofs — gerardsans · 2026-10-08
- 177B Qwen 3.8 Flash Next Hits Steady ~45 tps Decode on a 128GB Strix Halo — deepu105 · 2026-10-08
- DeepSeek's weekly OpenRouter usage topped OpenAI, Google, Anthropic and xAI combined — aigclink · 2026-10-08