ME-Decoding from RUC treats LLM token selection as ensemble pruning, beats Top-p across benchmarks
机器之心 · wechat · 2026-09-30
A Renmin University team (advised by Wenlin Dai and Cheng Meng) proposes Mahalanobis-Ensemble Decoding, accepted to EMNLP 2026 Main. Instead of truncating by single-token probability like Top-p/Min-p, it jointly scores confidence and semantic redundancy of the candidate set, greedily growing it with an early stop until the Mahalanobis-Ensemble Score peaks.
Results: highest average accuracy on GSM8K (72.66%) and GPQA (32.96%) across Qwen3-4B, Phi-4-mini and Mistral-7B at temperatures 1.0–2.0, staying robust where truncation methods degrade at high temperature; top AlpacaEval winrate and MT-Bench scores. Gains persist when support size, entropy and cosine similarity are matched, and end-to-end GPU latency rises only 3% over the fastest truncation baseline. Paper: arxiv.org/abs/2609.18723, code open-sourced.
More from Research
- NanoStories: 50M-story TinyStories-style dataset generated by rules in 10 minutes — Tall_Abrocoma_3533 · 2026-10-23
- Is formally verifying an AI agent sandbox a Millennium Prize-level problem? — teortaxesTex · 2026-10-23
- General Reasoning's RLCR trains agent organizations with collective rewards — hierarchies emerge — daniel_mac8 · 2026-10-23
- JUICE: one flow-matching Transformer unifies image and text generation, hitting 0.88 GenEval — pratyusha_PS · 2026-10-23
- DeepMind's Verena Rieser to keynote EMNLP 2026 on a science of constitutional alignment — verena_rieser · 2026-10-23
- Anthropic uses Claude to write 180M lines of proofs, verifying 123 open-source libraries — The Decoder · 2026-10-23