Wrong Prediction, Right Answer: EMNLP paper shows a 'readout bottleneck' hides LLM reasoning
机器之心 · wechat · 2026-09-18
A PKU/YiXin AI Lab paper accepted to EMNLP 2026 identifies the 'readout bottleneck': LLMs often encode the correct answer in hidden states (probe accuracy up to 0.830 on Qwen3.5-9B hard slices) but lose it in the unembedding and sequence-score mapping — Qwen3.5-9B scored the same label on 999/1000 test items. Adding just 2 label-agnostic scalar offsets fit on 25 unlabeled samples recovers massive accuracy: ProofWriter jumps from 33.3% to 66%, with 9–29 point gains on FOLIO/ANLI and similar recovery on OLMo-2-1B and Llama-3.1-8B. Controls (TF-IDF hard slices, permutation nulls) rule out shallow shortcuts. The work reframes benchmark scores and explains why chain-of-thought helps: it distributes the output layer's readout burden across tokens.
More from Models
- 25M tokens later, a local Qwen user says prompt adherence is the real breakthrough — mateszhun · 2026-09-18
- Nate Silver: AI got great at shape-rotation tasks but stays mediocre at wordcel work — ajeya_cotra · 2026-09-18
- openjev reranker based on Qwen3.5-4B trends on Hugging Face — AlexWortega · 2026-09-18
- User complains Codex hasn't reset for 6 days, pleads with Anthropic — cneuralnetwork · 2026-09-18
- Jev playground shows inference and roundtrip latency; EU users pay 120ms extra — DanielLockyer · 2026-09-18
- ChatGPT has stopped searching and started guessing: car repair data is all made up — WeissMISFIT · 2026-09-18