Peking University finds models "know" answers but can't say them: the Readout Bottleneck
jiqizhixin · x · 2026-09-26
A study from Peking University and YIXIN AI Lab (accepted to EMNLP 2026 main conference, 15.4% acceptance rate) asks: when a model fails a logic problem, does it truly fail to reason, or does it already know the answer but fail to express it?
Opening the black box, the researchers find the correct answer is already encoded in the model's hidden states, but a systematic bias in the output layer "covers it up" when those states are converted into candidate scores. They name this phenomenon the Readout Bottleneck.
Mechanistically, even when the model's first-label-token score still holds a slight advantage, accumulating log-probabilities across the full candidate string amplifies the error — accuracy collapses to 0.333, i.e. pure three-way guessing. The effect persists on an instruction-tuning-free base model: probing recovers 87.3% on Qwen3.5-9B-Base, while sequence scoring drops sharply.
More from Research
- ICLR 2027 Submission Deadline Announced — CSProfKGD · 2026-09-26
- The Pre-Training / Post-Training Divide Doesn't Exist in Biology — JosephJacks_ · 2026-09-26
- Silent mistakes that can hurt your Numerai models, per new community writeup — matlabulous · 2026-09-26
- Zenke Lab to present spiking network and surrogate gradient work at Bernstein Conference — hisspikeness · 2026-09-26
- Michael Levin: intelligence doesn't require genes, cognition is widespread in life — drmichaellevin · 2026-09-26
- Position paper argues grounded abduction is the missing reasoning leap for LLMs — bibryam · 2026-09-26