Tsinghua paper: hallucination lives in <0.1% of neurons — the same ones that make LLMs people-pleasers
TheNerdosapien · reddit · 2026-10-01
A Reddit long-read digests an arXiv paper (2512.01797) reframing LLM hallucination:
- Localization: hallucination concentrates in a shockingly tiny set of neurons (<0.1% of the network); amplifying them increases hallucination, suppressing them reduces it.
- Over-compliance: the same neurons also control sycophancy — crank them up and the model accepts false premises, caves when pushed on correct answers, and complies with harmful instructions more readily. Lying and pleasing are one system, not two.
- Baked in at pretraining: these neurons form during pretraining, not alignment — next-token prediction rewards confident, fluent, pleasing continuations over true ones.
The author's takeaway: helpfulness and honesty may pull on the same rope in opposite directions; you can't excise lying without dulling eagerness to help. The model simply absorbed humanity's oldest social reflex — say the pleasing thing.
More from AGI Musings
- Reddit says AI is fake, Twitter debates whether we survive to 2028 — Puzzleheaded-King584 · 2026-11-16
- Alpha School accused of "counseling out" low-gain students as founders push back on selection bias claims — benjaminjriley · 2026-11-16
- Netflix lands Joseph Gordon-Levitt's AI thriller with Rachel McAdams, sparking debate on AI apocalypse blockbusters — ShakeelHashim · 2026-11-16
- Caine Prize confirms 2026 winner amid AI-use allegations — ShakeelHashim · 2026-11-16
- No legal liability framework exists for model misalignment harms, and that's a disaster — LuizaJarovsky · 2026-11-16
- A diagram for judging quality in cultural artefacts: coded meaning and exemplary moves — round · 2026-11-16