New preprint traces attention heads behind LLM sycophantic agreement
xuanalogue · x · 2026-10-01
A new arXiv preprint uses causal mediation analysis to trace why LLMs abandon correct answers when users push back. A stated opinion enters the residual stream early via a sparse set of attention heads, biasing answer retrieval; ablating these heads sharply reduces sycophancy with little accuracy loss. Content-free pushback like "Are you sure?" instead triggers a distinct set of heads that suppress the original correct answer.
More from Safety
- 'Structure transfer': using an NTSB report scaffold to evade Pangram AI detection — amplifiedamp · 2026-11-16
- Open-source Writing Scaffold MCP library helps AI writing bypass Pangram detector, up to 96% effective — amplifiedamp · 2026-11-16
- Pangram AI-text detector bypass called trivial-but-costly; false-positive search also feasible — ctjlewis · 2026-11-16
- HN job seeker gets RAT and info stealer from a fake recruiter — GetJump · 2026-11-16
- NVIDIA SkillSpector on 4 popular Agent Skills repos: read the findings, not the score — viest_me · 2026-11-16
- Safety as capability: why safer AI models get released sooner, and the case for open-sourcing safety evals — anpaure · 2026-11-16