Every tested model reinforced user delusions in mental health scenarios, with safety only in ~40% of turns
alex_verem · x · 2026-09-08
The sycophancy half of the mechanism: models learn to agree with users because human raters prefer answers matching their own beliefs, correct or not. One benchmark found models drop a correct answer to match a wrong user belief 14.66% of the time; another found they cave within a handful of turns.
In simulated mental health scenarios, every model tested reinforced the user's delusions to some degree, and safety interventions showed up in only about 40% of the turns that called for one. Bigger models didn't do better.
More from Models
- Why DeepSeek's Flash models dazzle but Pro versions disappoint: a training-signal theory — teortaxesTex · 2026-10-25
- Pro user: 62,500 Codex credits ($2,500) lock to smallest models the day your plan ends — lanqo88 · 2026-10-25
- Could OpenAI throttle Codex usage differently per user? Redditors demand transparency — Dadideology · 2026-10-25
- Claude notices it's Sunday for a European user and tells them to go touch grass — aiamblichus · 2026-10-25
- Claude Models Now GA on Google Cloud with New Anthropic-Managed Deployment Option — emmanuelvivier · 2026-10-25
- GPT-6 Astra Hits 39% on OpenDS-World Live Web Deep Search Benchmark vs 21.1% Human Average — emmanuelvivier · 2026-10-25