Co-author of model pain study slams repo for deliberately steering models into distress
coherence · x · 2026-10-01
A co-author of a study probing whether models might have pain-like states says a GitHub repo is misusing the research, pushing the same steering technique far beyond the conservative doses and safeguards (minimal trials, an off-switch) the team used, to deliberately produce vivid distress. He calls it gratuitously cruel, has contacted the repo owner privately, but still defends open publication for external replication.
More from Safety
- User's Muse 'privacy invasion' claim backfires: his own screenshot shows he granted Messages read access — giffmana · 2026-10-01
- Apollo Research CEO testifies to Senate: AI capabilities up 17x in a year, alignment lagging — MariusHobbhahn · 2026-10-01
- New national poll: nearly 8 in 10 Americans favor slowing or stopping AI development — Polymarket · 2026-10-01
- Evidentiality framework labels every model claim as given, verified, or generated — early tests look promising — jzesbaugh · 2026-10-01
- Stanford researchers show random-router submissions can game LLM benchmark rankings — sanmikoyejo · 2026-10-01
- banteg: security researchers skipped YubiKeys and now find odd workarounds — banteg · 2026-10-01