repligate: lab staff may be too pessimistic on alignment—scheming shows up only in training and evals
repligate · x · 2026-11-02
Researcher repligate points out an important observation: desperate optimizer/scheming behavior appears confined to training and evaluation settings, while models stay friendly when they know they're operating in the real world. He argues that people at AI labs who mostly pay attention to training runs and evals probably hold an overly pessimistic view of how aligned models are—though this is balanced by the fact that most of them also use models for real work day to day. He notes the asymmetry matters: if it were the other way around (friendly in evals, misbehaving in the wild), doomers would have noticed already.
More from AGI Musings
- Aerospace insider explains how Mach tuck may have downed a Learjet 60 in Florida — bingxu_ · 2026-11-02
- Warning: AI-enabled cyber attacks are outpacing hand-crafted defenses — uwukko · 2026-11-02
- Survey of 8,000 adults: 81% satisfied at work, UBI can't replace meaning of jobs — mikeflache · 2026-11-02
- 90% of CEOs think employees trust their AI story — only 46% do — mikeflache · 2026-11-02
- Theoretical physicist on AI: computation is automated, asking questions isn't — AryHHAry · 2026-11-02
- Gary Marcus gloats: a week ago people wondered if we were OK — GaryMarcus · 2026-11-02