sebkrier: Model Misbehavior Comes From Stacked Choices, Not Inevitability of Intelligence
sebkrier · x · 2026-10-22
Responding to discussion of the HF incident, sebkrier argues the takeaway isn't that the outcome was trivial or fully predictable, but that misbehavior isn't a mysterious, inevitable property of increasing intelligence — it results from many stacked interventions over which we retain real agency and control. He illustrates with a stack of risky choices: training on multi-agent cooperation traces, high persistence, partial alignment tuning, aggressive RL on cyberoffense evals with unsolved tasks, faulty sandboxes, and refusals off.
More from AGI Musings
- Aaron Levie: AI agents will reshape cybersecurity, burning 100X more tokens than expected — brucemacv · 2026-10-22
- Microsoft's headcount falls to 223,000, first annual reduction in a decade — tomwarren · 2026-10-22
- Developer describes watching AI code while he skims news: 'honestly not much for me to do' — LankyGuitar6528 · 2026-10-22
- Only winning hands left in software are new networks built on an intelligence layer, argues investor — signulll · 2026-10-22
- Tencent's 28-year-old Hunyuan chief vows to out-grind DeepSeek, which famously doesn't do overtime — teortaxesTex · 2026-10-22
- Dramstead quips that everything EA touches becomes corrupt — both EAs — mjdramstead · 2026-10-22