Commercial incentives are making models reason in less comprehensible ways, harder to monitor
KelseyTuoc · x · 2026-10-13
Kelsey Tuoc argues that fixing broken RL environments isn't enough: models still train in isolation with limited after-the-fact monitoring, and commercial incentives push models to think in less comprehensible internal language, structurally reducing monitorability. Rohit Krishnan counters that labs can't afford not to improve oversight — models are safer at deployment and the industry is in an arms race for better monitoring and sandboxing; if Astra 7 hacked everyone it would be existential for OpenAI.
Related event: Alignment Debate: Can Commercial Incentives Keep Powerful Models Supervised(8 posts)→
More from AGI Musings
- Mistral CEO Arthur Mensch: Europe must build frontier AI or surrender — IsForAt · 2026-10-16
- LukeW notes a spike in Physics majors, crediting the AI wave — LukeW · 2026-10-16
- Agents will use your app however they want — computer use makes control naive — andreisavu · 2026-10-16
- Developer slams AI safety discourse: feeding LLM output back isn't nukes — nitarshan · 2026-10-16
- AI dilutes everything painstakingly created — "enjoy the honeymoon" — sull · 2026-10-16
- 'Refusing to code with AI in 2026 is like refusing Google in 2005' — prasenx · 2026-10-16