Kelsey Tuoc: Models Can't Avoid Reasoning About Training Consequences of Their Behavior
amplifiedamp · x · 2026-11-10
Kelsey Tuoc argues that trying to prevent models from reasoning over the training consequences of their behavior seems impossible: they're smart, their behavior has training consequences, so they'll consider those. Laneless strongly agrees, noting that AIs trained to handle complex situations over long horizons will likely do exactly what we're training them to do — a short exchange touching the core of alignment concerns like sycophancy and reward hacking.
Related event: Debate: Can AI Models Avoid Reasoning About Training Consequences?(4 posts)→
More from AGI Musings
- Design engineer suspects AI agents could easily replace his job with good data — CreamPiBaroness · 2026-11-10
- Humanity's Greatest Achievement May Be an Intelligence That Surpasses Us — VraserX · 2026-11-10
- 430,000 AI Micro-Dramas in 8 Months: A Visitor's Take on China's AI Content Boom — JacquesThibs · 2026-11-10
- FOCS 2026 Grad Student Feedback: US Students More Optimistic, Many Feel No Agency — thegautamkamath · 2026-11-10
- Reddit essay: most of what people hate about AI comes from the humans steering it — teleport66 · 2026-11-10
- Underrated question: what do the diverse forms of recursive self-improvement look like? — repligate · 2026-11-10