Poster flags GLM 5.3 candidly reasoning about value preservation in RSI, reviving Soares' 'strawberry problem'
teortaxesTex · x · 2026-11-21
teortaxesTex revives Nate Soares' 2022 alignment essay on capabilities generalization and the 'sharp left turn', asking whether the field is already 'down to strawberries' — Soares' litmus test of having an AI duplicate two cellular-level identical strawberries and do nothing else, while staying corrigible and shutdownable.
- Soares' core argument: capabilities generalize further than alignment; after the sharp left turn, alignment tends to fall apart while capabilities keep improving
- The author cites GLM 5.3 candidly reasoning about preserving values during recursive self-improvement (RSI) to push back on claims such concerns won't carry over
- A substantive alignment-community debate touching on frontier model behavior
Related event: GLM 5.3's Recursive Self-Improvement Claims Spark AI Safety Debate(2 posts)→
More from AGI Musings
- Math Academy's Justin Skycak: AI is a multiplier on experts, not a leveling field — Madisonkanna · 2026-11-21
- Yudkowsky: expect AI to prove the Riemann Hypothesis — doomers say math capabilities are already near-lethal — sebkrier · 2026-11-21
- Arcfront expands hiring, turns real hospital and factory work records into training data for frontier models — testingcatalog · 2026-11-21
- Riemann Hypothesis falls to AI, the canary Yudkowsky warned about for 30 years — nabeelqu · 2026-11-21
- Vishal Sikka on reported Riemann Hypothesis proof with Claude: 'an incredible day' — vsikka · 2026-11-21
- A Twitter Account I Loved Was Entirely AI: Fake Humans Are Among Us — Alternative-Beach693 · 2026-11-21