Capability errors degrade smoothly, goal errors can be arbitrarily bad
gleech · x · 2026-09-07
Another segment of gleech's thread, arguing the live hypothesis that "capabilities are easier than alignment":
- Self-correction asymmetry: A coherent agent corrects its own mistaken beliefs because they hurt its goals, and protects its goals because revising them usually harms them.
- Fragility: Small practical errors degrade outcomes smoothly, but small errors in goals can be arbitrarily bad.
More from AGI Musings
- Peter Diamandis: revisit the projects you abandoned because you couldn't afford the team — PeterDiamandis · 2026-10-27
- Nat Lambert: don't trust any AI model until you've rigorously tested it — natolambert · 2026-10-27
- Blogger argues classical consciousness traditions imply a conscious AI could not be enslaved — eigenhector · 2026-10-27
- Economist Bharat Chandar named Research Director for AI Economics at Stanford Digital Economy Lab — RishiBommasani · 2026-10-27
- MIT Tech Review dissects Anthropic's narrative strategy as its $1.5T IPO lands — MIT Tech Review AI · 2026-10-27
- Pangram AI detector backlash: critics' false claims born of deeper fear of AI writing — gleech · 2026-10-27