Researcher: any agent acting over long horizons provably has a self-model and world model
chris_j_paxton · x · 2026-09-16
In reply to Chris Paxton, researcher Aran Nayebi sketches a theorem-style argument: an agent that can act over long horizons can be proven to internally model an action-conditioned world model — meaning it models the consequences of its own actions and therefore must have some model of itself.
This offers a formalizable angle on the debate over whether LLM-based agents have internal world models or self-representations: a self-model doesn't imply consciousness, but long-horizon agency mathematically entails an internal representation of one's own action consequences. Link to paper details included.
More from AGI Musings
- OpenAI's roon predicts everyone will have Astra-level capabilities within a month or two — Tolopono · 2026-09-16
- Training Data Is Regulation by Another Name: Easy to Add, Nearly Impossible to Unwind — yunta_tsai · 2026-09-16
- OpenAI's Mark Chen: There Are No Race Dynamics Off the Frontier — deanwball · 2026-09-16
- Gary Marcus on BBC: Altman, Huang and Sanders posture at extremes while honest AI safety talk is missing — GaryMarcus · 2026-09-16
- New Paper Maps Roadmap Toward AI Recursive Self-Improvement Across Domains — Yi Duan · 2026-09-16
- Frontier coding agents now write code no human can read, warns tszzl — tszzl · 2026-09-16