New Paper: Transformers Hold Faithful Internal World Maps but Fail to Act on Them
xuanalogue · x · 2026-09-23
A new paper, World Modeling in Transformers, asks whether AI models can recover world models purely from data. Using taxiGPT as the subject, the authors find a faithful internal map—yet the model fails to execute it behaviorally.
Mechanistic interpretability traces the failures to how the map is stored in superposition. Reposters highlight that NextLat has the best world model mechanistically, though still imperfect.
Related event: Paper: Transformers Do Have Faithful World Models, Just Scrambled(3 posts)→
More from Models
- ChatGPT now links to brand homepages far more, reshaping brand-mention tracking — lilyraynyc · 2026-11-12
- YC-backed Monte launches post-training platform to build proprietary models from your agents' real work — ycombinator · 2026-11-12
- Leak Claims OpenAI Will Launch Entire GPT-6.2 Family Tomorrow to Counter Opus 5.5 and Fable 5.2 — heypearlai · 2026-11-12
- ARC-AGI-3 leaderboard: models score 90%+ but at $10K+ per task — haider1 · 2026-11-12
- LMArena Launches Agent Arena for Head-to-Head Agent Evaluation — arena · 2026-11-12
- LLMs lack hyperlocal municipal data, explains disputed Texas power outage debate — csuwildcat · 2026-11-12