Karpathy: LLMs Lack Cultural Accumulation and Self-Play
i_dg23 · x · 2026-08-01
Andrej Karpathy highlighted that current Large Language Models (LLMs) are missing two critical elements that propelled human intelligence: cultural accumulation and self-play.
- Cultural Accumulation: He questioned why an LLM can't write a book for other LLMs to read, be inspired by, or be shocked by.
- Self-play: Noting AlphaGo's success with self-play, he asked why an LLM can't generate increasingly difficult problems for another LLM to solve and learn from.
Researcher Noah added that self-play works well in chess, Go, and poker because they are two-player zero-sum games, which simplifies the problem. The real world is much messier, which is why we haven't seen many self-play successes in LLMs yet.
Related event: Karpathy: LLMs Lack Cultural Accumulation and Self-Play(2 posts)→
More from AGI Musings
- Report: OpenAI is building Astra, a multi-agent model family for tasks lasting hours or days — The Decoder · 2026-08-01
- V4-Pro Improvement Debate: Stronger Base vs. Saturated Evals — teortaxesTex · 2026-08-01
- Rich Sutton Questions Whether LLMs Are the Bitter Lesson, Suggesting They May Be Superseded — teropa · 2026-08-01
- Vibe Coding Could Cut Costs: AI Reduces Need for High-Paid Engineers — georgemillo · 2026-08-01
- Terence Tao on Careers in the AI Era: Humans Will Outlive Their Expertise — r0ck3t23 · 2026-08-01
- Chris Manning's COLM Keynote: History of LLMs, Intelligence, and Agents — stanfordnlp · 2026-08-01