Terence Tao: LLM math is simple undergrad stuff — the real mystery is why they work
rohanpaul_ai · x · 2026-09-14
Terence Tao argues the math behind today's LLMs is simple — linear algebra, matrix multiplication, and a bit of calculus an undergraduate can handle. The real mystery is why they excel on some tasks and fail on others, with no reliable rules for predicting performance. He traces the gap to natural text sitting between pure noise and fully structured data, a middle regime where mathematics is as thin as physics at meso-scales, leaving capability jumps empirical rather than explainable.
More from AGI Musings
- We Must Pace the Frontier: A Case for Slowing Down Frontier AI — gaganghotra_ · 2026-09-14
- iamtrask: Follow the Incentives and AI Twitter Makes Sense — iamtrask · 2026-09-14
- Dario Amodei's new essay calls to pace the frontier; Anthropic opens systems to third-party evaluators — ZeroStateReflex · 2026-09-14
- RL luminary Szepesvári: researchers actually want AI models that generate insight — CsabaSzepesvari · 2026-09-14
- Pedro Domingos: swap "AI" for "interpolation" and doomer screeds fall apart — pmddomingos · 2026-09-14
- Commentary: frontier AI leaders don't trust each other, so they ask government to chaperone — bilawalsidhu · 2026-09-14