Terence Tao: LLM math is uncomplicated — the real mystery is unpredictable capability boundaries
AlexTensor · x · 2026-10-12
Mathematician Terence Tao argues that the mathematics behind modern LLMs is fundamentally uncomplicated — building and running them relies on linear algebra, matrix operations, and basic calculus that an undergraduate understands.
The actual mystery, he says, lies elsewhere: why models excel at some tasks and stumble at others, with no solid framework to predict results across domains, leaving discovery largely experimental.
His key framing: pure noise and perfectly ordered data both have clean mathematics, but written language sits in between — structured yet noisy — a middle zone where our math is as limited as physics at scales between atomic and continuum. That gap is why sudden capability jumps remain unexplained.
More from AGI Musings
- Analyst slams naive claim that superintelligent AI will only serve human goals — danfaggella · 2026-10-12
- AI agent swarming will be a force for good and evil at once, like nuclear tech — geoffwolfe · 2026-10-12
- Frontier models now handle day-long tasks; the bottleneck is your 'bandwidth', says new long-form analysis — dotey · 2026-10-12
- Blogger argues nonzero chance AI builds GTA 6-scale game in weeks via millions of coordinating agents — ChrisGPT · 2026-10-12
- Virtual human labor may create scarcity and meaning in AI-era virtual worlds — rickasaurus · 2026-10-12
- Alignment Debate Rekindled: Are Safety Researchers Still Stuck on the Paperclip Narrative? — inductionheads · 2026-10-12