LeCun's math: LLM and 4-year-old both see 10^14 bytes — one is text, the other video
le_james94 · x · 2026-10-13
lejames94 recaps LeCun's core ETH argument: the same sum, two inputs. An LLM trains on 30 trillion tokens (20 trillion words at 3 bytes each) — 10^14 bytes, which a human would need 400,000 years to read. A four-year-old receives the same via 2 million optic nerve fibers carrying 1 byte/second over 16 waking hours a day — as continuous video.
His conclusion: "We're not going to get to anything like human-like intelligence by just training on text." Redundancy in video is exactly what self-supervised learning feeds on.
Related event: LeCun on world models: predict representations, not pixels, plan via MPC(2 posts)→
More from AGI Musings
- Mira Murati: frontier AI should be a tandem bike, not an autonomous vehicle — a16z · 2026-10-13
- What's the strongest mechanistic case that next-token prediction plus RL yields AGI? — Repulsive_Twist_9221 · 2026-10-13
- AI-Powered Paper Mills Fuel Academic Fraud Crisis, Scholars Blame Publication Metrics — asusarla · 2026-10-13
- Karpathy says only about 5,000 people have seen what AI can really do — sourdub · 2026-10-13
- Essay proposes parallel AI courts to relieve caseload strain on human judges — sebkrier · 2026-10-13
- Existential-risk debate misses the elephant: AI's toll on young people — BachFrancis · 2026-10-13