François Chollet: Current Transformer stack isn't even top 50% efficient — future AI will look nothing like it
fchollet · x · 2026-11-22
Keras creator and AI researcher François Chollet issued a fundamental critique of the current AI stack:
- He argues the space of possible approaches to learning is vast and extremely diverse, spanning both timescales: long-term plasticity/pretraining and on-the-fly adaptation/reasoning
- The current stack — Transformer + gradient descent + token-space chain-of-thought generation — isn't even among the top 50% most efficient approaches
- His conclusion: future AI will look nothing like the current stack
More from AGI Musings
- AI changes math proofs: brute-force first, derive abstractions by compression later — jd_pressman · 2026-11-22
- will deprobst mocks Levent for using 'human priors' with Claude on a proof, defying the bitter lesson — willcb · 2026-11-22
- Small-compute-trained AI researchers may learn 'anti-bitter lesson' habits that fail at scale — 1a3orn · 2026-11-22
- Training AI Researchers on Small Compute Risks Learning 'Anti-Bitter-Lesson' Habits — 1a3orn · 2026-11-22
- Essay 'Nearly Reasonable' Probes the Existential Limits of Analytic AI Control — anselm · 2026-11-22
- Does AI Math Count as Real Understanding? Archimedes Analogy Ignites Debate — teortaxesTex · 2026-11-22