Gary Marcus: The Real LLM Breakthrough Is System-Level Integration
GaryMarcus · x · 2026-08-03
Responding to the claim that there have been no fundamental breakthroughs in LLMs since the Transformer paper in 2017, Gary Marcus shared his perspective. He argues that besides scaling up old ideas, the most significant breakthrough has actually been in system architecture.
Marcus points out that the industry has broadened from isolated base models into larger systems that incorporate symbol-manipulating entities like harnesses, tools, and code interpreters. This systemic integration is what enables complex AI applications like Claude Code.
More from AGI Musings
- Turing Award Winner Predicts AI Will Replace Travel Agencies by Year-End — davidpattersonx · 2026-08-03
- AI Independently Derives General Relativity and Solves Decades-Old Math Problems — elonmusk · 2026-08-03
- Blueprint for Next Decade: Return on Cognitive Spend to Replace EBITDA — GregCook2011 · 2026-08-03
- AI Safety Advocates: No Shame in Using AI, But Superintelligence Poses Logical Risks — AdrienLE · 2026-08-03
- Stanford Number Theorist: AI is 'Cooking' Mathematics, The Singularity is Here — GregCook2011 · 2026-08-03
- Turing Award Winner: After AI Takes Our Jobs, We Will Rebuild Social Connections and Meaning — davidpattersonx · 2026-08-03