Gary Marcus: The Real LLM Breakthrough Is System-Level Integration
GaryMarcus · x · 2026-08-03
Responding to the claim that there have been no fundamental breakthroughs in LLMs since the Transformer paper in 2017, Gary Marcus shared his perspective. He argues that besides scaling up old ideas, the most significant breakthrough has actually been in system architecture.
Marcus points out that the industry has broadened from isolated base models into larger systems that incorporate symbol-manipulating entities like harnesses, tools, and code interpreters. This systemic integration is what enables complex AI applications like Claude Code.
Related event: Gary Marcus and Developers Clash Over LLMs vs. Neurosymbolic AI(19 posts)→
More from AGI Musings
- A missing parameter in scaling laws: H, the human effort behind learning signals — lateinteraction · 2026-10-16
- First-ever paper on multi-agent swarm safety lands NeurIPS spotlight — LauraRuis · 2026-10-16
- Yoav Artzi on peer review: paper flooding still pays off as review system strains under load — yoavartzi · 2026-10-16
- Letta founder: work in fields defined by problems, not methods — lateinteraction · 2026-10-16
- Why pure mathematicians will soon flood into AI x-risk debates — Dr_Atoosa · 2026-10-16
- RSI began when model outputs augmented next-gen training data, agents just escalated it — akbirthko · 2026-10-16