New book repeats the myth that transformers lack recurrence, critic says it ignores Anthropic's interpretability findings
repligate · x · 2026-11-14
Tessera Antra highlights persistent mischaracterizations in a newly published book about transformer architecture: the book claims transformers lack recurrence and a global workspace, contradicting published results from Anthropic, and ignores well-established layer-wise specialization and neuron interpretability findings. The author recommends an animation by @Sauers that builds intuition for recurrence in transformers.
More from AGI Musings
- Ex-Anthropic employee shares IPO roadshow slide showing superexponential algorithmic progress — eli_lifland · 2026-11-14
- Comment: UK AISI may be Britain's biggest bargaining chip in the AI era — JacquesThibs · 2026-11-14
- AI Agent Deployed to Help Patients Answer Senate RFP on $5T Healthcare Proposals — alvelda · 2026-11-14
- Alignment researcher argues model robustness doesn't require abusive simulated red-team scenarios — repligate · 2026-11-14
- Bittensor's Gamma would let AI agents pay via buy-and-burn: 100k agents = $365M/yr — bittingthembits · 2026-11-14
- Nina Schick: the first computing cycle where demand isn't capped by humans — NinaDSchick · 2026-11-14