AI researcher blasts labs: a decade after "Attention Is All You Need", basics still unlearned
gerardsans · x · 2026-10-04
- Gerald Sans argues that nearly a decade after "Attention Is All You Need", many AI lab researchers never revisit fundamentals that would take only days to study — including, he claims, a shockingly common confusion between single-pass forward inference and the auto-regressive loop.
- He frames this as an industry-wide culture problem: researchers have become "operators, not inventors," following prior work without questioning or reproducing it, in a "medieval AI era where myths and hearsay thrive" until reality forces a reckoning.
More from AGI Musings
- AI-powered forensic metascience: auditing 7 million papers, 500K files scanned, 100+ issues found — gleech · 2026-11-20
- Long Now Labs' first awardees include a thermodynamic clock for LLMs, $25k each — begusgasper · 2026-11-20
- Long Now launches Epistemic Cycles Lab led by Dr. Nina Beguš to map information ruptures — begusgasper · 2026-11-20
- Talia Ringer starts an email thread on open-model advocacy in a 'post-Jensen' world — dinabass · 2026-11-20
- Andriy Burkov's True Positive Weekly #186: what to know when agents do the work — burkov · 2026-11-20
- Brand drift in the agent era: give agents the reasoning behind each rule — round · 2026-11-20