AI researcher blasts labs: a decade after "Attention Is All You Need", basics still unlearned
gerardsans · x · 2026-10-04
- Gerald Sans argues that nearly a decade after "Attention Is All You Need", many AI lab researchers never revisit fundamentals that would take only days to study — including, he claims, a shockingly common confusion between single-pass forward inference and the auto-regressive loop.
- He frames this as an industry-wide culture problem: researchers have become "operators, not inventors," following prior work without questioning or reproducing it, in a "medieval AI era where myths and hearsay thrive" until reality forces a reckoning.
More from AGI Musings
- Even in Good AGI Scenarios, "Humans as We Are" May Not Survive, Argues AI Commentator — danfaggella · 2026-10-04
- Jagged Frontier 12 Months On: AI Got Spikier, Experts Say, With No Middle Ground — connoraxiotes · 2026-10-04
- Decades of art hating knowledge work — now people are furious it's ending — Aryvyo · 2026-10-04
- Aave DAO can't own its trademark, so Aave Labs proposes a memberless Cayman foundation — LexSokolin · 2026-10-04
- When most products are $200 away from being recreated, how do startups even survive? — Aryvyo · 2026-10-04
- Engineer pushes back on the Platonic Representation Hypothesis hype — gerardsans · 2026-10-04