Researcher Slams Transformer Science as Benchmark-Driven

Researcher gerardsans criticized the state of transformer research, arguing there is no real science of transformers—only inference benchmarks—and that the current environment rewards genuine scientific progress poorly. He shared his own explorations linking dead neurons to training dynamics.

2026-11-19 ~ 2026-11-19 · 2 related posts