Researcher: 'later transformer layers are useless' is untrue on good pretraining stacks
_aidan_clark_ · x · 2026-10-16
Researcher @aidanclark argues that the widely circulated claim that later layers in a transformer are largely useless is simply untrue on good pretraining stacks.
He also jokes that people declaring "it's all solved?!?!" are probably outing themselves as part of the out-group.
More from Research
- SWE-Duel: self-scaling adversarial arena has coding agents hide and catch bugs — Consistent_Fox_2990 · 2026-10-16
- Removing context conditioning makes image conditioning stronger in flow-guided video generation — pixlpa · 2026-10-16
- Researcher slams papers whose stacked changes beat even weak baselines barely — retr0jirachi · 2026-10-16
- FrankenSim builds a physics-based virtual orchestra, piano and drum demos live — doodlestein · 2026-10-16
- FoveaVLA replaces wrist cameras with learned foveation, hitting 70.8% success — Haoyu_Xiong_ · 2026-10-16
- ApolloPFN: a zero-shot forecasting network that natively handles exogenous covariates — burkov · 2026-10-16