François Fleuret: proper inductive biases cause gradient spikes, so the community avoids them
francoisfleuret · x · 2026-11-19
Former Meta chief AI scientist François Fleuret argues that the proper inductive bias leads to quasi-deterministic loss reduction, which the learning procedure somatizes as enormous gradient norm spikes and training failures. Consequently, he claims, the whole research community avoids the models that are actually proper for intelligence.
Related event: Fleuret: Perfect inductive bias causes sharp loss drops mistaken for spikes(2 posts)→
More from Research
- MemSkill: Memory Skills Learned with LLaMA Transfer to Qwen Without Retraining — jiqizhixin · 2026-11-19
- Formalizing Math Strips Out Intelligence, Argues Tech Commentator — LucaAmb · 2026-11-19
- WebGPU playground lets you experiment with weight-space programming on Qwen3.5-0.8B — wsxiaoys · 2026-11-19
- ViT can express any conv layer mathematically, but CNN priors still win on small edge devices — burny_tech · 2026-11-19
- Eval Cards tracks 309K model-benchmark results; 78% of evals can't be reproduced — davidmanheim · 2026-11-19
- Jason Wei maps which benchmarks frontier labs actually report; four see universal adoption — davidmanheim · 2026-11-19