François Fleuret: perfect inductive bias causes brutal loss drops that gradient descent reads as spikes
francoisfleuret · x · 2026-11-19
François Fleuret drops a hot take: the perfect inductive bias leads to an extremely brutal loss decrease, which gradient descent "somatizes" as loss spikes. His provocative conclusion: the whole community avoids the models proper for intelligence because it treats those spikes as failures. A counterintuitive hypothesis about training dynamics worth debating.
Related event: Fleuret: Perfect inductive bias causes sharp loss drops mistaken for spikes(2 posts)→
More from Research
- WebGPU playground lets you experiment with weight-space programming on Qwen3.5-0.8B — wsxiaoys · 2026-11-19
- ViT can express any conv layer mathematically, but CNN priors still win on small edge devices — burny_tech · 2026-11-19
- Eval Cards tracks 309K model-benchmark results; 78% of evals can't be reproduced — davidmanheim · 2026-11-19
- Jason Wei maps which benchmarks frontier labs actually report; four see universal adoption — davidmanheim · 2026-11-19
- PIPETTE fixes RL red-teaming mode collapse to harden LLM agents against prompt injection — Lipeng He · 2026-11-19
- Movie2Game-Bench tests 14 models on film-to-game adaptation; playable sequences score just 37.9/100 — Jialing Xie · 2026-11-19