François Fleuret asks: after a loss spike and recovery, is final loss worse, same, or better?
francoisfleuret · x · 2026-11-08
François Fleuret polls people doing serious training runs: when training hits a loss spike followed by recovery, does the final loss tend to end up (1) worse, (2) unchanged, or (3) better? A practical pretraining-stability question drawing front-line training experience in the replies.
More from Infra
- Data centers could buy community support for ~$20M/year by paying residents' power bills — beffjezos · 2026-11-08
- SpaceX plans 6 GW AI campus near Willcox, Arizona with dry cooling — aronchick · 2026-11-08
- ResearchBoost Agent speeds up real research workloads 2.51x on average across 12 tasks — YouJiacheng · 2026-11-08
- Dev builds Cray-2 emulator from scratch, runs a GPT-3-class LLM at 1 token per 4s — inductionheads · 2026-11-08
- Hyperbolic CEO on compute market-making: bridging 3-year contracts and 3-week needs — YiMaTweets · 2026-11-08
- Ben Bajarin mocks 'we don't need data centers' take: CSPs were 'useless' too — BenBajarin · 2026-11-08