Ex-Google Brain: key algorithmic wins came under 1e20 FLOPs, not scale
_arohan_ · x · 2026-09-07
The author argues that while compute access matters, good ideas matter more. Many innovations from the Google Brain era that became "algorithmic progress" in frontier models were discovered with less than 1e20 FLOPs and only later scaled to 1e25. From this he concludes automated research is still nascent — largely a "local minima finding machine."
More from AGI Musings
- Apollo chief economist: AI has delivered zero financial payoff outside Big Tech — rseroter · 2026-10-27
- OpenAI acquires Orchestra Bio to bring clinical-trial monitoring agents into drug development — dl_weekly · 2026-10-27
- Do AI agents work better when they pace themselves and take breaks? — EpistemicHope · 2026-10-27
- Experts Clash: Today's Models Dwarf GPT-4, But Journal Delays Lock Out New LLMs — DeryaTR_ · 2026-10-27
- Why would breaks help a transformer? Questioning the AI pacing hypothesis — EpistemicHope · 2026-10-27
- Venkatesh Rao essay: On Learning to Ride Your Brain — jger15 · 2026-10-27