Ex-Google Brain: key algorithmic wins came under 1e20 FLOPs, not scale
_arohan_ · x · 2026-09-07
The author argues that while compute access matters, good ideas matter more. Many innovations from the Google Brain era that became "algorithmic progress" in frontier models were discovered with less than 1e20 FLOPs and only later scaled to 1e25. From this he concludes automated research is still nascent — largely a "local minima finding machine."
More from AGI Musings
- Reddit debate: should expert corrections to AI outputs count as paid work? — gareth789 · 2026-09-07
- LLMs aren't creating a new intellectual program — they're reviving symbolic AI's old questions — Amichayg · 2026-09-07
- MIT's 13-lecture Society of Mind course by Marvin Minsky is free online — TheMoonMidas · 2026-09-07
- Delip Rao: scaling was necessary but not sufficient, OpenAI got lucky — deliprao · 2026-09-07
- Analyst speculates OpenAI's Astra is a World Model 3.0 after abandoning Sora — teortaxesTex · 2026-09-07
- Journal argues academia structurally selects against creativity — a decades-old consensus — davidmanheim · 2026-09-07