Ex-Google Brain researcher: key algorithmic wins were found under 1e20 FLOPs, then scaled to 1e25
_arohan_ · x · 2026-09-07
Former Google Brain member arohan notes that many innovations from his Google Brain era that made it into frontier models — now credited as 'algorithmic progress' — were discovered with less than 1e20 FLOPs of compute and only later scaled to 1e25 FLOPs. The observation implies major algorithmic breakthroughs don't require massive compute; small-scale experiments can yield frontier-shaping methods.
More from Infra
- DEX-Comp Compresses RAG Context 16x While Matching or Beating Uncompressed Baselines — _reachsumit · 2026-09-07
- Embedding Surgery: query-time localized vector edits fix dense retrieval rankings, +60% nDCG@10 — _reachsumit · 2026-09-07
- VDN-H3 ported to Vpipe hits ~2.6x faster H3 generation on M5 Pro 24GB — TgoAI · 2026-09-07
- SC Asia 2027 opens call for papers on supercomputing and AI infra, due Oct 7, 2026 — thoefler · 2026-09-07
- Hands-on test: AI ops agent Ghost diagnoses and fixes three simultaneous service failures — BLUECOW009 · 2026-09-07
- Guide: Run DLSS 5 Real-Time Video Playback on RTX 20/30 Series via MPV — Miserable-Option5488 · 2026-09-07