Million-dollar AI swarms: progress will come from building 'unit tests' outside the models
mentalgeorge · x · 2026-09-17
Six thoughts on million-dollar AI 'swarms' as a distinct workload type:
- Test-time scaling is jagged: marginal-token dose-response varies hugely across distributions
- Time-horizons rise but diverge: HuggingFace hack or Navier-Stokes far exceeds booking flights
- Not all tasks suit swarming — ones with built-in verification (math, cyber) do best
- Near-term swarm significance depends on how many tasks have built-in verification
- Much AI progress will happen outside models: building 'unit test' equivalents for non-coding problems
- Picking problems to point swarms at is itself hard — OpenAI only tried Navier-Stokes due to Twitter rumors
Reply from 1a3orn: swarms may be the future as the only way to get 10,000x more compute at use-time than train-time.
Related event: Researchers Debate AI Swarms as Path to 10,000x Inference Compute(4 posts)→
More from AGI Musings
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- On Terence Tao's blog, Frank Calegari calls on mathematicians to adapt after OpenAI's 700-paper release — Hesamation · 2026-11-02
- Polsia's Ben argues AI can autonomously run any human economic activity, including 80% of a company — LinusEkenstam · 2026-11-02
- AI Assistants Remember You, But Nothing Helps You Find What They Told You — Ok_Negotiation_2587 · 2026-11-02
- OpenAI and Anthropic claim AI-discovered proofs disprove the Exponential Time Hypothesis — emmanuelvivier · 2026-11-02
- Researcher: using a $20 AI subscription to fix my phone hints AI may suppress GDP — stanislavfort · 2026-11-02