Toby Ord quantifies swarm scaling: 16x parallel sampling equals just 4.9x longer chain-of-thought
tobyordoxford · x · 2026-09-22
Philosopher Toby Ord used benchmark data published on OpenAI's GPT-5.6 launch page to quantitatively compare two scaling paths: widening parallel sampling (swarm scaling) versus lengthening the chain of thought.
- His earlier derivation put the swarm scaling curve at 57% as steep as the serial curves, i.e. an exponent of λ = 0.57.
- That implies scaling the swarm size by 16x yields the same performance gain as extending the chain of thought by only 16^0.57 ≈ 4.9x.
- He notes OpenAI's GPT-5.6 launch page provides three benchmark data points sufficient for this kind of fit.
The takeaway: per unit of compute, longer reasoning chains currently buy more performance than fanning out parallel samples.
More from Research
- Stanford/Tsinghua paper claims 'dopamine neurons' in LLMs, researchers push back — aran_nayebi · 2026-09-22
- Xiaomi's MiMo-V2.6 lands: 1.02T-param MoE takes top open model spot on Artificial Analysis — _AndrewZhao · 2026-09-22
- O'Reilly builds a working data vocabulary for the semantic era, from warehouses to ontologies — rseroter · 2026-09-22
- Researchers Speed Up DSA Method for Comparing Neural Dynamics, Making It Generalizable Across Domains — GretaTuckute · 2026-09-22
- Glance reads structured visual answers from a frozen 4B VLM, cutting GPU cost up to 85% — multiply_matrix · 2026-09-22
- krea2-bbox-turbo Full-Rank Finetune Hits Epoch 14, Release Planned at Epoch 20 — Amazing_Painter_7692 · 2026-09-22