Stanford's 'relative scaling laws' show LLM scale isn't a universal equalizer
Diyi_Yang · x · 2026-10-07
A COLM 2026 poster by William B. Held, Percy Liang and Diyi Yang introduces relative scaling laws, tracking how performance gaps between test distributions evolve with scale instead of aggregate error. Training 255 decoder-only Transformers under matched-compute IsoFLOP budgets (3×10^18–3×10^20 FLOPs), they find MMLU academic domains converge toward parity, regional English dialects shift with population size, and AI-risk behavior clusters split: capability- and influence-related risks rise during pretraining while adversarial risks don't. Conclusion: scaling improves overall performance but is not a universal equalizer. All checkpoints are released.
More from Research
- OpenAI proves matrix multiplication solvable in O(n^2.25) operations — but no algorithm yet — Pascallisch · 2026-10-07
- Melanie Mitchell fires back: cites recent LLM research, two NeurIPS papers — MelMitchell1 · 2026-10-07
- Integer multiplication faster than N log N? Algorithm fans call it "cursed" — QuintinPope5 · 2026-10-07
- OpenAI Researcher Surprised AI Lab Math Results So Far All Hold Up — willdepue · 2026-10-07
- Frontier LLMs as simulators of human biologists will land faster than 'virtual cells' — CatAstro_Piyush · 2026-10-07
- AI claims progress on 90 of 500 major open math problems, including partial Riemann Hypothesis results — DavidSKrueger · 2026-10-07