Stanford team forecasts GSM8K accuracy from zero-score models using hinted joint scaling laws
stanfordnlp · x · 2026-11-05
A Stanford team led by Percy Liang tackles the problem that scaling laws can't be extrapolated when small models score zero on a benchmark: "you can't extrapolate from a bunch of zeros."
Their key idea is to turn a benchmark into a series of easier "hinted" versions and fit a joint scaling law Accuracy(scale, hint). Since more hinting makes the task easier in a controlled way, even small models score non-zero, enabling extrapolation to the no-hint case. Using this approach, they forecast GSM8K accuracy at full scale (H=0) within 3 points using 5600x less compute.
Related event: Stanford's Hinted Scaling Laws Predict GSM8K Accuracy with 1/5600 Compute(8 posts)→
More from Research
- MiniMMD PhD thesis on arXiv: minimizing MMD from theory to practice — LesterMackey · 2026-11-05
- AAAI Rebuttals Turn Brusque: Bare Data Dumps Replace Courteous Responses This Year — Warm_Ad4302 · 2026-11-05
- Data Analysis Finds Top-Percentile Lawyers' Clients Face 1.6x Higher SEC/DOJ Scrutiny — doodlestein · 2026-11-05
- CUHK and Microsoft's EchoChat hits 8.64 on EchoEval for empathetic speech dialogue — jiqizhixin · 2026-11-05
- AgentVQA From Princeton Offers Offline Agentic Visual Benchmark to Cut Costly Online Evals — Key_Recording9766 · 2026-11-05
- GameVerse releases minute-scale gameplay dataset for long-horizon interactive world modeling — AlayaLab · 2026-11-05