Scaling laws emerge at 4M params—on the fully tuned frontier
karen_ullrich · x · 2026-10-29
Thread sub-post with key figure: scaling laws emerge even for 4M-parameter models, but only on the fully tuned frontier. The chart shows validation loss vs compute for models from 4M to 268M—most runs sit far above the dashed tuned frontier, showing how hard that frontier is to find at small scales.
Related event: Meta/NYU study shows scaling laws hold down to 4M-parameter models(5 posts)→
More from Research
- Ben Recht Debunks 'Silicon Sampling': Why Using LLMs as Survey Respondents Is Confusing — beenwrekt · 2026-10-29
- Lean Creator Says AI-Powered Formal Verification Could Reshape Software Engineering — bigdata · 2026-10-29
- Mila's open-source Harpia uses drones + AI to survey hundreds of tree crowns in a day — Mila_Quebec · 2026-10-29
- Sakana AI takes 1st place in OpenMath 2026 math hackathon with 14 mathematical families, 6 full resolutions — SakanaAILabs · 2026-10-29
- Helm activation steering: only 10 of 53 pushes held, author shares full data — leolambertini · 2026-10-29
- Ofir Press: Millennium Problems is AI's best benchmark; 70% of open math could fall in a year — OfirPress · 2026-10-29