Marin's dlwh Explains the Scaling Law Fit: Estimating the Loss Floor Early
dlwh · x · 2026-10-27
In a follow-up to Marin's scaling law thread, dlwh breaks down the simple fit: loss = E + A·compute^(-alpha), where E is the asymptotic loss floor, A the scale, and alpha the slope. The fun part, he says, is estimating that floor while the small models are still far above it — done by training a ladder of models with the same recipe and fitting curves at different training fractions to forecast the full trajectory.
More from Research
- Exponential launches RepoInject and BrowserInject benchmarks for prompt injection robustness — maksym_andr · 2026-10-27
- Redwood: CoT controllability evals are under-elicited; better prompts boost scores 2-3x — dfrsrchtwts · 2026-10-27
- Inverse-designed optical system solves MNIST using BeamZ — S_Conradi · 2026-10-27
- DeCert supervises generative reward models' rationales without reference explanations — Guiquan Sun · 2026-10-27
- VLM harnesses act as compositional generalizers, new blog post shows — a1zhang · 2026-10-27
- Article proposes a Human Genome Project for AI interpretability — ninamiolane · 2026-10-27