Skaling: Chinchilla's Exponents Meet Kaplan's Coupling
Mathurin Videau, Badr Youbi-Idrissi, David Lopez-Paz, Kartik Ahuja
cs.CL
2026-08-07
Chinchilla treats model size and data as independent and biases at the extremes; Skaling adds one coupling exponent, cuts error 1.5-3x, and profiles with about 10x less compute.
Scaling laws predict loss from model size N and training data D, and they are central to compute allocation. The popular Chinchilla form is additive: L = A/N^alpha + B/D^beta + E. This structure assumes that N and D affect loss independently, so the cross-derivative is exactly zero. At the data-scarce and overtraining extremes, Chinchilla is systematically biased, with errors concentrated at the corners of the (N, D) grid.
Skaling raises the additive sum to an outer exponent k: L = (A/N^alpha + B/D^beta)^k + E. At k equal to 1 it reduces to Chinchilla; for k not equal to 1 it reinstates coupling between N and D (non-zero cross-derivative), like the earlier Kaplan form, but it keeps Chinchilla's independent inner exponents (Kaplan tied the inner terms together). It adds only one parameter. Two desirable properties survive: with k greater than 0 the loss is strictly decreasing in both N and D, and the closed-form compute-optimal allocation (the D/N ratio) is inherited from Chinchilla.
A companion "L-shape" sampling strategy notes that a full grid spends almost all compute in the top-right corner (largest models, longest training), so it sweeps D only for the smallest models and N only for the shortest runs, using about 10x less compute to fit parameters. An additive coupling is not the only option; the appendix compares multiplicative coupling with an additive interaction term and finds the exponentiated multiplicative form fits better. From FAIR at Meta.
| Regime (Farseer, full grid) | Chinchilla MAPE | Skaling MAPE |
| Extrapolation N | 1.48 | 0.47 |
| Extrapolation D | 1.98 | 0.88 |
| Far extrapolation | 2.46 | 2.31 |
On SK-Grid, far-extrapolation MAPE drops from 5.17 to 0.70. On the L-shape grid (about 10x less compute), Skaling still matches or beats full-grid Chinchilla. Overall MAPE drops 1.5 to 3x. The compute-optimal token-to-parameter ratio predicted by Chinchilla drifts away from Skaling's, accumulating to a 100-fold discrepancy at frontier compute; Skaling tracks the true trend more closely.
The practical payoff is predicting large models from small experiments. One extra parameter buys 1.5 to 3x prediction accuracy, and combined with L-shape sampling it extrapolates a full grid using about 10x less profiling compute, which is real savings on a scaling-law sweep. For frontier labs that lock a fixed token-to-parameter ratio (DeepSeek is the cited example), a mis-estimated optimal ratio penalizes performance directly, so the 100-fold ratio drift matters more than the loss-prediction error itself.
It is validated on only two grids (the public Farseer data and the authors' own SK-Grid), so whether it holds across other architectures or data distributions is untested. The extra parameter k could in principle overfit; the authors argue stability via cross-validation variance, but far-extrapolation MAPE on the Farseer full grid is still 2.31 (the highest of Skaling's numbers). Whether the additive or coupled form is physically correct is unsettled; Skaling is a more useful parameterization, not an explanation of the mechanism behind loss scaling.