Token multiplier: 2x means the baseline needs twice the tokens

_katieeverett · x · 2026-09-10

Following prior work, the token multiplier is defined as the ratio of tokens required by a baseline optimizer versus the compared optimizer to reach the same loss at a fixed model size. A value of 2x means the baseline needs twice as many tokens.

Related event: Momentum-Scheduled ADANA Changes Scaling Exponents on the Overtraining Axis(13 posts)→

Original post →

More from Research

Research channel →