Pretraining perplexity alone can't rank base models anymore

stochasticchasm · x · 2026-11-23

stochasticchasm observes that the community is again ranking base models by pretraining perplexity, 'like it's 2023'. He clarifies perplexity isn't irrelevant, but without context like the dataset composition and context length, these numbers alone can't be meaningfully interpreted — base model evaluation needs fuller context.

Original post →

More from Models

Models channel →