Pretraining perplexity alone can't rank base models anymore
stochasticchasm · x · 2026-11-23
stochasticchasm observes that the community is again ranking base models by pretraining perplexity, 'like it's 2023'. He clarifies perplexity isn't irrelevant, but without context like the dataset composition and context length, these numbers alone can't be meaningfully interpreted — base model evaluation needs fuller context.
More from Models
- AI agents play a 3D murder mystery: GPT-6.2 solves it in 3m14s — venturetwins · 2026-11-23
- Users say GPT-5.6 was the last model 'with the sauce' as newer ones feel robotic — thederbiedone · 2026-11-23
- Keeping up with five flagship models: 'Opus 5.5, Fable 5.2, GPT-6.2 Astra... I'm tired' — Angaisb_ · 2026-11-23
- Chinese open-weight models near closed frontier on coding and math, reshaping enterprise choice — ingliguori · 2026-11-23
- Commenter pushes back on perplexity claim: math gains likely came from RL, not pretraining — stochasticchasm · 2026-11-23
- Influencer's overall model ranking puts opus 5.5 on top, weighing cost and speed — MatthewBerman · 2026-11-23