Commenter pushes back on perplexity claim: math gains likely came from RL, not pretraining
stochasticchasm · x · 2026-11-23
stochasticchasm challenges the framing that upgrading to a model with log-perplexity 0.76 produced groundbreaking math results: such numbers are hard to interpret without knowing the dataset, context length, and other details. He argues the explosive math capability gains were primarily driven by RL rather than pretraining.
More from Models
- AI agents play a 3D murder mystery: GPT-6.2 solves it in 3m14s — venturetwins · 2026-11-23
- Users say GPT-5.6 was the last model 'with the sauce' as newer ones feel robotic — thederbiedone · 2026-11-23
- Pretraining perplexity alone can't rank base models anymore — stochasticchasm · 2026-11-23
- Keeping up with five flagship models: 'Opus 5.5, Fable 5.2, GPT-6.2 Astra... I'm tired' — Angaisb_ · 2026-11-23
- Chinese open-weight models near closed frontier on coding and math, reshaping enterprise choice — ingliguori · 2026-11-23
- Influencer's overall model ranking puts opus 5.5 on top, weighing cost and speed — MatthewBerman · 2026-11-23